Merge pull request instructlab#3147 from instructlab/mergify/bp/relea…

…se-v0.23/pr-3060

fix(RAG): Fix RAG support for taxonomies with compositional skills (backport instructlab#3060)

Merge pull request instructlab#3134 from instructlab/cap-dependencies…

…-0.24

deps: Cap dependencies for 0.24.0 release

fix: remove duplicate log messages in detached SDG (backport instruct…

…lab#3031) (instructlab#3038)

currently, since we add a log file handler and capture the logs from the runnning process when running -dt, there are duplicate log messages. Only add the log file handler if running in attached mode









**Checklist:**

- [ ] **Commit Message Formatting**: Commit titles and messages follow guidelines in the
  [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/#summary).
- [ ] [Changelog](https://github.com/instructlab/instructlab/blob/main/CHANGELOG.md) updated with breaking and/or notable changes for the next minor release.
- [ ] Documentation has been updated, if necessary.
- [ ] Unit tests have been added, if necessary.
- [ ] Functional tests have been added, if necessary.
- [ ] E2E Workflow tests have been added, if necessary.
<hr>This is an automatic backport of pull request instructlab#3031 done by [Mergify](https://mergify.com).


Approved-by: cdoern

Approved-by: nathan-weinberg

deps: cap dependencies for 0.23.0 release (instructlab#3015)

**Checklist:**

- [x] **Commit Message Formatting**: Commit titles and messages follow guidelines in the
  [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/#summary).
- [ ] [Changelog](https://github.com/instructlab/instructlab/blob/main/CHANGELOG.md) updated with breaking and/or notable changes for the next minor release.
- [ ] Documentation has been updated, if necessary.
- [ ] Unit tests have been added, if necessary.
- [ ] Functional tests have been added, if necessary.
- [ ] E2E Workflow tests have been added, if necessary.



Approved-by: alinaryan

Approved-by: nathan-weinberg

Approved-by: booxter

feat: DK-Bench ilab implementation (instructlab#2940)

This commit adds Domain Knowledge bench (DK-Bench) as an evaluation option as part of
`ilab model evaluate`.

This commit adds the cli flags to enable run the
benchmark.









**Checklist:**

- [ ] **Commit Message Formatting**: Commit titles and messages follow guidelines in the
  [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/#summary).
- [ ] [Changelog](https://github.com/instructlab/instructlab/blob/main/CHANGELOG.md) updated with breaking and/or notable changes for the next minor release.
- [ ] Documentation has been updated, if necessary.
- [ ] Unit tests have been added, if necessary.
- [ ] Functional tests have been added, if necessary.
- [ ] E2E Workflow tests have been added, if necessary.



Approved-by: RobotSail

Approved-by: cdoern

Approved-by: alinaryan

Improvement to get_argument (instructlab#2939)

Based on instructlab#2927 (comment)
```
$ python test-test.py
prefix:  --foo
args:  ['--foo', '4']
flag:  True
value:  4

$ python test-test.py
prefix:  --foo
args:  ['--foo', '4', '--foo']
flag:  True
value:  None

$ python test-test.py
prefix:  --foo
args:  ['--foo', '--foo']
flag:  True
value:  None

$ python test-test.py
prefix:  --foo
args:  ['--foo', '4', '--foo=2']
flag:  True
value:  2

$ python test-test.py
prefix:  --foo
args:  ['foo']
flag:  False
value:  None
```

**Checklist:**

- [ ] **Commit Message Formatting**: Commit titles and messages follow guidelines in the
  [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/#summary).
- [ ] [Changelog](https://github.com/instructlab/instructlab/blob/main/CHANGELOG.md) updated with breaking and/or notable changes for the next minor release.
- [ ] Documentation has been updated, if necessary.
- [ ] Unit tests have been added, if necessary.
- [ ] Functional tests have been added, if necessary.
- [ ] E2E Workflow tests have been added, if necessary.



Approved-by: danmcp

Approved-by: nathan-weinberg

feat: support llama-cpp-python v0.3.2 (backport instructlab#2825) (in…

…structlab#2883)

version 0.3.5 of llama-cpp-python has a known issue abetlen/llama-cpp-python#1861 version 0.3.2 has granite 3.0 support and does not have this issue. Bump to this version

this required some additions to how we handle chat exceptions. As of these newer 0.3.z llama-cpp-python versions,
a bad request causes the server to die. This requires us to know the max_ctx_size of the server before passing a completions request so that
we can maintain the behavior of trimming messages until we can respond to one that fits.

in order to do this, the config now contains a `current_max_ctx_size` field that we will update when spinning up a server.
in the case that a user implicitly starts a llama-cpp-python server when calling `ilab model chat`, we set the max_tokens to the
current `max_ctx_size` in the serve config.









**Checklist:**

- [ ] **Commit Message Formatting**: Commit titles and messages follow guidelines in the
  [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/#summary).
- [ ] [Changelog](https://github.com/instructlab/instructlab/blob/main/CHANGELOG.md) updated with breaking and/or notable changes for the next minor release.
- [ ] Documentation has been updated, if necessary.
- [ ] Unit tests have been added, if necessary.
- [ ] Functional tests have been added, if necessary.
- [ ] E2E Workflow tests have been added, if necessary.
<hr>This is an automatic backport of pull request instructlab#2825 done by [Mergify](https://mergify.com).


Approved-by: cdoern

Approved-by: alinaryan

Add legacy template override option to general config (backport instr…

…uctlab#2789) (instructlab#2794)

Adds option to select legacy template in the general section of the config.
Solves instructlab#2790 







**Checklist:**

- [ ] **Commit Message Formatting**: Commit titles and messages follow guidelines in the
  [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/#summary).
- [ ] [Changelog](https://github.com/instructlab/instructlab/blob/main/CHANGELOG.md) updated with breaking and/or notable changes for the next minor release.
- [ ] Documentation has been updated, if necessary.
- [ ] Unit tests have been added, if necessary.
- [ ] Functional tests have been added, if necessary.
- [ ] E2E Workflow tests have been added, if necessary.
<hr>This is an automatic backport of pull request instructlab#2789 done by [Mergify](https://mergify.com).


Approved-by: cdoern

Approved-by: alinaryan

Merge pull request instructlab#2788 from courtneypacheco/cap-deps-rel…

…ease-0.22

deps: Cap dependencies for 0.22 release

fix: don't error out if serve/chat model don't match (backport instru…

…ctlab#2756) (instructlab#2762)

We supply a default model to `ilab model chat`. If this does not match the model being served separately (if there is one), instead of erroring out we should inform the user that we are ignoring what they supplied in favor of what is being served 





**Checklist:**

- [ ] **Commit Message Formatting**: Commit titles and messages follow guidelines in the
  [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/#summary).
- [ ] [Changelog](https://github.com/instructlab/instructlab/blob/main/CHANGELOG.md) updated with breaking and/or notable changes for the next minor release.
- [ ] Documentation has been updated, if necessary.
- [ ] Unit tests have been added, if necessary.
- [ ] Functional tests have been added, if necessary.
- [ ] E2E Workflow tests have been added, if necessary.
<hr>This is an automatic backport of pull request instructlab#2756 done by [Mergify](https://mergify.com).


Approved-by: cdoern

Approved-by: jaideepr97

Approved-by: alinaryan