Add a dedicated OpenAI-compatible LLM adapter#1895
Merged
athul-rs merged 18 commits intoMay 19, 2026
Zipstack:mainZipstack/unstract:mainfrom
jimmyzhuu:codex/openai-compatible-llm-adapterjimmyzhuu/unstract:codex/openai-compatible-llm-adapterCopy head branch name to clipboard
Merged
Add a dedicated OpenAI-compatible LLM adapter#1895athul-rs merged 18 commits intoZipstack:mainZipstack/unstract:mainfrom jimmyzhuu:codex/openai-compatible-llm-adapterjimmyzhuu/unstract:codex/openai-compatible-llm-adapterCopy head branch name to clipboard
athul-rs merged 18 commits into
Zipstack:mainZipstack/unstract:mainfrom
jimmyzhuu:codex/openai-compatible-llm-adapterjimmyzhuu/unstract:codex/openai-compatible-llm-adapterCopy head branch name to clipboard
Conversation
|
|
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughThis PR adds support for OpenAI-compatible LLM providers (e.g., via LiteLLM) by introducing a new adapter type, parameter validation with model normalization, improved usage token handling, and corresponding tests and configuration. ChangesOpenAI-Compatible LLM Adapter Feature
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
for more information, see https://pre-commit.ci
Contributor
|
| Filename | Overview |
|---|---|
| unstract/sdk1/src/unstract/sdk1/adapters/base1.py | Adds OpenAICompatibleLLMParameters; validate() normalises blank api_key to None but model_dump() includes it, risking credential leak or auth failure for keyless endpoints. |
| unstract/sdk1/src/unstract/sdk1/adapters/llm1/openai_compatible.py | New adapter file; minimal and correct — ID, metadata, provider, icon, and adapter type all look right. |
| unstract/sdk1/src/unstract/sdk1/adapters/llm1/static/custom_openai.json | New JSON schema; model is intentionally excluded from required (relying on Python-layer validation), consistent with the validate_model guard. |
| unstract/sdk1/tests/test_openai_compatible_adapter.py | New test file covering registration, model normalisation, schema loading, and usage recording; magic module stub is documented inline. |
| unstract/sdk1/src/unstract/sdk1/adapters/llm1/init.py | Adds import and all export for OpenAICompatibleLLMAdapter; straightforward. |
Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 1
unstract/sdk1/src/unstract/sdk1/adapters/base1.py:356-363
**Blank `api_key` becomes `None` in `model_dump()` output**
`validate()` normalises blank strings to `None`, but Pydantic's `model_dump()` includes `None` fields by default. The resulting dict is spread directly into `litellm.completion()`. For the `custom_openai` provider, LiteLLM initialises an OpenAI client; receiving a `None` key causes the client to fall back to any service key present in the process environment — potentially forwarding it to a third-party endpoint — or raises an auth error when no environment key exists, breaking keyless-endpoint configurations that the schema description explicitly supports.
Fix: pop the key field from the `model_dump()` result when it is `None`, matching the pattern in `_resolve_bedrock_aws_credentials` which explicitly drops optional credential fields.
Reviews (11): Last reviewed commit: "Merge branch 'main' into codex/openai-co..." | Re-trigger Greptile
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
unstract/sdk1/src/unstract/sdk1/llm.py (1)
542-557: Avoid unconditional token estimation when usage already includes prompt tokens.This currently computes
token_counter()even when provider usage already has prompt tokens, which can create repeated warnings/noise for unmapped models without improving recorded usage.♻️ Proposed refinement
- try: - prompt_tokens = token_counter(model=model, messages=messages) - except Exception as e: - prompt_tokens = 0 - logger.warning( - "[sdk1][LLM][%s][%s] Failed to estimate prompt tokens: %s", - model, - llm_api, - e, - ) usage_data: Mapping[str, int] = usage or {} + prompt_tokens = usage_data.get("prompt_tokens") + if prompt_tokens is None: + try: + prompt_tokens = token_counter(model=model, messages=messages) + except Exception as e: + prompt_tokens = 0 + logger.warning( + "[sdk1][LLM][%s][%s] Failed to estimate prompt tokens: %s", + model, + llm_api, + e, + ) all_tokens = TokenCounterCompat( - prompt_tokens=usage_data.get("prompt_tokens", 0), + prompt_tokens=usage_data.get("prompt_tokens", prompt_tokens or 0), completion_tokens=usage_data.get("completion_tokens", 0), total_tokens=usage_data.get("total_tokens", 0), )🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@unstract/sdk1/src/unstract/sdk1/llm.py` around lines 542 - 557, The code unconditionally calls token_counter(model, messages) even when usage already contains prompt token counts; change the logic in the block around token_counter and TokenCounterCompat so you first check usage (usage_data = usage or {}) and if usage_data.get("prompt_tokens") is present use that value for prompt_tokens instead of calling token_counter; only call token_counter(model, messages) inside the try/except when usage_data lacks prompt_tokens, preserving the existing exception handling and the logger.warning path, and then construct TokenCounterCompat using the values from usage_data (falling back to the estimated prompt_tokens when used).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@unstract/sdk1/src/unstract/sdk1/adapters/llm1/static/custom_openai.json`:
- Around line 15-20: The schema for the "api_key" property currently only allows
a string which fails when runtime metadata contains null; update the "api_key"
entry in the JSON schema (the "api_key" property in custom_openai.json) to
permit null values by changing its type to accept both string and null (or add a
nullable:true equivalent) so stored configs with null pass validation and
editing flows.
---
Nitpick comments:
In `@unstract/sdk1/src/unstract/sdk1/llm.py`:
- Around line 542-557: The code unconditionally calls token_counter(model,
messages) even when usage already contains prompt token counts; change the logic
in the block around token_counter and TokenCounterCompat so you first check
usage (usage_data = usage or {}) and if usage_data.get("prompt_tokens") is
present use that value for prompt_tokens instead of calling token_counter; only
call token_counter(model, messages) inside the try/except when usage_data lacks
prompt_tokens, preserving the existing exception handling and the logger.warning
path, and then construct TokenCounterCompat using the values from usage_data
(falling back to the estimated prompt_tokens when used).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: bf841637-54b7-4802-9156-7f56e899ca54
📒 Files selected for processing (7)
README.mdunstract/sdk1/src/unstract/sdk1/adapters/base1.pyunstract/sdk1/src/unstract/sdk1/adapters/llm1/__init__.pyunstract/sdk1/src/unstract/sdk1/adapters/llm1/openai_compatible.pyunstract/sdk1/src/unstract/sdk1/adapters/llm1/static/custom_openai.jsonunstract/sdk1/src/unstract/sdk1/llm.pyunstract/sdk1/tests/test_openai_compatible_adapter.py
Contributor
Author
|
Addressed the review follow-ups.
Validation re-run:
|
Contributor
Author
|
Gentle follow-up on this PR in case it slipped through the queue. When someone has bandwidth, I would really appreciate a review. Happy to make any follow-up changes quickly. Thanks! |
ritwik-g
reviewed
Apr 23, 2026
jaseemjaskp
reviewed
Apr 23, 2026
jaseemjaskp
left a comment
Contributor
There was a problem hiding this comment.
PR Review Toolkit — consolidated findings
Automated review aggregating six specialist agents (code-reviewer, code-simplifier, silent-failure-hunter, type-design-analyzer, pr-test-analyzer, comment-analyzer). No blocking defects; the adapter follows existing sibling-adapter conventions and the scope is appropriately narrow.
High-signal items worth addressing before merge
llm.py:556— redundant/confusing fallback:prompt_tokensis already resolved at line 543 with an explicit estimation branch, so theusage_data.get("prompt_tokens", prompt_tokens or 0)expression double-handles the default and, worse, silently coerces an explicitNonefromusage_datato 0 without logging.llm.py:547— broadexcept Exceptioncombined with a warning that says "failed to estimate" but not "recording 0 tokens" means billing/audit can silently under-report. Narrow the exception and either uselogger.exceptionor rewrite the message to name the consequence.base1.py:232/ schemaapi_base— the Pydantic type is plainstr; URL shape lives only in the JSON schema, so direct construction accepts garbage. ConsiderHttpUrl/ afield_validator.base1.py:239— prefix logic is only invoked by thevalidateclassmethod.AzureOpenAILLMParametersuses@model_validator(mode="before")which cannot be bypassed; mirroring that tightens the invariant.custom_openai.json—api_baseis listed as required yet ships with a placeholder default URL, which lets users save an unchanged form and hit 404s at request time instead of validation errors. Vendor-specific examples (ERNIE-4.0-8K (Baidu Qianfan),qianfan.baidubce.com) are prone to rot and should be generic.test_openai_compatible_adapter.py— the "tolerates unmapped models" test only assertspush_usage_data.assert_called_once(); it does not verifyprompt_tokens=0was actually pushed, so a regression silently pushingNoneor crashing insideTokenCounterCompatwould still pass.
Suggested follow-ups (non-blocking)
- Add tests for:
api_basemissing (Pydantic ValidationError),usage=None,usage["prompt_tokens"]=None, and the success branch oftoken_counter. - Drop the unused
@lru_cacheon_load_llm_moduleand the dead_load_llm_class. - Replace the tautological
get_description()/ metadata description with user-facing copy that distinguishes this adapter fromOpenAILLMAdapter.
Inline comments below flag each item at its exact line with a concrete fix suggestion.
|
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (5)
unstract/sdk1/tests/test_openai_compatible_adapter.py (3)
102-192: LGTM — good coverage of the three_record_usagebranches.Tests exercise: (a) provider-supplied
prompt_tokensbypassestoken_counter, (b)token_counterraising falls back to 0 with a warning, and (c)prompt_tokens=Nonetriggers estimation. The use of__new__to bypass__init__and the targetedpatch.object(llm_module, ...)are appropriate here. Nice to see the warning-message assertion on line 162 pinning the audit-visible text.One small suggestion: assert
mock_warningis called exactly once and withmodel/llm_apisubstituted into the format string, to catch regressions that accidentally change the log signature.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@unstract/sdk1/tests/test_openai_compatible_adapter.py` around lines 102 - 192, Update the test_record_usage_tolerates_unmapped_models_without_prompt_tokens test to assert the warning logger was called exactly once and that the warning message includes the model ("custom_openai/gateway-model") and llm_api ("complete") values; locate the test function and the mock_warning (patched via patch.object(llm_module.logger, "warning")) and after calling llm._record_usage add assertions that mock_warning.assert_called_once() and that the call_args contains both the model and llm_api strings in the formatted warning message to catch signature regressions.
18-31: Minor:lru_cachearoundimport_moduleis largely redundant.
sys.modulesalready caches modules after first import, so thelru_cache(maxsize=1)only saves thepatch.dictcontext-manager overhead. Leaving it is harmless, but on second and subsequent calls themagicstub will not be re-installed (because the cached branch returns early), so any test that newly triggersimport magicafter the first call would see the real module. Not a problem today, but worth documenting with a short comment to prevent surprise.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@unstract/sdk1/tests/test_openai_compatible_adapter.py` around lines 18 - 31, The `@lru_cache` on _load_llm_module() prevents the patch.dict stub for "magic" from being re-applied on subsequent calls, which can lead to surprising behavior if tests later import the real magic module; either remove the `@lru_cache` decorator or (preferred) keep it but add a brief comment inside _load_llm_module explaining that sys.modules already caches imports and that the cached result means the "magic" stub will not be re-installed on later calls so tests should call this once or manage stubbing themselves — reference the _load_llm_module function and the patch.dict usage when adding the comment.
34-35: Dead helper:_load_llm_classis never called.Every test uses
_load_llm_module().LLMdirectly (e.g., lines 104, 135, 167). Consider removing_load_llm_classor using it in place of the inlinellm_module.LLMlookups for consistency.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@unstract/sdk1/tests/test_openai_compatible_adapter.py` around lines 34 - 35, The helper function _load_llm_class is unused (dead code); either remove _load_llm_class entirely or replace direct usages of _load_llm_module().LLM in tests (e.g., the inline lookups at places that call llm_module.LLM) with calls to _load_llm_class() for consistency. Locate the definition of _load_llm_class and the test files referencing _load_llm_module().LLM and either delete the unused _load_llm_class function or update those tests to call _load_llm_class() instead, ensuring imports and type annotations still match.unstract/sdk1/src/unstract/sdk1/adapters/base1.py (1)
234-242: Minor:validate()mutates the caller's dict.Lines 236 and 241 write back into
adapter_metadata(same pattern asOpenAILLMParameters, but unlikeVertexAILLMParameterswhich copies first via{**adapter_metadata}). GivenLLM.complete()callsself.adapter.validate({**self.kwargs, **kwargs})(a fresh dict), there's no current bug — but if a future caller passes a long-lived dict, theapi_keywould be mutated in place and the model prefix would double-rewrite on a second call (thestartswith("custom_openai/")guard invalidate_modelmitigates the latter).Optional: copy first for defensive hygiene, matching the
VertexAILLMParameters.validate()style.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@unstract/sdk1/src/unstract/sdk1/adapters/base1.py` around lines 234 - 242, The validate() method currently mutates the incoming adapter_metadata dict (it writes to adapter_metadata["model"] and adapter_metadata["api_key"]); to avoid in-place side effects make a shallow copy first (e.g., metadata = {**adapter_metadata}) and perform all modifications against that copy before passing it to OpenAICompatibleLLMParameters.validate_model and constructing OpenAICompatibleLLMParameters(**metadata).model_dump(); keep references to the same symbols (validate, OpenAICompatibleLLMParameters.validate_model, OpenAICompatibleLLMParameters, adapter_metadata) so the change is local and preserves existing behavior while preventing caller dict mutation.unstract/sdk1/src/unstract/sdk1/llm.py (1)
557-557: Minor:prompt_tokens or 0also zeroes out legitimate0from provider.If a provider ever reports
usage.prompt_tokens == 0(unusual, but possible for zero-content requests or certain gateways), the truthiness check collapses it the same asNone. Given theprompt_tokens is Nonebranch already assigns an int (or 0 on exception), thisor 0is only needed to satisfy the type checker. A more precise form:Proposed tweak
- all_tokens = TokenCounterCompat( - prompt_tokens=prompt_tokens or 0, + all_tokens = TokenCounterCompat( + prompt_tokens=prompt_tokens if prompt_tokens is not None else 0, completion_tokens=usage_data.get("completion_tokens", 0), total_tokens=usage_data.get("total_tokens", 0), )Low-impact since 0 vs None ends up the same in the audit row, but semantically cleaner.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@unstract/sdk1/src/unstract/sdk1/llm.py` at line 557, Replace the truthiness fallback that zeroes out legitimate zero values: instead of using "prompt_tokens=prompt_tokens or 0" keep the explicit None-check so only None becomes 0 (e.g., use a conditional expression that assigns prompt_tokens if prompt_tokens is not None else 0). Locate the occurrence of the "prompt_tokens=prompt_tokens or 0" assignment and change it to an explicit None-check for the variable prompt_tokens so a reported 0 remains 0.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@unstract/sdk1/src/unstract/sdk1/llm.py`:
- Around line 543-560: The current catch in the prompt token estimation around
token_counter (used when building TokenCounterCompat) silently sets
prompt_tokens=0; update this to (1) narrow the except to only expected errors
from the estimator (e.g., KeyError/ValueError and litellm-specific exceptions
raised by token_counter) so unexpected errors still propagate, and (2) add a
sentinel field to the usage payload (e.g., prompt_tokens_source or
estimation_failed) before calling Audit().push_usage_data to mark that prompt
tokens were estimated/failed, and/or increment an ops metric/counter when the
fallback path occurs; reference the token_counter call, TokenCounterCompat
construction, Audit().push_usage_data, and the existing logger to emit a clear
warning and metric.
---
Nitpick comments:
In `@unstract/sdk1/src/unstract/sdk1/adapters/base1.py`:
- Around line 234-242: The validate() method currently mutates the incoming
adapter_metadata dict (it writes to adapter_metadata["model"] and
adapter_metadata["api_key"]); to avoid in-place side effects make a shallow copy
first (e.g., metadata = {**adapter_metadata}) and perform all modifications
against that copy before passing it to
OpenAICompatibleLLMParameters.validate_model and constructing
OpenAICompatibleLLMParameters(**metadata).model_dump(); keep references to the
same symbols (validate, OpenAICompatibleLLMParameters.validate_model,
OpenAICompatibleLLMParameters, adapter_metadata) so the change is local and
preserves existing behavior while preventing caller dict mutation.
In `@unstract/sdk1/src/unstract/sdk1/llm.py`:
- Line 557: Replace the truthiness fallback that zeroes out legitimate zero
values: instead of using "prompt_tokens=prompt_tokens or 0" keep the explicit
None-check so only None becomes 0 (e.g., use a conditional expression that
assigns prompt_tokens if prompt_tokens is not None else 0). Locate the
occurrence of the "prompt_tokens=prompt_tokens or 0" assignment and change it to
an explicit None-check for the variable prompt_tokens so a reported 0 remains 0.
In `@unstract/sdk1/tests/test_openai_compatible_adapter.py`:
- Around line 102-192: Update the
test_record_usage_tolerates_unmapped_models_without_prompt_tokens test to assert
the warning logger was called exactly once and that the warning message includes
the model ("custom_openai/gateway-model") and llm_api ("complete") values;
locate the test function and the mock_warning (patched via
patch.object(llm_module.logger, "warning")) and after calling llm._record_usage
add assertions that mock_warning.assert_called_once() and that the call_args
contains both the model and llm_api strings in the formatted warning message to
catch signature regressions.
- Around line 18-31: The `@lru_cache` on _load_llm_module() prevents the
patch.dict stub for "magic" from being re-applied on subsequent calls, which can
lead to surprising behavior if tests later import the real magic module; either
remove the `@lru_cache` decorator or (preferred) keep it but add a brief comment
inside _load_llm_module explaining that sys.modules already caches imports and
that the cached result means the "magic" stub will not be re-installed on later
calls so tests should call this once or manage stubbing themselves — reference
the _load_llm_module function and the patch.dict usage when adding the comment.
- Around line 34-35: The helper function _load_llm_class is unused (dead code);
either remove _load_llm_class entirely or replace direct usages of
_load_llm_module().LLM in tests (e.g., the inline lookups at places that call
llm_module.LLM) with calls to _load_llm_class() for consistency. Locate the
definition of _load_llm_class and the test files referencing
_load_llm_module().LLM and either delete the unused _load_llm_class function or
update those tests to call _load_llm_class() instead, ensuring imports and type
annotations still match.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: bb92694d-b745-40e9-8bd9-0bc1fa3628b1
⛔ Files ignored due to path filters (1)
frontend/public/icons/adapter-icons/OpenAICompatible.pngis excluded by!**/*.png
📒 Files selected for processing (5)
unstract/sdk1/src/unstract/sdk1/adapters/base1.pyunstract/sdk1/src/unstract/sdk1/adapters/llm1/openai_compatible.pyunstract/sdk1/src/unstract/sdk1/adapters/llm1/static/custom_openai.jsonunstract/sdk1/src/unstract/sdk1/llm.pyunstract/sdk1/tests/test_openai_compatible_adapter.py
✅ Files skipped from review due to trivial changes (2)
- unstract/sdk1/src/unstract/sdk1/adapters/llm1/openai_compatible.py
- unstract/sdk1/src/unstract/sdk1/adapters/llm1/static/custom_openai.json
Contributor
|
Hi @jimmyzhuu - sorry we let this sit so long without a review, that's on us. The change looks reasonable in scope and the validation looks solid. If you're still interested, we'd be happy to have you reopen it and we'll get a maintainer on it this week. Either way, thanks for the careful work and the patience. |
Contributor
Author
|
Absolutely — I’d be happy to work on this. Thanks for the suggestion! |
Contributor
Author
@athul-rs Updated the PR description. |
Resolve conflict in unstract/sdk1/src/unstract/sdk1/llm.py by taking main's version of _record_usage. This PR's prompt_tokens fallback is subsumed by main's existing litellm.token_counter() fallback, and the TokenCounterCompat / Audit().push_usage_data() shape this PR retained is dead post-Lookups-V2 refactor (usage now flows through self._pending_usage). Update the three _record_usage tests in test_openai_compatible_adapter.py to assert on the _pending_usage carrier rather than the removed Audit pattern. Also fix a latent sys.modules rollback in _load_llm_module that broke litellm attribute resolution after the patch.dict context exited. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
validate_model previously produced "custom_openai/" for an empty model, surfacing as a confusing LiteLLM error at call time. Match the existing GeminiLLMParameters.validate_model pattern: strip whitespace, raise ValueError on empty input. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ritwik-g
reviewed
May 11, 2026
Addresses Ritwik's review feedback. The new BaseAdapter.SCHEMA_PATH
class variable and the conditional branch in get_json_schema() are
unnecessary: OpenAICompatibleLLMAdapter.get_provider() returns
"custom_openai", and the default path resolution already builds
…/llm1/static/{get_provider()}.json. Renaming the schema file lets
the default lookup find it and keeps the base class untouched, which
is the convention every other adapter follows.
- Rename openai_compatible.json -> custom_openai.json
- Drop SCHEMA_PATH class var and the if-None branch from BaseAdapter
- Drop SCHEMA_PATH override (and unused os/ClassVar imports) from
OpenAICompatibleLLMAdapter
- Update test_openai_compatible_schema_is_loadable to read schema via
get_json_schema() instead of touching SCHEMA_PATH directly
Contributor
|
Hi @jimmyzhuu — thanks for the PR! I pushed a small follow-up commit (27215c2) addressing @ritwik-g's |
ritwik-g
reviewed
May 13, 2026
ritwik-g
approved these changes
May 13, 2026
pk-zipstack
approved these changes
May 19, 2026
|
harini-venkataraman
pushed a commit
that referenced
this pull request
May 21, 2026
* Add OpenAI-compatible LLM adapter * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review feedback for custom OpenAI adapter * Fix import formatting after rebase * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address follow-up review comments for OpenAI-compatible adapter * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Refine OpenAI compatible adapter schema naming * Reject empty model string in OpenAICompatibleLLMParameters validate_model previously produced "custom_openai/" for an empty model, surfacing as a confusing LiteLLM error at call time. Match the existing GeminiLLMParameters.validate_model pattern: strip whitespace, raise ValueError on empty input. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Revert SCHEMA_PATH plumbing; rename schema to custom_openai.json Addresses Ritwik's review feedback. The new BaseAdapter.SCHEMA_PATH class variable and the conditional branch in get_json_schema() are unnecessary: OpenAICompatibleLLMAdapter.get_provider() returns "custom_openai", and the default path resolution already builds …/llm1/static/{get_provider()}.json. Renaming the schema file lets the default lookup find it and keeps the base class untouched, which is the convention every other adapter follows. - Rename openai_compatible.json -> custom_openai.json - Drop SCHEMA_PATH class var and the if-None branch from BaseAdapter - Drop SCHEMA_PATH override (and unused os/ClassVar imports) from OpenAICompatibleLLMAdapter - Update test_openai_compatible_schema_is_loadable to read schema via get_json_schema() instead of touching SCHEMA_PATH directly --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Hari John Kuriakose <hari@zipstack.com> Co-authored-by: Chandrasekharan M <chandrasekharan@zipstack.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Athul <athul@zipstack.com> Co-authored-by: Athul <89829560+athul-rs@users.noreply.github.com> Co-authored-by: vishnuszipstack <117254672+vishnuszipstack@users.noreply.github.com>
5 tasks
harini-venkataraman
added a commit
that referenced
this pull request
Jun 29, 2026
) * [MISC] Decommission prompt-service, old tools, and SDK1 prompt module (Phase 5) Remove prompt-service source, Dockerfiles, and docker-compose entries. Remove tools/classifier, tools/structure, tools/text_extractor directories. Remove SDK1 prompt.py module and its tests. Clean up PROMPT_HOST/PROMPT_PORT from backend settings, sample envs, docker configs, and CI workflows. Remove prompt-service from uv-lock scripts and production build workflow. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * [MISC] Remove prompt-service from tox.ini env_list The prompt-service directory was deleted in the prior commit but tox.ini still referenced it, which would break CI test runs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * UN-2888 [FIX] Add hook for setting default triad for invited users (#1877) * [FIX] Add hook for setting default adapters for invited users Add setup_default_adapters_for_user() hook to AuthenticationService and call it from set_user_organization() when an invited user joins an existing organization. This allows the cloud plugin to set up default triad adapters (LLM, embedding, vector DB, x2text) for invited users, fixing silent failures in API deployment creation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Update backend/account_v2/authentication_controller.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> Signed-off-by: Praveen Kumar <praveen@zipstack.com> * [FIX] Improve log message for setup_default_adapters_for_user Address review comment: log user email and explain that default adapters will not be set when the method is not implemented. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * [MISC] Rename Default Triad to Default LLM Profile in UI Update display label from "Default Triad" to "Default LLM Profile" in the page heading and side navigation menu. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Signed-off-by: Praveen Kumar <praveen@zipstack.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> Co-authored-by: Deepak K <89829542+Deepak-Kesavan@users.noreply.github.com> * UN-3465 [FIX] Wrap set_user_organization in transaction.atomic (#1954) * [FIX] Wrap set_user_organization in transaction.atomic The new-org branch creates the org row, then calls frictionless onboarding and the initial platform key. Failures mid-flow leave an orphan org with no adapters or key, and subsequent logins skip onboarding entirely (gated on new_organization). Atomic ensures the org rolls back on any failure so retries get a clean fresh-org path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [MISC] Worktree skill — use --no-track to prevent accidental main pushes Without --no-track, a later `git push -u origin <branch>` can be reported by the server as also fast-forwarding main, landing commits on main. * [FIX] Use logger.exception in authorization_callback Preserves the traceback when the OAuth callback hits the safety-net catch. Behaviour unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Athul <89829560+athul-rs@users.noreply.github.com> Co-authored-by: vishnuszipstack <117254672+vishnuszipstack@users.noreply.github.com> * UN-3386 [FEAT] Add Prompt Studio HITL change indicator plugin slot (#1930) * UN-3386 [FEAT] Add Prompt Studio HITL change indicator plugin slot Wires up the host-side hooks for the prompt-change-indicator plugin (implementation lives in unstract-cloud): a dynamic-import slot in the prompt card Header for the indicator button, and a route at :orgName/review/readonly/:documentId for the read-only audit view. Both gates fall through gracefully when the plugin is absent (OSS). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * UN-3386 [FIX] Warn when ReadOnlyReviewPage loads without ReviewLayout Addresses review feedback: the readonly route nests inside ReviewLayout (manual-review plugin), so a deployment that ships prompt-change-indicator without manual-review would silently fail to register the route. Log a console.warn in that case to make the misconfiguration discoverable. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * UN-3386 [FIX] Surface real plugin import errors in route loader Bare catch in the prompt-change-indicator dynamic import was swallowing syntax/runtime errors in the plugin file alongside the expected "plugin missing in OSS" case. Detect the missing-module messages explicitly and console.error anything else so a broken cloud plugin no longer disables the readonly route silently. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Add a dedicated OpenAI-compatible LLM adapter (#1895) * Add OpenAI-compatible LLM adapter * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review feedback for custom OpenAI adapter * Fix import formatting after rebase * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address follow-up review comments for OpenAI-compatible adapter * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Refine OpenAI compatible adapter schema naming * Reject empty model string in OpenAICompatibleLLMParameters validate_model previously produced "custom_openai/" for an empty model, surfacing as a confusing LiteLLM error at call time. Match the existing GeminiLLMParameters.validate_model pattern: strip whitespace, raise ValueError on empty input. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Revert SCHEMA_PATH plumbing; rename schema to custom_openai.json Addresses Ritwik's review feedback. The new BaseAdapter.SCHEMA_PATH class variable and the conditional branch in get_json_schema() are unnecessary: OpenAICompatibleLLMAdapter.get_provider() returns "custom_openai", and the default path resolution already builds …/llm1/static/{get_provider()}.json. Renaming the schema file lets the default lookup find it and keeps the base class untouched, which is the convention every other adapter follows. - Rename openai_compatible.json -> custom_openai.json - Drop SCHEMA_PATH class var and the if-None branch from BaseAdapter - Drop SCHEMA_PATH override (and unused os/ClassVar imports) from OpenAICompatibleLLMAdapter - Update test_openai_compatible_schema_is_loadable to read schema via get_json_schema() instead of touching SCHEMA_PATH directly --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Hari John Kuriakose <hari@zipstack.com> Co-authored-by: Chandrasekharan M <chandrasekharan@zipstack.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Athul <athul@zipstack.com> Co-authored-by: Athul <89829560+athul-rs@users.noreply.github.com> Co-authored-by: vishnuszipstack <117254672+vishnuszipstack@users.noreply.github.com> * ReverseMerge: V0.163.4 hotfix (#1980) * [HOTFIX] Use importlib.util.find_spec for pluggable worker discovery (#1918) * [FIX] Use importlib.util.find_spec for pluggable worker discovery _verify_pluggable_worker_exists() previously checked for the literal file `pluggable_worker/<name>/worker.py` on disk, which breaks when the plugin has been compiled to a .so (Nuitka, Cython, or any C extension) — the module is perfectly importable but the pre-check rejects it because only the .py extension is considered. Replace the filesystem check with importlib.util.find_spec(), which is Python's standard way to ask "is this module resolvable by the import system?". It honors every registered finder — source .py, compiled .so, bytecode .pyc, namespace packages, zipimports — so the function now matches what its docstring claims: verifying the module can be loaded, not that a specific file extension is present. Behavior is preserved for existing deployments: - Images with no `pluggable_worker/<name>/` subpackage → find_spec raises ModuleNotFoundError (ImportError subclass) → returns False. - Images with source .py → find_spec resolves the .py → returns True. - Images with compiled .so → find_spec resolves the .so → returns True. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [FIX] Handle ValueError from find_spec in pluggable worker verification Greptile-flagged edge case: importlib.util.find_spec() can raise ValueError (not just ImportError) when sys.modules has a partially initialised module entry with __spec__ = None from a prior failed import. Broaden the except to catch both. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [FIX] Resolve api-deployment worker directory from enum import path worker.py:452 did worker_type.value.replace("-", "_") to derive the on-disk dir name. All WorkerType enum values already use underscores, so the replace was a no-op; for API_DEPLOYMENT whose dir is "api-deployment" (hyphen), it resolved to "api_deployment" and the os.path.exists() check failed. Boot then logged a spurious "❌ Worker directory not found: /app/api_deployment" at ERROR level. The task registration path (builder + celery autodiscover via to_import_path) is unaffected, so this was purely log noise — but noise at ERROR level that masks real failures in log scans. Fix: derive the directory from the authoritative to_import_path() which already handles the hyphen case (api_deployment -> api-deployment). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [HOTFIX] Add IAM Role / Instance Profile auth mode to AWS Bedrock adapter (#1944) * [FEAT] Allow Bedrock to fall through to boto3's default credential chain Match the S3/MinIO connector pattern: when AWS access keys are left blank on the Bedrock LLM and embedding adapter forms, drop them from the kwargs dict so boto3's default credential chain handles authentication. This unlocks IAM role / instance profile / IRSA / AWS Profile scenarios on hosts that already have ambient AWS credentials (e.g. EKS workers with IRSA, EC2 with an instance profile). - llm1/static/bedrock.json: clarify access-key descriptions to mention IRSA and instance profile (already non-required at v0.163.2 base). - embedding1/static/bedrock.json: drop aws_access_key_id and aws_secret_access_key from top-level required; same description fix; expose aws_profile_name for parity with the LLM form. - base1.py: AWSBedrockLLMParameters and AWSBedrockEmbeddingParameters now strip empty access-key values from the validated kwargs before returning, so empty strings don't override boto3's default chain. AWSBedrockEmbeddingParameters fields gain explicit None defaults and an aws_profile_name field. Backward-compatible: existing adapters with access keys filled in continue to work unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [FEAT] Add Authentication Type selector to Bedrock adapter form Add an explicit `auth_type` selector with two options, making the auth choice clear to users: - "Access Keys" (default): existing flow, keys required - "IAM Role / Instance Profile (on-prem AWS only)": no fields; relies on boto3's default credential chain (IRSA on EKS, task role on ECS, instance profile on EC2). Description on the selector explicitly notes this option is only for AWS-hosted Unstract deployments. The form-only auth_type field is stripped before LiteLLM validation in both AWSBedrockLLMParameters.validate() and AWSBedrockEmbeddingParameters. validate(). Empty access keys continue to be stripped so boto3 falls through to the default chain even when the access_keys arm is selected without values (matches the S3/MinIO connector pattern). Backward-compatible: legacy adapters without auth_type behave as "Access Keys" mode (the default), and existing keys are forwarded unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [REVIEW] Address Bedrock auth_type review feedback Fixes the P0/P1 issues raised by greptile-apps and jaseemjaskp on PR #1944. Behaviour fixes: - Stale-key leak in IAM Role mode: switching an existing adapter from Access Keys to IAM Role would carry truthy stored access keys through the strip-empty-only loop, so boto3 silently authenticated with the old long-lived credentials instead of falling through to the host's IRSA / instance-profile identity. Both LLM and embedding paths were affected. - Silent acceptance of unknown auth_type: a typo (e.g. "access_key") or a malformed payload from a non-UI client passed through the dict comprehension untouched, with no enum guard. - Cross-field validation gap: explicit Access Keys mode with blank or whitespace-only values silently fell through to the default credential chain instead of surfacing the misconfiguration. Implementation: - Add a module-level _resolve_bedrock_aws_credentials helper used by both AWSBedrockLLMParameters.validate() and AWSBedrock EmbeddingParameters.validate(), so the auth-type contract is expressed once. - Validates auth_type against an allowlist (None | "access_keys" | "iam_role"); raises ValueError on anything else. - iam_role: unconditionally drops aws_access_key_id and aws_secret_access_key. - access_keys (explicit): requires non-blank values; raises ValueError if either is empty or whitespace-only. - Legacy (auth_type absent): retains the lenient strip behaviour so pre-PR adapter configurations continue to deserialise unchanged. - Restore aws_region_name as required (no `= None` default) on AWSBedrockEmbeddingParameters; only credentials may legitimately be absent. - Drop the orphan aws_profile_name field from embedding1/static/bedrock.json: it was added for parity with the LLM form but lives outside the auth_type oneOf and contradicts the selector's "no further input" semantics. The LLM form already had aws_profile_name pre-PR and is left alone for backwards compatibility. Tests: - New tests/test_bedrock_adapter.py covers 15 cases across LLM and embedding adapters: legacy-no-auth-type, explicit access_keys with valid/blank/whitespace keys, iam_role with stale/no keys, unknown auth_type rejection, cross-field validation, and preservation of unrelated params (model_id, aws_profile_name, region, thinking). Skipped (P2 nice-to-have): - Comment-scope clarification, MinIO reference rewording, validate-mutates-caller'\''s-dict, and the LLM form description nit about aws_profile_name visibility. These don'\''t change behaviour and can be addressed in a follow-up. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [HOTFIX] Bump litellm to 1.83.10 from PyPI to clear CVE-2026-42208 (#1976) Hotfix for cloud v0.159.3 (OSS v0.163.4). Customer scanner flagged litellm 1.82.3 for CVE-2026-42208 (SQL injection in litellm proxy auth path, affects 1.81.16-1.83.6). We do not use litellm.proxy, but vulnerability scanners flag the installed package regardless of which code path is reachable. Bump to 1.83.10 — the exact version recommended by the upstream advisory (v1.83.10-stable) and the smallest jump that clears the CVE range while keeping python-dotenv==1.0.1 compatible (1.83.14 would force bumping python-dotenv across 7+ pyproject.toml files). Only tiktoken needed to move 0.9 -> 0.12 to satisfy litellm's pin. Switch source back to PyPI now that the PyPI quarantine is over, reversing the temporary fork in #1873. Cohere embed timeout patch: verified that litellm/llms/cohere/embed/handler.py is byte-identical between v1.82.3, v1.83.10-stable, and v1.83.14-stable (the timeout-not-forwarded bug fixed in #1848 is still present upstream — BerriAI/litellm#14635 remains OPEN). Version guard bumped 1.82.3 -> 1.83.10; 6/6 patch tests pass on the new version, confirming the monkey-patch still binds correctly. Other cleanup from #1873: - Drop git apt-install from worker-unified and tool Dockerfiles (no git-sourced deps remain in any uv.lock) - Bump tool versions: structure 0.0.100 -> 0.0.101, classifier 0.0.79 -> 0.0.80, text_extractor 0.0.75 -> 0.0.76 Note on root uv.lock churn: the v0.163.4 root uv.lock had a pre-existing corruption (banks v2.4.1 entry pointing at banks-2.2.0 wheel) that blocked incremental resolution. Regenerated from scratch. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [FIX] Align cohere patch docstring with version-guard semantics Reviewer flagged that the docstring claimed the patch is "confirmed in every release between 1.82.3 and 1.83.14-stable", but the guard at _PATCHED_LITELLM_VERSION activates only on the exact pinned version. A future maintainer reading the old text could reasonably expect bumping to e.g. 1.83.11 to keep the fix active; in reality it silently turns off. Rewritten to reference _PATCHED_LITELLM_VERSION as the single source of truth and to drop the rot-prone "as of 2026-05-20" calendar date. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Chandrasekharan M <117059509+chandrasekharan-zipstack@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * UN-3476 [FIX] Revert atomic wrap on set_user_organization (#1977) The atomic wrap from #1954 uncommits the new org row when frictionless_onboarding HTTP-calls the LLMW portal mid-transaction. The portal runs on a separate DB session and under READ COMMITTED cannot see the uncommitted row, so the call returns 400 and the caller silently persists an adapter with an empty unstract_key. Every new signup since 2026-05-19 09:47 UTC ships a broken free-trial X2Text adapter (401 on first OCR). Hotfix only — Phase 2 (UN-3476) restructures the function so the atomic guarantee is reapplied around just the pure-DB writes, with HTTP and non-DB side effects moved outside the transaction. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Restore text_extractor tool removed in Phase 5 decommission The Phase 5 decommission commit removed classifier, structure, text_extractor, and prompt-service. However, text_extractor is still in active use by customers. This surgically restores only the text_extractor tool while keeping the other decommissions in place. - Restore tools/text_extractor/ directory (14 files from origin/main) - Add tool-text_extractor back to docker-compose.build.yaml - Add tool-text-extractor back to docker-tools-build-push.yaml workflow Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Restore classifier tool removed in Phase 5 decommission Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove unit-prompt-service group from test rig manifest The prompt-service directory was deleted in the decommission PR, but the test rig groups.yaml still referenced it, causing CI to fail with "workdir does not exist" during validate and integration steps. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove deleted prompt-service and structure tool refs from bump script prompt-service/ and tools/structure/ are deleted by this PR, so remove their variables, reset_file calls, and the entire update_structure_tool_version function from bump_sdk_v0_version.sh. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix stale references from decommissioned components - Fix tool-text_extractor image name to tool-text-extractor in docker-compose.build.yaml to match CI, registry, and cloud naming - Remove stale tool-structure from run-platform.sh ignore list - Drop prompt-service from is_retryable_error docstring in retry_utils.py Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Trigger CI re-run Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Signed-off-by: Praveen Kumar <praveen@zipstack.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Praveen Kumar <praveen@zipstack.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> Co-authored-by: Deepak K <89829542+Deepak-Kesavan@users.noreply.github.com> Co-authored-by: Chandrasekharan M <117059509+chandrasekharan-zipstack@users.noreply.github.com> Co-authored-by: Athul <89829560+athul-rs@users.noreply.github.com> Co-authored-by: vishnuszipstack <117254672+vishnuszipstack@users.noreply.github.com> Co-authored-by: jimmy <ziming_zhu2002@163.com> Co-authored-by: Hari John Kuriakose <hari@zipstack.com> Co-authored-by: Chandrasekharan M <chandrasekharan@zipstack.com> Co-authored-by: Athul <athul@zipstack.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds a dedicated
OpenAI CompatibleLLM adapter for OpenAI-style chat completion endpoints that are not the official OpenAI service.The implementation is intentionally small in scope:
OpenAI CompatibleLLM adapter backed by LiteLLM'scustom_openaipathOpenAIadapter unchangedWhy
Users may already have access to OpenAI-compatible endpoints behind a private gateway or third-party provider, but the current
OpenAIadapter is specifically shaped around official OpenAI semantics.Using a separate adapter keeps those semantics explicit and avoids broadening the meaning of the existing
OpenAIadapter.Refs #1894
Refs #856
Refs #1443
Scope
This PR is limited to:
OpenAIadapter behaviorNotes
LLM._record_usagenow prefers provider-reportedprompt_tokenswhen they are present in the usage payload.If
prompt_tokensare missing,_record_usagestill falls back to LiteLLM token estimation. If that estimation raises, it now logs a warning and records0prompt tokens for usage audit instead of bubbling the exception up after a successful LLM call.This behavior change is in the shared
_record_usagepath, so it applies to every SDK1 LLM adapter that uses it, not justcustom_openai.This keeps successful LLM calls from failing at the usage-audit / billing step while preserving the current pricing semantics in this PR.
Validation
UV_SKIP_WHEEL_FILENAME_CHECK=1 uv run pytest tests/test_openai_compatible_adapter.pyUV_SKIP_WHEEL_FILENAME_CHECK=1 uv run ruff check src/unstract/sdk1/adapters/base1.py src/unstract/sdk1/adapters/llm1/__init__.py src/unstract/sdk1/adapters/llm1/openai_compatible.py src/unstract/sdk1/llm.py tests/test_openai_compatible_adapter.py