Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

LiteLLM ADK wrapper does not handle instructions properly. #3657

Copy link
Copy link
@thesynapses

Description

@thesynapses
Issue body actions

In ADK 1.19.0 - I have observed:

Gemini Native ADK:

  • Uses [role='system'] directly in [Content] objects
    OR passes [system_instruction] in [GenerateContentConfig] (not as a message)
  • Gemini API natively understands this field

LiteLLM ADK Wrapper:

  • Converts [system_instruction] to a message with [role='developer']
  • Tries to use OpenAI's convention
  • But the "developer" role is OpenAI-specific!
  • Claude/Anthropic expect [role='system']
  • LiteLLM's Anthropic handler filters for [role='system'], not [role='developer']

Result and bug: System instruction is ignored or mishandled.
This causes system instructions to be lost for Claude, resulting in a poorly formatted output.

The best is to patch ADK to use [role="system"] and that would work for everyone.
Here is a proposed patch.

# in lite_llm.py around like 902
messages.insert(
    0,
    ChatCompletionSystemMessage(  # Changed from ChatCompletionDeveloperMessage
        role="system",  # Changed from "developer"
        content=llm_request.config.system_instruction,
    ),
)
Reactions are currently unavailable

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

    Morty Proxy This is a proxified and sanitized view of the page, visit original site.