Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

OpenRouter: Fix token usage response #560

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Apr 17, 2025
Merged

OpenRouter: Fix token usage response #560

merged 2 commits into from
Apr 17, 2025

Conversation

lukalocniskar
Copy link
Contributor

@lukalocniskar lukalocniskar commented Apr 16, 2025 •
edited by iBotPeaches
Loading

What:

  • Bug Fix
  • [] New Feature

fixes: #524

Description:

Inconsistencies in how token usage details are handled from provider OpenRouter.

Changes

  • Ensures consistent output structure for token usage details (prompt_tokens_details and completion_tokens_details)
  • Includes all token fields in responses
  • Fix error on TypeError for CreateResponseUsageCompletionTokensDetails where $acceptedPredictionTokens must be of type int, null given

Before

OpenRouter and OpenAI responses had different structures, with OpenRouter omitting zero-valued token fields while OpenAI included them:

  • OpenRouter omitted audio_tokens, accepted_prediction_tokens, and rejected_prediction_tokens
  • OpenAI included all token fields regardless of value

After

Both providers now return consistent response structures with all token fields present:

  • All responses include complete token usage details for better consistency
  • Makes responses more predictable for consumers of the API client
  • Prevents unexpected behavior when switching between providers

This change ensures consistent behavior across both supported providers by the client.

@iBotPeaches
Copy link
Collaborator

I'm not really leaning towards this being the right fix. This seems to more lie and keep a structured approach, which reads as returning all 0's for implementations that don't support it. That doesn't seem right to me.

Nullable/removed sure, but falling back to 0's to keep parity doesn't seem like the play.

@lukalocniskar
Copy link
Contributor Author

Thanks for feedback, i have taken it into account.

@iBotPeaches
Copy link
Collaborator

Thanks, so just to confirm the test you added - chatCompletionOpenRouter is a real life copy of the usage ? I just want to visualize the difference between OpenAI/OpenRouter for that. I believe so based on your comment here:

OpenRouter omitted audio_tokens, accepted_prediction_tokens, and rejected_prediction_tokens
OpenAI included all token fields regardless of value

@lukalocniskar
Copy link
Contributor Author

Test is a real openRouter response.

If this is a good direction, i'll also check non completion request. Might be good to wait with review.

@iBotPeaches
Copy link
Collaborator

Thanks - I guess I'm just a bit confused. You changed the detail objects, but those are missing entirely in your OpenRouter summary payload.

So in my head - those objects in entirety (details) would be just be missing. You instead patched the object to be nullable on each property inside of it (audio, reasoning, rejected, accepted) - which was moot since the detail object (completion_tokens_details) is not even provided by OpenRouter.

@lukalocniskar
Copy link
Contributor Author

Depends on a model. Will share a few.

OpenAI\ValueObjects\Transporter\Response^ {#5785
  -data: array:8 [
    "id" => "gen-1744900650-1zQPItTWw6iX4VlC65Xy"
    "provider" => "OpenAI"
    "model" => "openai/gpt-4o-mini"
    "object" => "chat.completion"
    "created" => 1744900650
    "choices" => array:1 [
      0 => array:5 [
        "logprobs" => null
        "finish_reason" => "stop"
        "native_finish_reason" => "stop"
        "index" => 0
        "message" => array:4 [
          "role" => "assistant"
          "content" => "Hello! How can I assist you today?"
          "refusal" => null
          "reasoning" => null
        ]
      ]
    ]
    "system_fingerprint" => "fp_0392822090"
    "usage" => array:5 [
      "prompt_tokens" => 21
      "completion_tokens" => 21
      "total_tokens" => 42
      "prompt_tokens_details" => array:1 [
        "cached_tokens" => 0
      ]
      "completion_tokens_details" => array:1 [
        "reasoning_tokens" => 0
      ]
    ]
  ]
  -meta: OpenAI\Responses\Meta\MetaInformation^ {#5796
    +requestId: null
    +openai: OpenAI\Responses\Meta\MetaInformationOpenAI^ {#5784
      +model: null
      +organization: null
      +version: null
      +processingMs: null
    }
    +requestLimit: null
    +tokenLimit: null
  }
}

OpenAI\ValueObjects\Transporter\Response^ {#5785
  -data: array:7 [
    "id" => "gen-1744910839-Cmvs50bFBTLoqRE1lqft"
    "provider" => "Google"
    "model" => "google/gemini-2.5-pro-preview-03-25"
    "object" => "chat.completion"
    "created" => 1744910839
    "choices" => array:1 [
      0 => array:5 [
        "logprobs" => null
        "finish_reason" => "stop"
        "native_finish_reason" => "STOP"
        "index" => 0
        "message" => array:4 [
          "role" => "assistant"
          "content" => "Hello there! I'm a large language model, trained by Google."
          "refusal" => null
          "reasoning" => null
        ]
      ]
    ]
    "usage" => array:3 [
      "prompt_tokens" => 10
      "completion_tokens" => 138
      "total_tokens" => 148
    ]
  ]
  -meta: OpenAI\Responses\Meta\MetaInformation^ {#5796
    +requestId: null
    +openai: OpenAI\Responses\Meta\MetaInformationOpenAI^ {#5784
      +model: null
      +organization: null
      +version: null
      +processingMs: null
    }
    +requestLimit: null
    +tokenLimit: null
  }
}

OpenAI\ValueObjects\Transporter\Response^ {#5785
  -data: array:8 [
    "id" => "gen-1744911228-ObeD7rDHRVxtQtRo7UdZ"
    "provider" => "xAI"
    "model" => "x-ai/grok-3-mini-beta"
    "object" => "chat.completion"
    "created" => 1744911228
    "choices" => array:1 [
      0 => array:5 [
        "logprobs" => null
        "finish_reason" => "stop"
        "native_finish_reason" => "stop"
        "index" => 0
        "message" => array:4 [
          "role" => "assistant"
          "content" => "Hello! I'm Grok, an AI model created by xAI. I'm here to help with any questions or tasks you have, so feel free to ask away! 😊"
          "refusal" => null
          "reasoning" => """
            First, the user is asking "Hello! what model are you?" // ...
            """
        ]
      ]
    ]
    "system_fingerprint" => "fp_d133ae3397"
    "usage" => array:5 [
      "prompt_tokens" => 21
      "completion_tokens" => 36
      "total_tokens" => 392
      "prompt_tokens_details" => array:1 [
        "cached_tokens" => 0
      ]
      "completion_tokens_details" => array:1 [
        "reasoning_tokens" => 335
      ]
    ]
  ]
  -meta: OpenAI\Responses\Meta\MetaInformation^ {#5796
    +requestId: null
    +openai: OpenAI\Responses\Meta\MetaInformationOpenAI^ {#5784
      +model: null
      +organization: null
      +version: null
      +processingMs: null
    }
    +requestLimit: null
    +tokenLimit: null
  }
}

@iBotPeaches
Copy link
Collaborator

ahh - thank you. This helps a lot more. Its dependent on the model routed through OpenRouter.

@iBotPeaches
Copy link
Collaborator

So I believe what we should do is augment one or both of these tests to confirm these partial returns you see from Grok/Gemini above. That way full confidence it returns as we expect with the different iterations of (null, full, partial)

@lukalocniskar
Copy link
Contributor Author

Tested on response schemas of models:

  • openai/gpt-4o-mini
  • google/gemini-2.5-pro-preview-03-25
  • google/gemma-3-4b-it:free
  • x-ai/grok-3-mini-beta
  • cohere/command-r7b-12-2024
  • qwen/qwen2.5-coder-7b-instruct
  • mistralai/mistral-small-3.1-24b-instruct:free
  • deepseek/deepseek-r1-distill-llama-8b
  • openrouter/auto

iBotPeaches
iBotPeaches approved these changes Apr 17, 2025
View reviewed changes
Copy link
Collaborator

@iBotPeaches iBotPeaches left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks! I think we are set now

@iBotPeaches iBotPeaches merged commit 205be4a into openai-php:main Apr 17, 2025
10 checks passed
@arthmelikyan
Copy link

Hi, I was running into this issue as well—thanks for the quick fix!

Do you know when the fix will be released? Or should I switch to dev-main in the meantime? I’m was using v0.10.3, by the way.

@iBotPeaches
Copy link
Collaborator

Do you know when the fix will be released? Or should I switch to dev-main in the meantime? I’m was using v0.10.3, by the way.

I'm the newest maintainer so I cannot release until I'm more trusted. I have pinged the other maintainers in email (5d ago and 1d ago). I'm trying my best to get a release out.

This was referenced May 27, 2025
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@iBotPeaches iBotPeaches iBotPeaches approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Bug]: $acceptedPredictionTokens must be of type int, null given
3 participants
@lukalocniskar @iBotPeaches @arthmelikyan
Morty Proxy This is a proxified and sanitized view of the page, visit original site.