You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Some clients have reported pipelines failing in production due to provider quota limits (for example, AWS throttling errors).
Describe the solution you'd like
Introduce a new component that accepts a list of ChatGenerators. If one fails (e.g., quota exceeded), it falls back to the next available generator in the list.
Additional context
We should consider how the fallback strategy or strategies should be implemented and perhaps consider allowing a custom function to be passed in to allow for custom fallback strategies.