Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

feat: add TwelveLabs ModelClient (Marengo embedder + Pegasus analyze)#494

Open
mohit-twelvelabs wants to merge 3 commits into
SylphAI-Inc:mainSylphAI-Inc/AdalFlow:mainfrom
mohit-twelvelabs:feat/twelvelabs-integrationmohit-twelvelabs/AdalFlow:feat/twelvelabs-integrationCopy head branch name to clipboard
Open

feat: add TwelveLabs ModelClient (Marengo embedder + Pegasus analyze)#494
mohit-twelvelabs wants to merge 3 commits into
SylphAI-Inc:mainSylphAI-Inc/AdalFlow:mainfrom
mohit-twelvelabs:feat/twelvelabs-integrationmohit-twelvelabs/AdalFlow:feat/twelvelabs-integrationCopy head branch name to clipboard

Conversation

@mohit-twelvelabs

@mohit-twelvelabs mohit-twelvelabs commented Jun 25, 2026

Copy link
Copy Markdown

Hi! I'm Mohit, I work at TwelveLabs (@mohit-twelvelabs).

What does this PR do?

Adds an opt-in TwelveLabsClient model client that wraps the official twelvelabs Python SDK, following the same convention as the existing provider clients (groq_client, cohere_client, etc.):

  • Marengo (marengo3.0) as an Embedder backend (ModelType.EMBEDDER) — multimodal embeddings, 512-dim. Plugs straight into adalflow.core.Embedder.
  • Pegasus (pegasus1.5) as a Generator backend (ModelType.LLM) — video understanding via client.analyze(), given a prompt plus a video_url / video_id / video_asset_id.

Why it helps AdalFlow

It gives AdalFlow native, video-first embeddings and video understanding — a modality the current providers don't cover — usable in RAG/retrieval and generation pipelines without any new abstractions.

Opt-in / non-breaking

  • New optional dependency + twelvelabs extra (pip install adalflow[twelvelabs]), lazily imported via the existing OptionalPackages/LazyImport mechanism.
  • Registered in components/model_client/__init__.py, ModelClientType, and the API docs autosummary the same way as other providers.
  • No existing defaults or behavior change; nothing imports twelvelabs unless the client is used.

How it was tested

  • adalflow/tests/test_twelvelabs_client.py: 11 no-network unit tests (input mapping for both model types, response parsing, sync/async call paths) mocking the SDK, mirroring test_groq_client.py.
  • 1 TWELVELABS_API_KEY-gated live test that calls Marengo and asserts a 512-dim embedding (verified passing against the real API). Pegasus wiring is unit-tested; a full analyze run is skipped in CI as it requires an uploaded/served video.
  • black + ruff clean on changed files.

Note: per the PR template, this wasn't pre-discussed in an issue — happy to open one or adjust naming/placement to your preference.

You can grab a free API key at https://twelvelabs.io — there's a generous free tier.

@mohit-twelvelabs
feat: add TwelveLabs ModelClient (Marengo embedder + Pegasus analyze)
ee4b2b9 
Adds an opt-in TwelveLabsClient that wraps the official twelvelabs SDK:
- ModelType.EMBEDDER -> Marengo multimodal embeddings (512-dim)
- ModelType.LLM -> Pegasus video understanding via analyze()

Registered as a lazy/optional provider (extras: twelvelabs), with a
no-network unit test and a TWELVELABS_API_KEY-gated live test.
@mohit-twelvelabs
chore: regenerate poetry.lock for twelvelabs dependency
ba3d7d0
@mohit-twelvelabs

mohit-twelvelabs commented Jun 25, 2026

Copy link
Copy Markdown
Author

Regenerated adalflow/poetry.lock to match pyproject.toml after adding the twelvelabs optional dependency, so the build/lock-check (poetry install / poetry check --lock) passes again. The diff is minimal — just the new twelvelabs package entry, its extras mapping, and the refreshed content-hash; no other pinned versions changed. Pushed as commit ba3d7d0.

— Mohit (@mohit-twelvelabs, TwelveLabs)

@mohit-twelvelabs

mohit-twelvelabs commented Jun 25, 2026

Copy link
Copy Markdown
Author

Follow-up: with the lock regenerated, poetry install now succeeds, which surfaced a second, unrelated CI failure — tests/test_twelvelabs_client.py was importing the optional twelvelabs SDK at module top level, so pytest errored at collection (ModuleNotFoundError: No module named 'twelvelabs') since CI runs a plain poetry install without extras.

Fixed in commit 4336bba by wrapping that import in try/except ImportError + a module-level pytest.skip, matching the repo's existing optional-dependency pattern (see tests/test_anthropic_client.py). The tests run normally when installed with the twelvelabs extra and skip cleanly otherwise. Verified the skip path locally against a missing-SDK environment.

— Mohit (@mohit-twelvelabs, TwelveLabs)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@mohit-twelvelabs
Morty Proxy This is a proxified and sanitized view of the page, visit original site.