[sglang] feat: Add multi-interaction registry support and testing #2184

SwordFaith · Jun 24, 2025

What does this PR do?

Add concise overview of what this PR aims to achieve or accomplish. Reference related GitHub issues and PRs that help with the review.

This PR implements multi-interaction support in SGLangRollout, enabling sample-level interaction selection similar to the existing tools system. The implementation includes a new interaction registry system that allows multiple named
interactions to be configured and used within a single rollout instance. #1630

Core Implementation

New Interaction Registry System: Created verl/interactions/utils/interaction_registry.py with functions to dynamically load and manage multiple interaction instances from configuration files
Enhanced SGLangRollout:
- Replaced single interaction attribute with interaction_map: dict[str, BaseInteraction]
- Updated _initialize_interactions() method to support multiple interactions via registry
- Modified interaction selection logic to use interaction_kwargs.name for sample-level binding
Configuration Updates: Added name field support in interaction config format with automatic name generation fallback

Data Processing

Updated GSM8K Preprocessing: Modified examples/data_preprocess/gsm8k_multiturn_w_interaction.py to inject name field in interaction_kwargs
Enhanced Configuration: Updated examples/sglang_multiturn/config/interaction_config/gsm8k_interaction_config.yaml with explicit name field

Testing & Quality

Comprehensive Test Suite: Added tests/interactions/test_interaction_registry.py with full coverage of registry functionality
Integration Tests: Created tests/workers/rollout/test_sglang_multi_interaction.py for multi-interaction scenarios
Updated Existing Tests: Modified existing interaction tests to support new name attribute and configuration format
Error Handling: Added validation for duplicate names, missing interactions, and edge cases

Backward Compatibility

Graceful Degradation: When no interaction config is provided, system works without interactions (empty interaction_map)
Default Name Handling: Falls back to "gsm8k" when no name is specified in interaction_kwargs
Existing API Preservation: All existing interaction functionality remains unchanged

Key Features

Sample-Level Selection: Each sample can specify which interaction to use via interaction_kwargs.name
Registry Pattern: Similar architecture to existing tools system for consistency
Automatic Naming: Intelligent name generation from class names (e.g., Gsm8kInteraction → gsm8k)
Duplicate Prevention: Runtime validation prevents naming conflicts
Flexible Configuration: Supports both explicit names and automatic derivation

Checklist Before Starting

Search for similar PRs. Paste at least one query link here: ...
Format the PR title as [{modules}] {type}: {description} (This will be checked by the CI)
- {modules} include fsdp, megatron, sglang, vllm, rollout, trainer, ci, training_utils, recipe, hardware, deployment, ray, worker, single_controller, misc, perf, model, algo, env, tool, ckpt, doc, data
- If this PR involves multiple modules, separate them with , like [megatron, fsdp, doc]
- {type} is in feat, fix, refactor, chore, test
- If this PR breaks any API (CLI arguments, config, function signature, etc.), add [BREAKING] to the beginning of the title.
- Example: [BREAKING][fsdp, megatron] feat: dynamic batching

Test

For changes that can not be tested by CI (e.g., algorithm implementation, new model support), validate by experiment(s) and show results like training curve plots, evaluation results, etc.

Unit tests for interaction registry functionality
Integration tests for multi-interaction rollout scenarios
Backward compatibility tests for existing single-interaction workflows
Error handling tests for edge cases and invalid configurations
Updated existing GSM8K interaction tests

API and Usage Example

Demonstrate how the API changes if any, and provide usage example(s) if possible.

Configuration (interaction_config.yaml):
interaction:
- name: "gsm8k"
class_name: "verl.interactions.gsm8k_interaction.Gsm8kInteraction"
config: {}
- name: "custom_solver"
class_name: "custom.interactions.CustomInteraction"
config: {"solver_type": "advanced"}

Data Sample:

  {
      "interaction_kwargs": {
          "name": "gsm8k",
          "query": "What is 2+2?",
          "ground_truth": "4"
      }
  }

High-Level Design

Demonstrate the high-level design if this PR is complex.

Specific Changes

List the specific changes.

Checklist Before Submitting

Important

Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.

Read the Contribute Guide.
Apply pre-commit checks: pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always
Add / Update the documentation.
Add unit or end-to-end test(s) to the CI workflow to cover all the code. If not feasible, explain why: ...
Once your PR is ready for CI, send a message in the ci-request channel in the verl Slack workspace.

chenhaiq · Jun 26, 2025

verl/interactions/utils/interaction_registry.py

+logger.setLevel(os.getenv("VERL_LOGGING_LEVEL", "WARN"))
+
+
+def get_interaction_class(cls_name):


可以复用已有的 verl/tools/utils/tool_registry.py::get_tool_class 函数

interaction and tool are different concepts. Interactions are playing user role which used in partial exposure/collaborative task, reflection task and workflow node training task.

chenhaiq · Jun 26, 2025

verl/interactions/utils/interaction_registry.py

+    return interaction_cls
+
+
+def initialize_interactions_from_config(interaction_config_file):


跟已有的 verl/tools/utils/tool_registry.py::initialize_tools_from_config 也有重复，能不能重构下已有的那个函数，让mcp也使用map的新格式？

chenhaiq · Jun 26, 2025

verl/workers/rollout/sglang_rollout/sglang_rollout.py

-        if _req.interaction_kwargs:
+        if _req.interaction_kwargs and self.interaction_map:
            interaction_kwargs = _req.interaction_kwargs
-            await self.interaction.start_interaction(_req.request_id, **interaction_kwargs)


跟上面的代码也是重复的，可以合并成 get_ interaction 方法

…lcengine#2184) ### What does this PR do? > Add **concise** overview of what this PR aims to achieve or accomplish. Reference related GitHub issues and PRs that help with the review. This PR implements multi-interaction support in SGLangRollout, enabling sample-level interaction selection similar to the existing tools system. The implementation includes a new interaction registry system that allows multiple named interactions to be configured and used within a single rollout instance. volcengine#1630 Core Implementation - New Interaction Registry System: Created verl/interactions/utils/interaction_registry.py with functions to dynamically load and manage multiple interaction instances from configuration files - Enhanced SGLangRollout: - Replaced single interaction attribute with interaction_map: dict[str, BaseInteraction] - Updated _initialize_interactions() method to support multiple interactions via registry - Modified interaction selection logic to use interaction_kwargs.name for sample-level binding - Configuration Updates: Added name field support in interaction config format with automatic name generation fallback Data Processing - Updated GSM8K Preprocessing: Modified examples/data_preprocess/gsm8k_multiturn_w_interaction.py to inject name field in interaction_kwargs - Enhanced Configuration: Updated examples/sglang_multiturn/config/interaction_config/gsm8k_interaction_config.yaml with explicit name field Testing & Quality - Comprehensive Test Suite: Added tests/interactions/test_interaction_registry.py with full coverage of registry functionality - Integration Tests: Created tests/workers/rollout/test_sglang_multi_interaction.py for multi-interaction scenarios - Updated Existing Tests: Modified existing interaction tests to support new name attribute and configuration format - Error Handling: Added validation for duplicate names, missing interactions, and edge cases Backward Compatibility - Graceful Degradation: When no interaction config is provided, system works without interactions (empty interaction_map) - Default Name Handling: Falls back to "gsm8k" when no name is specified in interaction_kwargs - Existing API Preservation: All existing interaction functionality remains unchanged Key Features 1. Sample-Level Selection: Each sample can specify which interaction to use via interaction_kwargs.name 2. Registry Pattern: Similar architecture to existing tools system for consistency 3. Automatic Naming: Intelligent name generation from class names (e.g., Gsm8kInteraction → gsm8k) 4. Duplicate Prevention: Runtime validation prevents naming conflicts 5. Flexible Configuration: Supports both explicit names and automatic derivation

Add multi-interaction registry support and testing

e0699ec

SwordFaith requested review from chenhaiq and zhaochenyang20 as code owners June 24, 2025 13:23

SwordFaith changed the title ~~[sglang][interaction] Add multi-interaction registry support and testing~~ [sglang] feat: Add multi-interaction registry support and testing Jun 24, 2025

Update related doc

f6f12e1

SwordFaith requested a review from eric-haibin-lin as a code owner June 24, 2025 13:27

SwordFaith added 9 commits June 24, 2025 23:51

Fix unit-test bug

50e2f87

Fix multi interaction test world_size issue

178024e

FIx world_size issue in mock

6b3f1fa

FIx dist env mock issue

69e1dcb

Fix dist env uninit

e16c29e

Try fix ci

ed24f1d

Try fix validation error

460b4b5

Fix ci bug

612d1f0

Fix mp isinstance issue

0ae0572

chenhaiq reviewed Jun 26, 2025

View reviewed changes

chenhaiq approved these changes Jun 27, 2025

View reviewed changes

chenhaiq merged commit b816d17 into volcengine:main Jun 27, 2025
37 of 40 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[sglang] feat: Add multi-interaction registry support and testing #2184

[sglang] feat: Add multi-interaction registry support and testing #2184

Uh oh!

SwordFaith commented Jun 24, 2025 •

edited

Loading

Uh oh!

chenhaiq Jun 26, 2025

Uh oh!

SwordFaith Jun 27, 2025

Uh oh!

chenhaiq Jun 26, 2025

Uh oh!

chenhaiq Jun 26, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		logger.setLevel(os.getenv("VERL_LOGGING_LEVEL", "WARN"))


		def get_interaction_class(cls_name):

		return interaction_cls


		def initialize_interactions_from_config(interaction_config_file):

Search code, repositories, users, issues, pull requests...

[sglang] feat: Add multi-interaction registry support and testing #2184

[sglang] feat: Add multi-interaction registry support and testing #2184

Uh oh!

Conversation

SwordFaith commented Jun 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Checklist Before Starting

Test

API and Usage Example

High-Level Design

Specific Changes

Checklist Before Submitting

Uh oh!

chenhaiq Jun 26, 2025

Choose a reason for hiding this comment

Uh oh!

SwordFaith Jun 27, 2025

Choose a reason for hiding this comment

Uh oh!

chenhaiq Jun 26, 2025

Choose a reason for hiding this comment

Uh oh!

chenhaiq Jun 26, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

SwordFaith commented Jun 24, 2025 •

edited

Loading