[None][fix] Enable LoRA in EAGLE3 speculative decoding by Funatiq · Pull Request #13005 · NVIDIA/TensorRT-LLM

Funatiq · Apr 13, 2026

Summary by CodeRabbit

Bug Fixes
- Improved robustness of LoRA adapter slot management by gracefully handling cases where PEFT cache data is unavailable, preventing potential errors.
Tests
- Added comprehensive unit and integration tests for LoRA functionality, including CUDA graph parameter updates and speculative decoding scenarios.

Description

Handle optional PEFT cache manager in AdapterSlotManager and update weight pointers in CudaGraphLoraParams.
Add unit tests for CudaGraphLoraParams and AdapterSlotManager to validate behavior when PEFT cache manager is missing.
Add integration test for LoRA in EAGLE3 speculative decoding with and without CUDA graph.

Test Coverage

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

Funatiq · Apr 13, 2026

/bot run

tensorrt-cicd · Apr 13, 2026

PR_Github #43093 [ run ] triggered by Bot. Commit: bca258d Link to invocation

tensorrt-cicd · Apr 13, 2026

PR_Github #43093 [ run ] completed with state SUCCESS. Commit: bca258d
/LLM/main/L0_MergeRequest_PR pipeline #33732 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

Funatiq · Apr 13, 2026

/bot run

tensorrt-cicd · Apr 13, 2026

PR_Github #43108 [ run ] triggered by Bot. Commit: a7e50d9 Link to invocation

tensorrt-cicd · Apr 13, 2026

PR_Github #43108 [ run ] completed with state SUCCESS. Commit: a7e50d9
/LLM/main/L0_MergeRequest_PR pipeline #33744 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

Funatiq · Apr 15, 2026

/bot run

tensorrt-cicd · Apr 15, 2026

PR_Github #43510 [ run ] triggered by Bot. Commit: 37caeac Link to invocation

tensorrt-cicd · Apr 15, 2026

PR_Github #43510 [ run ] completed with state SUCCESS. Commit: 37caeac
/LLM/main/L0_MergeRequest_PR pipeline #34023 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

- Handle optional PEFT cache manager in AdapterSlotManager and update weight pointers in CudaGraphLoraParams. - Add unit tests for CudaGraphLoraParams and AdapterSlotManager to validate behavior when PEFT cache manager is missing. - Add integration test for LoRA in EAGLE3 speculative decoding with and without CUDA graph. Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>

Funatiq · Apr 15, 2026

/bot run

tensorrt-cicd · Apr 15, 2026

PR_Github #43579 [ run ] triggered by Bot. Commit: 6b051cd Link to invocation

tensorrt-cicd · Apr 16, 2026

PR_Github #43579 [ run ] completed with state SUCCESS. Commit: 6b051cd
/LLM/main/L0_MergeRequest_PR pipeline #34074 completed with status: 'SUCCESS'
Pipeline passed with automatic retried tests. Check the rerun report for details.

CI Report

Link to invocation

coderabbitai · Apr 16, 2026

📝 Walkthrough

Walkthrough

The changes add null-safety to two LoRA parameter management methods by making PEFT-related inputs optional, enabling graceful handling when LoRA/PEFT context is unavailable. New unit tests validate null-parameter behavior, and integration testing extends speculative decoding to cover LoRA scenarios.

Changes

Cohort / File(s)	Summary
LoRA Parameter Management `tensorrt_llm/_torch/peft/lora/adapter_slot_manager.py`, `tensorrt_llm/_torch/peft/lora/cuda_graph_lora_params.py`	Updated method signatures to accept optional PEFT cache manager and PEFT table parameters. Added early-return checks and conditional logic to skip processing when inputs are `None`, preventing null-pointer dereferences.
LoRA Unit Tests `tests/unittest/_torch/lora/test_lora.py`	New test module with two unit tests validating null-safe behavior of `CudaGraphLoraParams.update_weight_pointers()` and `AdapterSlotManager.remove_evicted_slots_in_cpp()` when called with `None` arguments.
Test Configuration & Integration `tests/integration/test_lists/test-db/l0_h100.yml`, `tests/unittest/_torch/speculative/test_eagle3.py`	Added LoRA test suite path to H100 PyTorch test selection. Extended speculative decoding test with new parametrized LoRA integration test including per-request LoRA configuration and conditional CUDA graph setup.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 77.78% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check	❓ Inconclusive	The PR description includes a clear summary of the main changes and test coverage, but the 'Test Coverage' section is empty, leaving a required template section incomplete.	Complete the 'Test Coverage' section by explicitly listing the unit tests (test_lora.py) and integration test (test_eagle3_lora) that were added to validate the changes.

✅ Passed checks (1 passed)

Check name	Status	Explanation
Title check	✅ Passed	The PR title clearly summarizes the main change: enabling LoRA support in EAGLE3 speculative decoding, which aligns with the core objective of the changeset.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@tests/unittest/_torch/lora/test_lora.py`:
- Around line 1-5: This new test file is missing the required NVIDIA copyright
header—add the standard NVIDIA copyright header (with the current year) at the
very top of the file before any imports; ensure the header precedes the existing
imports for AdapterSlotManager and CudaGraphLoraParams and matches the project's
canonical header format.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 18218512-59c9-46b8-aea5-e32c2e8dedd1

📥 Commits

Reviewing files that changed from the base of the PR and between 51f7956 and 6b051cd.

📒 Files selected for processing (5)

tensorrt_llm/_torch/peft/lora/adapter_slot_manager.py
tensorrt_llm/_torch/peft/lora/cuda_graph_lora_params.py
tests/integration/test_lists/test-db/l0_h100.yml
tests/unittest/_torch/lora/test_lora.py
tests/unittest/_torch/speculative/test_eagle3.py

brb-nv

LGTM.

github-actions Bot assigned Funatiq Apr 13, 2026

Funatiq changed the title ~~[fix] Enable LoRA in EAGLE3 speculative decoding~~ [None][fix] Enable LoRA in EAGLE3 speculative decoding Apr 13, 2026

Funatiq force-pushed the dev/fix/eagle_lora branch from bca258d to a7e50d9 Compare April 13, 2026 19:41

Funatiq force-pushed the dev/fix/eagle_lora branch from a7e50d9 to 37caeac Compare April 15, 2026 14:14

Funatiq added 2 commits April 15, 2026 22:25

fixup! [fix] Enable LoRA in EAGLE3 speculative decoding

6b051cd

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>

Funatiq force-pushed the dev/fix/eagle_lora branch from 37caeac to 6b051cd Compare April 15, 2026 20:25

Funatiq marked this pull request as ready for review April 16, 2026 06:59

Funatiq requested a review from a team as a code owner April 16, 2026 06:59

Funatiq requested review from a team and schetlur-nv April 16, 2026 06:59

coderabbitai Bot reviewed Apr 16, 2026

View reviewed changes

Comment thread tests/unittest/_torch/lora/test_lora.py

brb-nv approved these changes Apr 17, 2026

View reviewed changes

mikeiovine approved these changes Apr 17, 2026

View reviewed changes

ziyixiong-nv approved these changes Apr 18, 2026

View reviewed changes

Funatiq merged commit 6428102 into NVIDIA:main Apr 18, 2026
8 of 9 checks passed

Funatiq deleted the dev/fix/eagle_lora branch April 18, 2026 06:39

Search code, repositories, users, issues, pull requests...

Conversation

Funatiq commented Apr 13, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

Funatiq commented Apr 13, 2026

Uh oh!

tensorrt-cicd commented Apr 13, 2026

Uh oh!

tensorrt-cicd commented Apr 13, 2026

Uh oh!

Funatiq commented Apr 13, 2026

Uh oh!

tensorrt-cicd commented Apr 13, 2026

Uh oh!

tensorrt-cicd commented Apr 13, 2026

Uh oh!

Funatiq commented Apr 15, 2026

Uh oh!

tensorrt-cicd commented Apr 15, 2026

Uh oh!

tensorrt-cicd commented Apr 15, 2026

Uh oh!

Funatiq commented Apr 15, 2026

Uh oh!

tensorrt-cicd commented Apr 15, 2026

Uh oh!

tensorrt-cicd commented Apr 16, 2026

Uh oh!

coderabbitai Bot commented Apr 16, 2026

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (1 warning, 1 inconclusive)

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

brb-nv left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Funatiq commented Apr 13, 2026 •

edited by coderabbitai Bot

Loading