Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Comments

Close side panel

Update vLLM to 0.6.6.post1#2892

Merged
mergify[bot] merged 1 commit intoinstructlab:maininstructlab/instructlab:mainfrom
fabiendupont:update-vllm-0.6.6.post1fabiendupont/instructlab:update-vllm-0.6.6.post1Copy head branch name to clipboard
Jan 28, 2025
Merged

Update vLLM to 0.6.6.post1#2892
mergify[bot] merged 1 commit intoinstructlab:maininstructlab/instructlab:mainfrom
fabiendupont:update-vllm-0.6.6.post1fabiendupont/instructlab:update-vllm-0.6.6.post1Copy head branch name to clipboard

Conversation

@fabiendupont
Copy link
Contributor

@fabiendupont fabiendupont commented Jan 9, 2025

This PR is a follow-up to #2865 that relaxed the PyTorch version range. Even with that range extension, we realized that PyTorch 2.4 is still used when installing instructlab[vllm-cuda], because vLLM 0.6.2 has a requirement on PyTorch 2.4.

This new PR updates the version of vLLM to 0.6.6.post1, which is the latest available in the Open Data Hub fork of vLLM. The vLLM changelog doesn't highlight much risk in this version bump.

Resolves #2702

@mergify mergify bot added documentation Improvements or additions to documentation dependencies Relates to dependencies labels Jan 9, 2025
@fabiendupont fabiendupont mentioned this pull request Jan 9, 2025
Merged
@mergify mergify bot added the ci-failure PR has at least one CI failure label Jan 9, 2025
@fabiendupont fabiendupont force-pushed the update-vllm-0.6.6.post1 branch from 8fe682b to d09d8f3 Compare January 9, 2025 08:17
@mergify mergify bot added ci-failure PR has at least one CI failure and removed ci-failure PR has at least one CI failure labels Jan 9, 2025
@cdoern cdoern added the hold In-progress PR. Tag should be removed before merge. label Jan 9, 2025
@nathan-weinberg
Copy link
Member

nathan-weinberg commented Jan 9, 2025

We'll need a new SDG z-stream that allows for a higher Torch version to resolve the dependency issue that will allow us to test this - that backport is being worked on here: instructlab/sdg#469

@bbrowning
Copy link
Contributor

bbrowning commented Jan 10, 2025

SDG v0.6.3 has been released - the only change from v0.6.2 (used by instructlab currently) is the raising of this PyTorch upper version limit, which should unblock this PR.

@nathan-weinberg
Copy link
Member

nathan-weinberg commented Jan 10, 2025

@fabiendupont can you rebase this? we'll see if CI is able to complete the install and then we can test if the new vLLM package will work

@fabiendupont fabiendupont force-pushed the update-vllm-0.6.6.post1 branch from d09d8f3 to 1d9654e Compare January 10, 2025 15:20
@mergify
Copy link
Contributor

mergify bot commented Jan 10, 2025

This pull request has merge conflicts that must be resolved before it can be
merged. @fabiendupont please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify bot added needs-rebase This Pull Request needs to be rebased and removed ci-failure PR has at least one CI failure labels Jan 10, 2025
@fabiendupont fabiendupont force-pushed the update-vllm-0.6.6.post1 branch from 1d9654e to 7dcdccf Compare January 10, 2025 15:28
@mergify mergify bot removed the needs-rebase This Pull Request needs to be rebased label Jan 10, 2025
@bbrowning
Copy link
Contributor

bbrowning commented Jan 10, 2025

Do we need to manually kick off a large or xlarge CI test for this so that vllm actually gets used? I think both the small and medium e2e tests use llama_cpp, right?

@nathan-weinberg
Copy link
Member

nathan-weinberg commented Jan 10, 2025

Do we need to manually kick off a large or xlarge CI test for this so that vllm actually gets used? I think both the small and medium e2e tests use llama_cpp, right?

Small is using only llama_cpp, Medium actually uses both: https://github.com/instructlab/instructlab/blob/main/docs/ci.md/#e2e-test-coverage-matrix

Now ilab was successfully installed here with the Medium job - that's good! But to be safe, we should definitely kick off at least a Large job for some functional confidence - I will do that right now.

@github-actions
Copy link

github-actions bot commented Jan 10, 2025

E2E (NVIDIA L40S x4) workflow launched on this PR: View run

@nathan-weinberg nathan-weinberg mentioned this pull request Jan 10, 2025
Closed
@github-actions
Copy link

github-actions bot commented Jan 10, 2025

e2e workflow succeeded on this PR: View run, congrats!

nathan-weinberg
nathan-weinberg approved these changes Jan 10, 2025
View reviewed changes
Copy link
Member

@nathan-weinberg nathan-weinberg left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI suggests no functional regressions from this vLLM bump, so I am approving from my side

@mergify mergify bot added the one-approval PR has one approval from a maintainer label Jan 10, 2025
@n1hility
Copy link
Contributor

n1hility commented Jan 11, 2025

We have some users using a combo with 0.6.4.post1, would love to see a CI test run pass on that.

@nathan-weinberg
Copy link
Member

nathan-weinberg commented Jan 13, 2025

We have some users using a combo with 0.6.4.post1, would love to see a CI test run pass on that.

Do you think we should be updating to that version instead? If not, can we run that in a separate PR so we can get this in as early before the 0.23.0 release as we can?

fabiendupont added a commit to fabiendupont/instructlab that referenced this pull request Jan 13, 2025
@fabiendupont
chore!: Update vLLM to 0.6.4.post1
8cfa99e 
As an intermediate step to updating vLLM to 0.6.6.post1 in instructlab#2892, this
pull request updates vLLM to 0.6.4.post1 to ensure that the CI tests
succeed with this version too.

Signed-off-by: Fabien Dupont <fdupont@redhat.com>
@fabiendupont fabiendupont mentioned this pull request Jan 13, 2025
Merged
@fabiendupont
Copy link
Contributor Author

fabiendupont commented Jan 13, 2025

@n1hility, @nathan-weinberg, I have created #2910 that updates vLLM to 0.6.4.post1. This will allow us to run the CI tests and be more comfortable with the bigger leap forward to 0.6.6.post1.

mergify bot added a commit that referenced this pull request Jan 14, 2025
@mergify
chore!: Update vLLM to 0.6.4.post1 (#2910)
5925fd4 
As an intermediate step to updating vLLM to 0.6.6.post1 in #2892, this pull request updates vLLM to 0.6.4.post1 to ensure that the CI tests succeed with this version too.


Approved-by: nathan-weinberg

Approved-by: n1hility
dmartinol pushed a commit to dmartinol/instructlab that referenced this pull request Jan 15, 2025
@fabiendupont @dmartinol
chore!: Update vLLM to 0.6.4.post1
631471a 
As an intermediate step to updating vLLM to 0.6.6.post1 in instructlab#2892, this
pull request updates vLLM to 0.6.4.post1 to ensure that the CI tests
succeed with this version too.

Signed-off-by: Fabien Dupont <fdupont@redhat.com>
@nathan-weinberg nathan-weinberg added this to the 0.24.0 milestone Jan 15, 2025
@mergify
Copy link
Contributor

mergify bot commented Jan 15, 2025

This pull request has merge conflicts that must be resolved before it can be
merged. @fabiendupont please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify bot added the needs-rebase This Pull Request needs to be rebased label Jan 15, 2025
mbestavros pushed a commit to mbestavros/instructlab that referenced this pull request Jan 16, 2025
@mergify
chore!: Update vLLM to 0.6.4.post1 (instructlab#2910)
324ffcb 
As an intermediate step to updating vLLM to 0.6.6.post1 in instructlab#2892, this pull request updates vLLM to 0.6.4.post1 to ensure that the CI tests succeed with this version too.

Approved-by: nathan-weinberg

Approved-by: n1hility
@nathan-weinberg nathan-weinberg removed this from the 0.24.0 milestone Jan 28, 2025
@nathan-weinberg nathan-weinberg removed the hold In-progress PR. Tag should be removed before merge. label Jan 28, 2025
@fabiendupont fabiendupont force-pushed the update-vllm-0.6.6.post1 branch from 7dcdccf to 0e60b4d Compare January 28, 2025 16:38
@mergify mergify bot added ci-failure PR has at least one CI failure and removed needs-rebase This Pull Request needs to be rebased labels Jan 28, 2025
@fabiendupont fabiendupont force-pushed the update-vllm-0.6.6.post1 branch from 0e60b4d to e3574f2 Compare January 28, 2025 16:50
@fabiendupont
chore!: Update vLLM to 0.6.6.post1
9c4a31c 
This PR is a follow-up to instructlab#2865 that relaxed the PyTorch version range.
Even with that range extension, we realized that PyTorch 2.4 is still
used when installing `instructlab[vllm-cuda]`, because vLLM 0.6.2 has a
requirement on PyTorch 2.4.

This new PR updates the version of vLLM to 0.6.6.post1, which is the
latest available in the Open Data Hub fork of vLLM. The vLLM changelog
doesn't highlight much risk in this version bump.

Resolves instructlab#2702

Signed-off-by: Fabien Dupont <fdupont@redhat.com>
@fabiendupont fabiendupont force-pushed the update-vllm-0.6.6.post1 branch from e3574f2 to 9c4a31c Compare January 28, 2025 17:10
@mergify mergify bot removed the ci-failure PR has at least one CI failure label Jan 28, 2025
@github-actions
Copy link

github-actions bot commented Jan 28, 2025

E2E (NVIDIA L40S x4) workflow launched on this PR: View run

@nathan-weinberg nathan-weinberg requested review from a team and removed request for courtneypacheco and n1hility January 28, 2025 18:24
@mergify mergify bot removed the one-approval PR has one approval from a maintainer label Jan 28, 2025
@mergify mergify bot merged commit a833900 into instructlab:main Jan 28, 2025
29 checks passed
@github-actions
Copy link

github-actions bot commented Jan 28, 2025

e2e workflow succeeded on this PR: View run, congrats!

kdubois pushed a commit to kdubois/instructlab that referenced this pull request Jan 29, 2025
@fabiendupont @kdubois
chore!: Update vLLM to 0.6.4.post1
295e577 
As an intermediate step to updating vLLM to 0.6.6.post1 in instructlab#2892, this
pull request updates vLLM to 0.6.4.post1 to ensure that the CI tests
succeed with this version too.

Signed-off-by: Fabien Dupont <fdupont@redhat.com>
Signed-off-by: Kevin Dubois <kevin.dubois@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

2 more reviewers

@nathan-weinberg nathan-weinberg nathan-weinberg approved these changes

@alinaryan alinaryan alinaryan approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

dependencies Relates to dependencies documentation Improvements or additions to documentation

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

outdated (0.62.) vllm causes dependency hell

6 participants

@fabiendupont @nathan-weinberg @bbrowning @n1hility @alinaryan @cdoern
Morty Proxy This is a proxified and sanitized view of the page, visit original site.