Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Comments

Close side panel

[hf_adapter] added GenerationMixin for transformers version compatibility#418

Merged
kcirred merged 1 commit intofoundation-model-stack:mainfoundation-model-stack/foundation-model-stack:mainfrom
kcirred:generation_mixinkcirred/foundation-model-stack:generation_mixinCopy head branch name to clipboard
May 28, 2025
Merged

[hf_adapter] added GenerationMixin for transformers version compatibility#418
kcirred merged 1 commit intofoundation-model-stack:mainfoundation-model-stack/foundation-model-stack:mainfrom
kcirred:generation_mixinkcirred/foundation-model-stack:generation_mixinCopy head branch name to clipboard

Conversation

@kcirred
Copy link
Collaborator

@kcirred kcirred commented May 27, 2025

Fix for the following:

HFDecoderModelArchitecture has generative capabilities, as `prepare_inputs_for_generation` is explicitly defined. However, it doesn't directly inherit from `GenerationMixin`. From 馃憠v4.50馃憟 onwards, `PreTrainedModel` will NOT inherit from `GenerationMixin`, and this model will lose the ability to call `generate` and other related functions.

>>> transformers.__version__
'4.52.0.dev0'

AttributeError: 'HFAdaptedLLaMAForCausalLM' object has no attribute 'generate'

@kcirred
[hf_adapter] added GenerationMixin for transformers compatibility
4e28d2e 
Signed-off-by: kcirred <16872435+kcirred@users.noreply.github.com>
@kcirred kcirred requested a review from JRosenkranz May 27, 2025 22:46
@kcirred kcirred self-assigned this May 27, 2025
@kcirred kcirred changed the title [hf_adapter] added GenerationMixin for transformers compatibility [hf_adapter] added GenerationMixin for transformers version compatibility May 27, 2025
@JRosenkranz
Copy link
Collaborator

JRosenkranz commented May 28, 2025

Is this the case with all generative models, they now need to inherit from GenerationMixin explicitly to get generate?

@kcirred
Copy link
Collaborator Author

kcirred commented May 28, 2025

Is this the case with all generative models, they now need to inherit from GenerationMixin explicitly to get generate?

yes

JRosenkranz
JRosenkranz approved these changes May 28, 2025
View reviewed changes
Copy link
Collaborator

@JRosenkranz JRosenkranz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@kcirred kcirred merged commit 1b1c275 into foundation-model-stack:main May 28, 2025
4 checks passed
@kcirred kcirred deleted the generation_mixin branch May 28, 2025 17:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@JRosenkranz JRosenkranz JRosenkranz approved these changes

Assignees

@kcirred kcirred

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@kcirred @JRosenkranz
Morty Proxy This is a proxified and sanitized view of the page, visit original site.