Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Training options only allow for well known/tested HW #1007

Copy link
Copy link
@derekhiggins

Description

@derekhiggins
Issue body actions

The instructlab userbase will have hardware that require various training configuration options

e.g. to train on colab I needed to disable bf16 and set
gradient_checkpointing=true
gradient_accumulation_steps=8

we should add a hidden, (perhaps temporary) argument to allow users to try various training options without
having to change the code

this could eventual be replaced with a happy set of arguments we know would cover all the cases
or code to auto sense the HW and set options based on what it finds.

Reactions are currently unavailable

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

    Morty Proxy This is a proxified and sanitized view of the page, visit original site.