You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The instructlab userbase will have hardware that require various training configuration options
e.g. to train on colab I needed to disable bf16 and set
gradient_checkpointing=true
gradient_accumulation_steps=8
we should add a hidden, (perhaps temporary) argument to allow users to try various training options without
having to change the code
this could eventual be replaced with a happy set of arguments we know would cover all the cases
or code to auto sense the HW and set options based on what it finds.