Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Do not treat an unreachable Hub as a missing model in 16bit merges#950

Open
danielhanchen wants to merge 1 commit into
mainunslothai/unsloth-zoo:mainfrom
fix-silent-noop-merge-exportunslothai/unsloth-zoo:fix-silent-noop-merge-exportCopy head branch name to clipboard
Open

Do not treat an unreachable Hub as a missing model in 16bit merges#950
danielhanchen wants to merge 1 commit into
mainunslothai/unsloth-zoo:mainfrom
fix-silent-noop-merge-exportunslothai/unsloth-zoo:fix-silent-noop-merge-exportCopy head branch name to clipboard

Conversation

@danielhanchen

Copy link
Copy Markdown
Member

What breaks

save_pretrained_merged(..., save_method = "merged_16bit") can return None, create no output directory at all, and report the problem only through a UserWarning:

UserWarning: Model unsloth/Llama-3.2-1B-Instruct not found locally or on HuggingFace

In a notebook that scrolls past. The user believes the 16bit export succeeded, and finds out later that nothing was written. The trigger is not a missing model, it is a Hub that was briefly unreachable: a 429, a 5xx, a DNS or proxy failure, a read timeout, or HF_HUB_OFFLINE.

Root cause

unsloth_zoo/saving_utils.py:check_hf_model_exists used a bare except::

def check_hf_model_exists(model_name, token=None):
    """Check if model exists on HuggingFace"""
    try:
        file_list = HfFileSystem(token=token).ls(model_name, detail=True)
        return any(x["name"].endswith(".safetensors") for x in file_list)
    except:
        return False

That maps every possible failure onto "this repo does not exist". From there:

  1. determine_base_model_source (saving_utils.py:4697) calls it, gets False, falls through all five priorities and returns (None, False, "", False, None).
  2. merge_and_overwrite_lora (saving_utils.py:2925) sees final_model_name is None and answers warnings.warn(...) plus return None.

No directory is created, no exception is raised, and the return value is None.

Why the two cases can be told apart

HfFileSystem already separates them, which is what makes a narrow fix possible rather than a guess. hf_file_system._raise_file_not_found (which raises a plain FileNotFoundError) is reached only after _repo_and_revision_exists catches RepositoryNotFoundError, RevisionNotFoundError or HFValidationError. Every other failure propagates out of ls untouched.

Confirmed empirically against huggingface_hub 1.24.0:

scenario what HfFileSystem.ls raises
missing repo builtins.FileNotFoundError: ... (repository not found)
HF_HUB_OFFLINE=1 huggingface_hub.errors.OfflineModeIsEnabled
public repo, online returns normally

Worth flagging because it is not the obvious answer: a genuinely missing repo surfaces as a plain FileNotFoundError, not as RepositoryNotFoundError. fsspec converts it. So FileNotFoundError has to be in the "absent" set, and including it does not swallow transport errors, because transport errors never reach that conversion.

The fix

Narrow the except to genuinely absent or inaccessible repos, and re-raise everything else as a RuntimeError that names the real cause:

_HUB_ABSENT_ERRORS = (
    FileNotFoundError,
    RepositoryNotFoundError,
    RevisionNotFoundError,
    EntryNotFoundError,
    GatedRepoError,
)

and turn the silent return None in merge_and_overwrite_lora into a RuntimeError that states plainly that nothing was written.

huggingface_hub>=0.34.0 is already a hard dependency, so all four error classes are guaranteed to import.

Behaviour change, stated explicitly

This turns a warning into an exception, and that is a behaviour change. Callers that currently swallow the UserWarning will now see a RuntimeError where they previously saw None.

The argument for doing it anyway: a None return that writes no files is never useful to a caller. There is no recovery a caller can perform with it that it could not perform with an exception, and the current behaviour converts an infrastructure blip into a silently lost training run. Failing loudly is strictly more informative here.

The honest cost: any code path that calls save_pretrained_merged inside a try/except that only expected a None return, or that checks the return value rather than catching, will now raise instead. A script that looped over many models and tolerated individual failures by testing for None will now stop at the first one. That is a real migration cost for a small number of callers, and it is the reason this is a separate PR rather than folded into something else.

Note also that a gated repo still returns False rather than raising. That preserves today's behaviour for gated models, where falling through to a local copy is usually what the user wants.

Measured before and after

New regression test, run against the same tree with and without the fix:

################ VARIANT: pristine ################
FAILED tests/test_check_hf_model_exists_transport_errors.py::test_transport_failure_raises_instead_of_reporting_absent[rate-limited-429]
FAILED tests/test_check_hf_model_exists_transport_errors.py::test_transport_failure_raises_instead_of_reporting_absent[server-error-503]
FAILED tests/test_check_hf_model_exists_transport_errors.py::test_transport_failure_raises_instead_of_reporting_absent[dns-failure]
FAILED tests/test_check_hf_model_exists_transport_errors.py::test_transport_failure_raises_instead_of_reporting_absent[read-timeout]
FAILED tests/test_check_hf_model_exists_transport_errors.py::test_transport_failure_raises_instead_of_reporting_absent[proxy-error]
FAILED tests/test_check_hf_model_exists_transport_errors.py::test_transport_failure_raises_instead_of_reporting_absent[hf-hub-offline]
FAILED tests/test_check_hf_model_exists_transport_errors.py::test_transport_failure_does_not_return_false
FAILED tests/test_check_hf_model_exists_transport_errors.py::test_no_warning_is_used_to_report_an_unreachable_hub
FAILED tests/test_check_hf_model_exists_transport_errors.py::test_determine_base_model_source_propagates_the_transport_error
9 failed, 7 passed, 14 warnings in 0.12s
################ VARIANT: patched ################
16 passed, 14 warnings in 0.05s

The 7 that pass both ways are the controls: the absent-repo cases that must keep returning False, and the success paths.

Existing suites, same tree with and without the fix, identical either way:

tests/test_saving_utils_lora_remap_count.py tests/test_saving_utils_quant_aware_merge.py
tests/test_assert_same_keys_base_model.py tests/test_dora_merge.py
tests/test_unsloth_zoo_lora_merge.py
-> 73 passed   (both pristine and patched)

Tests added

tests/test_check_hf_model_exists_transport_errors.py. Every test monkeypatches HfFileSystem.ls, so nothing touches the network.

  • Six transport failures (429, 503, DNS, read timeout, proxy, OfflineModeIsEnabled) must raise RuntimeError, name connectivity or rate limiting, and keep the original exception as __cause__.
  • A blunt separate assertion that a 429 does not come back as False, since that single step is what makes the export silently no-op.
  • Four absent cases (fsspec FileNotFoundError, RepositoryNotFoundError, RevisionNotFoundError, GatedRepoError) must still return False.
  • Both success paths unchanged (safetensors present -> True, absent -> False).
  • A warning is not accepted as the reporting channel for an unreachable Hub.
  • determine_base_model_source propagates the transport error instead of producing the (None, ...) that caused the silent no-op, and still returns (None, ...) for a genuinely absent repo with no local copy.

`check_hf_model_exists` wrapped its `HfFileSystem(...).ls(...)` in a bare
`except:` returning False, which mapped a 429, a 5xx, a DNS or proxy
failure, a read timeout and `HF_HUB_OFFLINE` all onto "this repo does not
exist". `determine_base_model_source` then fell through every priority and
returned `(None, ...)`, and `merge_and_overwrite_lora` answered a bare
`warnings.warn` plus `return None`.

The user visible result was that `save_pretrained_merged` returned None
and created no output directory at all, with the only signal a UserWarning
that scrolls past in a notebook. The user believed the 16bit export had
succeeded, and the cost was a whole training run.

Telling the two cases apart is possible because HfFileSystem already
separates them. `hf_file_system._raise_file_not_found`, which produces a
plain FileNotFoundError, is reached only after `_repo_and_revision_exists`
catches RepositoryNotFoundError, RevisionNotFoundError or
HFValidationError, while every transport failure propagates out of `ls`
untouched. Confirmed empirically against huggingface_hub 1.24.0: a missing
repo raises FileNotFoundError, and HF_HUB_OFFLINE raises
OfflineModeIsEnabled.

Narrow the except to genuinely absent or inaccessible repos and re-raise
everything else as a RuntimeError that names connectivity or rate limiting
as the cause. Turn the silent `return None` in the merge into a
RuntimeError that states plainly that nothing was written.

This is a behaviour change: a caller that currently swallows the warning
now sees an exception. That is the point. A None return that writes no
files is never useful to a caller, and staying silent here costs a
training run.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 96ed9577b5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +4456 to +4457
except Exception as e:
raise RuntimeError(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Skip the Hub probe when the base model is local

When model_name is a valid local directory such as base or ./base, HfFileSystem.ls can raise ValueError/HfUriError because the path is not a Hub repo ID, and this new handler turns that into a RuntimeError. determine_base_model_source calls this function before check_local_model_exists (lines 4699–4700), so the merge never reaches its documented local-model priorities; absolute local paths are likewise blocked whenever the Hub is offline or rate-limited. Check for the local source before probing the Hub, or allow this exception to propagate only after local resolution fails.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Morty Proxy This is a proxified and sanitized view of the page, visit original site.