Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Commit 3a10cec

Browse filesBrowse files
committed
add option type
1 parent 34bc55f commit 3a10cec
Copy full SHA for 3a10cec

File tree

Expand file treeCollapse file tree

1 file changed

+4
-1
lines changed
Filter options
Expand file treeCollapse file tree

1 file changed

+4
-1
lines changed

‎convert.py

Copy file name to clipboardExpand all lines: convert.py
+4-1Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@
2323
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
2424
from dataclasses import dataclass
2525
from pathlib import Path
26-
from typing import TYPE_CHECKING, Any, Callable, ClassVar, IO, Iterable, Literal, Protocol, TypeVar, runtime_checkable
26+
from typing import TYPE_CHECKING, Any, Callable, ClassVar, IO, Iterable, Literal, Protocol, TypeVar, runtime_checkable, Optional
2727

2828
import numpy as np
2929
from sentencepiece import SentencePieceProcessor
@@ -336,6 +336,7 @@ def load(model_plus: ModelPlus) -> Params:
336336

337337
return params
338338

339+
339340
@dataclass
340341
class Metadata:
341342
name: Optional[str] = None
@@ -370,10 +371,12 @@ def load(metadata_path: Path) -> "Metadata":
370371

371372
return metadata
372373

374+
373375
#
374376
# vocab
375377
#
376378

379+
377380
@runtime_checkable
378381
class BaseVocab(Protocol):
379382
tokenizer_model: ClassVar[str]

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.