From fcc7df6f12d50bc5cd40835549944ebe357095ab Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 3 May 2021 01:00:26 -0700 Subject: [PATCH 01/18] add CONTRIBUTING.md (#806) --- CONTRIBUTING.md | 83 +++++++++++++++++++++++++++++++++++++++++++++++++ README.md | 40 ++---------------------- 2 files changed, 86 insertions(+), 37 deletions(-) create mode 100644 CONTRIBUTING.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..14d7b216 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,83 @@ +This repository contains backports of the CPython `typing` module to earlier versions of +Python. Therefore, code in this repo should follow CPython's style guidelines and +contributors need to sign the PSF Contributor Agreement. + +# typing + +The `typing` module provided by this repository is a backport for Python versions that +do not have `typing` in the standard library: Python 2.7 and 3.4. These versions are no +longer officially supported by CPython, so there is little remaining interest in keeping +the backport up to date. We will accept contributions backporting new features to +`typing`, but we are no longer actively requiring Python 2 support for all +contributions. + +# typing_extensions + +The `typing_extensions` module provides a way to access new features from the standard +library `typing` module in older versions of Python. For example, Python 3.10 adds +`typing.TypeGuard`, but users of older versions of Python can use `typing_extensions` to +use `TypeGuard` in their code even if they are unable to upgrade to Python 3.10. + +If you contribute the runtime implementation of a new `typing` feature to CPython, you +are encouraged to also implement the feature in `typing_extensions`. Because the runtime +implementation of much of the infrastructure in the `typing` module has changed over +time, this may require different code for some older Python versions. + +`typing_extensions` may also include experimental features that are not yet part of the +standard library, so that users can experiment with them before they are added to the +standard library. Such features should ideally already be specified in a PEP or draft +PEP. + +`typing_extensions` still supports all Python versions supported by `typing`, down to +Python 2.7 and 3.4. However, it is OK to omit support for Python versions that have +reached end of life if doing so is too difficult or otherwise does not make sense. For +example, `typing_extensions.AsyncGenerator` only exists on Python 3.6 and higher, +because async generators were added to the language in 3.6. + +# Versioning scheme + +`typing_extensions` and `typing` are usually released together using the same version +numbers. The version number indicates the version of the standard library `typing` +module that is reflected in the backport. For example, `typing_extensions` version +3.10.0.0 includes features from the Python 3.10.0 standard library's `typing` module. A +new release that doesn't include any new standard library features would be called +3.10.0.1. + +# Workflow for PyPI releases + +- Do this for both `typing` and `typing_extensions` + +- Run tests under all supported versions. As of April 2021 this includes 2.7, 3.4, 3.5, + 3.6, 3.7, 3.8, 3.9. + +- On macOS, you can use `pyenv `\_ to manage multiple + Python installations. Long story short: + + - `xcode-select --install` + - `brew install pyenv` + - `echo 'eval "$(pyenv init -)"' >> ~/.bash_profile` + - Open a new shell + - `pyenv install 3.5.3` + - `pyenv install 3.4.6` + - (assuming you already have 2.7.13 and 3.6.1 from Homebrew) + - `pyenv global system 3.5.3 3.4.6` + - (or some more recent versions) + +- You can use `tox` to automate running tests. + +- Update the version number in `setup.py`. + +- Build the source and wheel distributions: + + - `pip3 install -U setuptools wheel` + - `pip2 install -U setuptools wheel` + - `rm -rf dist/ build/` + - `python3 setup.py sdist bdist_wheel` + - `rm -rf build/` (Works around + `a Wheel bug `\_) + - `python2 setup.py bdist_wheel` + +- Install the built distributions locally and test (if you were using `tox`, you already + tested the source distribution). + +- Make sure twine is up to date, then run `twine upload dist/*`. diff --git a/README.md b/README.md index 90b345cf..a1df5e21 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ This GitHub repo is used for three separate things: [typing-sig](https://mail.python.org/mailman3/lists/typing-sig.python.org/) is more appropriate these days. -- A copy of the `typing` module for older Python versions (2.7 and +- A backport of the `typing` module for older Python versions (2.7 and 3.4) is maintained here. Note that the canonical source lives [upstream](https://github.com/python/cpython/blob/master/Lib/typing.py) in the CPython repo. @@ -20,42 +20,8 @@ This GitHub repo is used for three separate things: Workflow -------- +* See [CONTRIBUTING.md](/CONTRIBUTING.md) for more. + * The typing.py module and its unittests are edited in the `src` subdirectory of this repo. The `python2` subdirectory contains the Python 2 backport. - -Workflow for PyPI releases --------------------------- - -* Run tests under all supported versions. As of April 2021 this includes - 2.7, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9. - -* On macOS, you can use `pyenv `_ to - manage multiple Python installations. Long story short: - - * ``xcode-select --install`` - * ``brew install pyenv`` - * ``echo 'eval "$(pyenv init -)"' >> ~/.bash_profile`` - * Open a new shell - * ``pyenv install 3.5.3`` - * ``pyenv install 3.4.6`` - * (assuming you already have 2.7.13 and 3.6.1 from Homebrew) - * ``pyenv global system 3.5.3 3.4.6`` - -* You can use ``tox`` to automate running tests. - -* Update the version number in ``setup.py``. - -* Build the source and wheel distributions: - - * ``pip3 install -U setuptools wheel`` - * ``pip2 install -U setuptools wheel`` - * ``rm -rf dist/ build/`` - * ``python3 setup.py sdist bdist_wheel`` - * ``rm -rf build/`` (Works around `a Wheel bug `_) - * ``python2 setup.py bdist_wheel`` - -* Install the built distributions locally and test (if you - were using ``tox``, you already tested the source distribution). - -* Make sure twine is up to date, then run ``twine upload dist/*``. From c4191ac15aa54c8269ccf2e6292b4a9cc928359e Mon Sep 17 00:00:00 2001 From: Bas van Beek <43369155+BvB93@users.noreply.github.com> Date: Tue, 4 May 2021 22:12:45 +0200 Subject: [PATCH 02/18] Add a missing comma to `__all__` (#808) * BUG: Added a missing comma in `__all__` * TST: Add a test to ensure that all objects in `__all__` are present in the module --- typing_extensions/src_py3/test_typing_extensions.py | 4 ++++ typing_extensions/src_py3/typing_extensions.py | 2 +- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/typing_extensions/src_py3/test_typing_extensions.py b/typing_extensions/src_py3/test_typing_extensions.py index e2889ce0..3f3c2f9e 100644 --- a/typing_extensions/src_py3/test_typing_extensions.py +++ b/typing_extensions/src_py3/test_typing_extensions.py @@ -2191,6 +2191,10 @@ def test_typing_extensions_includes_standard(self): self.assertIn('Protocol', a) self.assertIn('runtime', a) + # Check that all objects in `__all__` are present in the module + for name in a: + self.assertTrue(hasattr(typing_extensions, name)) + def test_typing_extensions_defers_when_possible(self): exclude = { 'overload', diff --git a/typing_extensions/src_py3/typing_extensions.py b/typing_extensions/src_py3/typing_extensions.py index 82d1c2dc..433b15fe 100644 --- a/typing_extensions/src_py3/typing_extensions.py +++ b/typing_extensions/src_py3/typing_extensions.py @@ -136,7 +136,7 @@ def _check_methods_in_mro(C, *methods): 'Counter', 'Deque', 'DefaultDict', - 'OrderedDict' + 'OrderedDict', 'TypedDict', # Structural checks, a.k.a. protocols. From 2de0a9324080da3ce9b1445b3729248a6f1cee53 Mon Sep 17 00:00:00 2001 From: Shannon Zhu Date: Thu, 3 Jun 2021 18:53:01 -0700 Subject: [PATCH 03/18] Initial sphinx docs setup (#814) Run Sphinx initialize to set up structure for Python typing documentation. See typing-sig discussion: https://mail.python.org/archives/list/typing-sig@python.org/thread/4E7V7MVO4FQVYPVELMCAYFEIYJRXDSY2/ Python docs community issue: python/docs-community#8 --- docs/Makefile | 20 +++++++++++++++++++ docs/conf.py | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++ docs/index.rst | 20 +++++++++++++++++++ docs/make.bat | 35 +++++++++++++++++++++++++++++++++ 4 files changed, 127 insertions(+) create mode 100644 docs/Makefile create mode 100644 docs/conf.py create mode 100644 docs/index.rst create mode 100644 docs/make.bat diff --git a/docs/Makefile b/docs/Makefile new file mode 100644 index 00000000..d4bb2cbb --- /dev/null +++ b/docs/Makefile @@ -0,0 +1,20 @@ +# Minimal makefile for Sphinx documentation +# + +# You can set these variables from the command line, and also +# from the environment for the first two. +SPHINXOPTS ?= +SPHINXBUILD ?= sphinx-build +SOURCEDIR = . +BUILDDIR = _build + +# Put it first so that "make" without argument is like "make help". +help: + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + +.PHONY: help Makefile + +# Catch-all target: route all unknown targets to Sphinx using the new +# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). +%: Makefile + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/docs/conf.py b/docs/conf.py new file mode 100644 index 00000000..70960ded --- /dev/null +++ b/docs/conf.py @@ -0,0 +1,52 @@ +# Configuration file for the Sphinx documentation builder. +# +# This file only contains a selection of the most common options. For a full +# list see the documentation: +# https://www.sphinx-doc.org/en/master/usage/configuration.html + +# -- Path setup -------------------------------------------------------------- + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# +# import os +# import sys +# sys.path.insert(0, os.path.abspath('.')) + + +# -- Project information ----------------------------------------------------- + +project = 'typing' +copyright = '2021, The Python Typing Team' +author = 'The Python Typing Team' + + +# -- General configuration --------------------------------------------------- + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = [ +] + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +# This pattern also affects html_static_path and html_extra_path. +exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] + + +# -- Options for HTML output ------------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +# +html_theme = 'alabaster' + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = ['_static'] diff --git a/docs/index.rst b/docs/index.rst new file mode 100644 index 00000000..f08f4cf9 --- /dev/null +++ b/docs/index.rst @@ -0,0 +1,20 @@ +.. typing documentation master file, created by + sphinx-quickstart on Mon May 24 16:43:52 2021. + You can adapt this file completely to your liking, but it should at least + contain the root `toctree` directive. + +Welcome to the Python Type System documentation! +================================================ + +.. toctree:: + :maxdepth: 2 + :caption: Contents: + + + +Indices and tables +================== + +* :ref:`genindex` +* :ref:`modindex` +* :ref:`search` diff --git a/docs/make.bat b/docs/make.bat new file mode 100644 index 00000000..2119f510 --- /dev/null +++ b/docs/make.bat @@ -0,0 +1,35 @@ +@ECHO OFF + +pushd %~dp0 + +REM Command file for Sphinx documentation + +if "%SPHINXBUILD%" == "" ( + set SPHINXBUILD=sphinx-build +) +set SOURCEDIR=. +set BUILDDIR=_build + +if "%1" == "" goto help + +%SPHINXBUILD% >NUL 2>NUL +if errorlevel 9009 ( + echo. + echo.The 'sphinx-build' command was not found. Make sure you have Sphinx + echo.installed, then set the SPHINXBUILD environment variable to point + echo.to the full path of the 'sphinx-build' executable. Alternatively you + echo.may add the Sphinx directory to PATH. + echo. + echo.If you don't have Sphinx installed, grab it from + echo.http://sphinx-doc.org/ + exit /b 1 +) + +%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% +goto end + +:help +%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% + +:end +popd From a1143792004b6e2e5482f76f419989d9c63a0aea Mon Sep 17 00:00:00 2001 From: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com> Date: Wed, 16 Jun 2021 20:50:57 +0800 Subject: [PATCH 04/18] Support most use cases for PEP 612 with Generic (#817) --- typing_extensions/README.rst | 4 +- .../src_py3/test_typing_extensions.py | 55 ++++++++++++++++--- .../src_py3/typing_extensions.py | 44 +++++++++++++-- 3 files changed, 87 insertions(+), 16 deletions(-) diff --git a/typing_extensions/README.rst b/typing_extensions/README.rst index 98f62135..4166510a 100644 --- a/typing_extensions/README.rst +++ b/typing_extensions/README.rst @@ -90,8 +90,8 @@ issues when mixing the differing implementations of modified classes. Certain types have incorrect runtime behavior due to limitations of older versions of the typing module. For example, ``ParamSpec`` and ``Concatenate`` -will not work with ``get_args``, ``get_origin`` or user-defined ``Generic``\ s -because they need to be lists to work with older versions of ``Callable``. +will not work with ``get_args``, ``get_origin``. Certain PEP 612 special cases +in user-defined ``Generic``\ s are also not available. These types are only guaranteed to work for static type checking. Running tests diff --git a/typing_extensions/src_py3/test_typing_extensions.py b/typing_extensions/src_py3/test_typing_extensions.py index 3f3c2f9e..06d4cc40 100644 --- a/typing_extensions/src_py3/test_typing_extensions.py +++ b/typing_extensions/src_py3/test_typing_extensions.py @@ -2012,25 +2012,38 @@ def test_valid_uses(self): P = ParamSpec('P') T = TypeVar('T') C1 = typing.Callable[P, int] + # Callable in Python 3.5.2 might be bugged when collecting __args__. + # https://github.com/python/cpython/blob/91185fe0284a04162e0b3425b53be49bdbfad67d/Lib/typing.py#L1026 + PY_3_5_2 = sys.version_info[:3] == (3, 5, 2) + if not PY_3_5_2: + self.assertEqual(C1.__args__, (P, int)) + self.assertEqual(C1.__parameters__, (P,)) C2 = typing.Callable[P, T] + if not PY_3_5_2: + self.assertEqual(C2.__args__, (P, T)) + self.assertEqual(C2.__parameters__, (P, T)) - # Note: no tests for Callable.__args__ and Callable.__parameters__ here - # because pre-3.10 Callable sees ParamSpec as a plain list, not a - # TypeVar. # Test collections.abc.Callable too. if sys.version_info[:2] >= (3, 9): + # Note: no tests for Callable.__parameters__ here + # because types.GenericAlias Callable is hardcoded to search + # for tp_name "TypeVar" in C. This was changed in 3.10. C3 = collections.abc.Callable[P, int] + self.assertEqual(C3.__args__, (P, int)) C4 = collections.abc.Callable[P, T] + self.assertEqual(C4.__args__, (P, T)) # ParamSpec instances should also have args and kwargs attributes. - self.assertIn('args', dir(P)) - self.assertIn('kwargs', dir(P)) + # Note: not in dir(P) because of __class__ hacks + self.assertTrue(hasattr(P, 'args')) + self.assertTrue(hasattr(P, 'kwargs')) def test_args_kwargs(self): P = ParamSpec('P') - self.assertIn('args', dir(P)) - self.assertIn('kwargs', dir(P)) + # Note: not in dir(P) because of __class__ hacks + self.assertTrue(hasattr(P, 'args')) + self.assertTrue(hasattr(P, 'kwargs')) self.assertIsInstance(P.args, ParamSpecArgs) self.assertIsInstance(P.kwargs, ParamSpecKwargs) self.assertIs(P.args.__origin__, P) @@ -2038,8 +2051,32 @@ def test_args_kwargs(self): self.assertEqual(repr(P.args), "P.args") self.assertEqual(repr(P.kwargs), "P.kwargs") - # Note: ParamSpec doesn't work for pre-3.10 user-defined Generics due - # to type checks inside Generic. + def test_user_generics(self): + T = TypeVar("T") + P = ParamSpec("P") + P_2 = ParamSpec("P_2") + + class X(Generic[T, P]): + pass + + G1 = X[int, P_2] + self.assertEqual(G1.__args__, (int, P_2)) + self.assertEqual(G1.__parameters__, (P_2,)) + + G2 = X[int, Concatenate[int, P_2]] + self.assertEqual(G2.__args__, (int, Concatenate[int, P_2])) + self.assertEqual(G2.__parameters__, (P_2,)) + + # The following are some valid uses cases in PEP 612 that don't work: + # These do not work in 3.9, _type_check blocks the list and ellipsis. + # G3 = X[int, [int, bool]] + # G4 = X[int, ...] + # G5 = Z[[int, str, bool]] + # Not working because this is special-cased in 3.10. + # G6 = Z[int, str, bool] + + class Z(Generic[P]): + pass def test_pickle(self): global P, P_co, P_contra diff --git a/typing_extensions/src_py3/typing_extensions.py b/typing_extensions/src_py3/typing_extensions.py index 433b15fe..e4d16440 100644 --- a/typing_extensions/src_py3/typing_extensions.py +++ b/typing_extensions/src_py3/typing_extensions.py @@ -2329,6 +2329,9 @@ def add_two(x: float, y: float) -> float: be pickled. """ + # Trick Generic __parameters__. + __class__ = TypeVar + @property def args(self): return ParamSpecArgs(self) @@ -2377,14 +2380,31 @@ def __reduce__(self): def __call__(self, *args, **kwargs): pass - # Note: Can't fake ParamSpec as a TypeVar to get it to work - # with Generics. ParamSpec isn't an instance of TypeVar in 3.10. - # So encouraging code like isinstance(ParamSpec('P'), TypeVar)) - # will lead to breakage in 3.10. - # This also means no accurate __parameters__ for GenericAliases. + if not PEP_560: + # Only needed in 3.6 and lower. + def _get_type_vars(self, tvars): + if self not in tvars: + tvars.append(self) # Inherits from list as a workaround for Callable checks in Python < 3.9.2. class _ConcatenateGenericAlias(list): + + # Trick Generic into looking into this for __parameters__. + if PEP_560: + __class__ = _GenericAlias + elif sys.version_info[:3] == (3, 5, 2): + __class__ = typing.TypingMeta + else: + __class__ = typing._TypingBase + + # Flag in 3.8. + _special = False + # Attribute in 3.6 and earlier. + if sys.version_info[:3] == (3, 5, 2): + _gorg = typing.GenericMeta + else: + _gorg = typing.Generic + def __init__(self, origin, args): super().__init__(args) self.__origin__ = origin @@ -2399,6 +2419,20 @@ def __repr__(self): def __hash__(self): return hash((self.__origin__, self.__args__)) + # Hack to get typing._type_check to pass in Generic. + def __call__(self, *args, **kwargs): + pass + + @property + def __parameters__(self): + return tuple(tp for tp in self.__args__ if isinstance(tp, (TypeVar, ParamSpec))) + + if not PEP_560: + # Only required in 3.6 and lower. + def _get_type_vars(self, tvars): + if self.__origin__ and self.__parameters__: + typing._get_type_vars(self.__parameters__, tvars) + @_tp_cache def _concatenate_getitem(self, parameters): if parameters == (): From 0fc0eda1e6b415811d4f94f1af0c90543db48485 Mon Sep 17 00:00:00 2001 From: Shannon Zhu Date: Sun, 27 Jun 2021 10:57:12 -0700 Subject: [PATCH 05/18] Add docs build options and theme requirements (#818) --- docs/Makefile | 32 ++++++++++++++++++++++++++++--- docs/README.rst | 44 +++++++++++++++++++++++++++++++++++++++++++ docs/conf.py | 2 +- docs/requirements.txt | 10 ++++++++++ 4 files changed, 84 insertions(+), 4 deletions(-) create mode 100644 docs/README.rst create mode 100644 docs/requirements.txt diff --git a/docs/Makefile b/docs/Makefile index d4bb2cbb..4f7c95a8 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -3,16 +3,42 @@ # You can set these variables from the command line, and also # from the environment for the first two. -SPHINXOPTS ?= -SPHINXBUILD ?= sphinx-build SOURCEDIR = . +SOURCES = BUILDDIR = _build +PYTHON = python3 +VENVDIR = ./venv +SPHINXBUILD = PATH=$(VENVDIR)/bin:$$PATH sphinx-build + +ALLSPHINXOPTS = -b $(BUILDER) -d build/doctrees $(SPHINXOPTS) . build/$(BUILDER) $(SOURCES) + +.PHONY: help clean build html text venv Makefile # Put it first so that "make" without argument is like "make help". help: @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) -.PHONY: help Makefile +clean: + -rm -rf build/* $(VENVDIR)/* + +build: + -mkdir -p build + $(SPHINXBUILD) $(ALLSPHINXOPTS) + @echo + +html: BUILDER = html +html: build + @echo "Build finished. The HTML pages are in build/html." + +text: BUILDER = text +text: build + @echo "Build finished; the text files are in build/text." + +venv: + $(PYTHON) -m venv $(VENVDIR) + $(VENVDIR)/bin/python3 -m pip install -U pip setuptools + $(VENVDIR)/bin/python3 -m pip install -r requirements.txt + @echo "The venv has been created in the $(VENVDIR) directory" # Catch-all target: route all unknown targets to Sphinx using the new # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). diff --git a/docs/README.rst b/docs/README.rst new file mode 100644 index 00000000..359fc031 --- /dev/null +++ b/docs/README.rst @@ -0,0 +1,44 @@ +Python Typing Documentation +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Building the docs +================= + +The documentation is built with tools which are not included in this +tree but are maintained separately and are available from +`PyPI `_. + +* `Sphinx `_ +* `python-docs-theme `_ + +The easiest way to install these tools is to create a virtual environment and +install the tools into there. + +Using make +---------- + +To get started on UNIX, you can create a virtual environment with the command :: + + make venv + +That will install all the tools necessary to build the documentation. Assuming +the virtual environment was created in the ``venv`` directory (the default; +configurable with the VENVDIR variable), you can run the following command to +build the HTML output files:: + + make html + +By default, if the virtual environment is not created, the Makefile will +look for instances of sphinxbuild and blurb installed on your process PATH +(configurable with the SPHINXBUILD and BLURB variables). + +Available make targets are: + +* "clean", which removes all build files. + +* "venv", which creates a virtual environment with all necessary tools + installed. + +* "html", which builds standalone HTML files for offline viewing. + +* "text", which builds a plain text file for each source file. diff --git a/docs/conf.py b/docs/conf.py index 70960ded..881c068d 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -44,7 +44,7 @@ # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. # -html_theme = 'alabaster' +html_theme = 'python_docs_theme' # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, diff --git a/docs/requirements.txt b/docs/requirements.txt new file mode 100644 index 00000000..35236bd9 --- /dev/null +++ b/docs/requirements.txt @@ -0,0 +1,10 @@ +# Requirements to build the Python documentation + +# Sphinx version is pinned so that new versions that introduce new warnings +# won't suddenly cause build failures. Updating the version is fine as long +# as no warnings are raised by doing so. +sphinx==3.2.1 + +# The theme used by the documentation is stored separately, so we need +# to install that as well. +python-docs-theme From f36dc8dbf19eb931d759f58f939fbb7f6d0ed764 Mon Sep 17 00:00:00 2001 From: Hugo van Kemenade Date: Wed, 18 Aug 2021 15:25:57 +0300 Subject: [PATCH 06/18] Replace deprecated unittest aliases (#836) --- src/test_typing.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/test_typing.py b/src/test_typing.py index bddad7bd..a987a8dc 100644 --- a/src/test_typing.py +++ b/src/test_typing.py @@ -1515,8 +1515,8 @@ def foo(a: c1_gth, b: c2_gth): self.assertEqual(List[c1], List[c1_gth]) self.assertNotEqual(List[c1], List[C]) self.assertNotEqual(List[c1_gth], List[C]) - self.assertEquals(Union[c1, c1_gth], Union[c1]) - self.assertEquals(Union[c1, c1_gth, int], Union[c1, int]) + self.assertEqual(Union[c1, c1_gth], Union[c1]) + self.assertEqual(Union[c1, c1_gth, int], Union[c1, int]) def test_forward_equality_hash(self): c1 = typing._ForwardRef('int') From 421e0a4672b75db6a2e91e2f85bf207808b5f138 Mon Sep 17 00:00:00 2001 From: Sebastian Rittau Date: Mon, 23 Aug 2021 19:23:06 +0200 Subject: [PATCH 07/18] Add docs/venv to .gitignore (#841) This directory is automatically created by "make -C docs venv". --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index 98a79f50..1997aade 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,7 @@ MANIFEST build/ dist/ +docs/venv/ .tox/ .vscode/ .idea/ From d7f07b71538db86e849c3391c26940adeb912d77 Mon Sep 17 00:00:00 2001 From: Sebastian Rittau Date: Mon, 23 Aug 2021 19:55:06 +0200 Subject: [PATCH 08/18] Add type stubs document (#844) This was originally intended to become a PEP and was written by: Sebastian Rittau Rebecca Chen Teddy Sudol Jelle Zijlstra Also, remove unnecessary modindex link from main index. --- docs/index.rst | 2 +- docs/stubs.rst | 1069 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 1070 insertions(+), 1 deletion(-) create mode 100644 docs/stubs.rst diff --git a/docs/index.rst b/docs/index.rst index f08f4cf9..97ab6bdf 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -10,11 +10,11 @@ Welcome to the Python Type System documentation! :maxdepth: 2 :caption: Contents: + stubs Indices and tables ================== * :ref:`genindex` -* :ref:`modindex` * :ref:`search` diff --git a/docs/stubs.rst b/docs/stubs.rst new file mode 100644 index 00000000..5e4a3681 --- /dev/null +++ b/docs/stubs.rst @@ -0,0 +1,1069 @@ +.. _stubs: + +********** +Type Stubs +********** + +Abstract +======== + +Optional type hints were introduced to the Python language in PEP 484 +[#pep484]_, based on the function annotation syntax from PEP 3107 +[#pep3107]_. Static type checkers can use type hints to prevent bugs, +documentation tools can automatically add type information, +and IDEs can offer improved autocompletion and support safer refactorings. + +PEP 484 also introduced *type stubs*, also called *stub files*, +that provide type information for untyped Python packages and modules. Type +stubs serve multiple purposes: + +* They are the only way to add type information to extension modules. +* They can provide type information for packages that do not wish to + add them inline. +* They can be distributed separately from the implementation. + This allows stubs to be developed at a different pace or by different + authors, which is especially useful when adding type annotations to + existing packages. +* They can act as documentation, succinctly explaining the external + API of a package, without including the implementation or private + members. + +This PEP aims to give guidance to both authors of type stubs and developers +of type checkers and other tools. It describes the constructs that can be used safely in type stubs, +suggests a style guide for them, and lists constructs that type +checkers are expected to support. + +Type stubs that only use constructs described in this PEP should work with +all type checkers that also follow this PEP. +Type stub authors can elect to use additional constructs, but +must be prepared that some type checkers will not parse them as expected. + +A type checker that conforms to this PEP will parse a type stub that only uses +constructs described here without error and will not interpret any +construct in a contradictory manner. However, type checkers are not +required to implement checks for all these constructs, and +can elect to ignore unsupported ones. Additionally type checkers +can support constructs not described in this PEP and tool authors are +encouraged to experiment with additional features. + +This PEP is intended as a living document and will be updated as new +features are supported or best practices evolve. + +Syntax +====== + +Type stubs are syntactically valid Python 3.7 files with a ``.pyi`` suffix. +The Python syntax used for type stubs is independent from the Python +versions supported by the implementation, and from the Python version the type +checker runs under (if any). Therefore, type stub authors should use the +latest available syntax features in stubs (up to Python 3.7), even if the +implementation supports older, pre-3.7 Python versions. +Type checker authors are encouraged to support syntax features from +post-3.7 Python versions, although type stub authors should not use such +features if they wish to maintain compatibility with all type checkers. + +For example, Python 3.7 added the ``async`` keyword (see PEP 492 [#pep492]_). +Stub authors should use it to mark coroutines, even if the implementation +still uses the ``@coroutine`` decorator. On the other hand, type stubs should +not use the positional-only syntax from PEP 570 [#pep570]_, introduced in +Python 3.8, although type checker authors are encouraged to support it. + +Stubs are treated as if ``from __future__ import annotations`` is enabled. +In particular, built-in generics and forward references can be used. + +Starting with Python 3.8, the ast_ module from the standard library supports +all syntax features required by this PEP. Older Python versions can use the +typed_ast_ package, which also supports Python 3.7 syntax and ``# type`` +comments. + +Distribution +============ + +Type stubs can be distributed with or separately from the implementation; +see PEP 561 [#pep561]_ for more information. The typeshed_ project +includes stubs for Python's standard library and several third-party +packages. These are usually distributed with type checkers and do not +require separate installation. + +Supported Constructs +==================== + +This sections lists constructs that type checkers will accept in type stubs. +Type stub authors can safely use these constructs. If a +construct is marked as "unspecified", type checkers may handle it +as they best see fit or report an error. Linters should usually +flag those constructs. Type stub authors should avoid using them to +ensure compatibility across type checkers. + +Unless otherwise mentioned, type stubs support all features from the +``typing`` module of the latest released Python version. If a stub uses +typing features from a later Python version than what the implementation +supports, these features can be imported from ``typing_extensions`` instead +of ``typing``. + +For example, a stub could use ``Literal``, introduced in Python 3.8, +for a library supporting Python 3.7+:: + + from typing_extensions import Literal + + def foo(x: Literal[""]) -> int: ... + +Comments +-------- + +Standard Python comments are accepted everywhere Python syntax allows them. + +Two kinds of structured comments are accepted: + +* A ``# type: X`` comment at the end of a line that defines a variable, + declaring that the variable has type ``X``. However, PEP 526-style [#pep526]_ + variable annotations are preferred over type comments. +* A ``# type: ignore`` comment at the end of any line, which suppresses all type + errors in that line. + +Imports +------- + +Type stubs distinguish between imports that are re-exported and those +that are only used internally. Imports are re-exported if they use one of these +forms:[#pep484]_ + +* ``import X as X`` +* ``from Y import X as X`` +* ``from Y import *`` + +Here are some examples of imports that make names available for internal use in +a stub but do not re-export them:: + + import X + from Y import X + from Y import X as OtherX + +Type aliases can be used to re-export an import under a different name:: + + from foo import bar as _bar + new_bar = _bar # "bar" gets re-exported with the name "new_bar" + +Sub-modules are always exported when they are imported in a module. +For example, consider the following file structure:: + + foo/ + __init__.pyi + bar.pyi + +Then ``foo`` will export ``bar`` when one of the following constructs is used in +``__init__.pyi``:: + + from . import bar + from .bar import Bar + +Stubs support customizing star import semantics by defining a module-level +variable called ``__all__``. In stubs, this must be a string list literal. +Other types are not supported. Neither is the dynamic creation of this +variable (for example by concatenation). + +By default, ``from foo import *`` imports all names in ``foo`` that +do not begin with an underscore. When ``__all__`` is defined, only those names +specified in ``__all__`` are imported:: + + __all__ = ['public_attr', '_private_looking_public_attr'] + + public_attr: int + _private_looking_public_attr: int + private_attr: int + +Type checkers can handle cyclic imports in stub files. + +Module Level Attributes +----------------------- + +Module level variables and constants can be annotated using either +type comments or variable annotation syntax:: + + x: int # recommended + x: int = 0 + x = 0 # type: int + x = ... # type: int + +The type of a variable is unspecified when the variable is unannotated or +when the annotation +and the assigned value disagree. As an exception, the ellipsis literal can +stand in for any type:: + + x = 0 # type is unspecified + x = ... # type is unspecified + x: int = "" # type is unspecified + x: int = ... # type is int + +Classes +------- + +Class definition syntax follows general Python syntax, but type checkers +are only expected to understand the following constructs in class bodies: + +* The ellipsis literal ``...`` is ignored and used for empty + class bodies. Using ``pass`` in class bodies is undefined. +* Instance attributes follow the same rules as module level attributes + (see above). +* Method definitions (see below) and properties. +* Method aliases. +* Inner class definitions. + +More complex statements don't need to be supported:: + + class Simple: ... + + class Complex(Base): + read_write: int + @property + def read_only(self) -> int: ... + def do_stuff(self, y: str) -> None: ... + doStuff = do_stuff + +The type of generic classes can be narrowed by annotating the ``self`` +argument of the ``__init__`` method:: + + class Foo(Generic[_T]): + @overload + def __init__(self: Foo[str], type: Literal["s"]) -> None: ... + @overload + def __init__(self: Foo[int], type: Literal["i"]) -> None: ... + @overload + def __init__(self, type: str) -> None: ... + +The class must match the class in which it is declared. Using other classes, +including sub or super classes, will not work. In addition, the ``self`` +annotation cannot contain type variables. + +Functions and Methods +--------------------- + +Function and method definition syntax follows general Python syntax. +Unless an argument name is prefixed with two underscores (but not suffixed +with two underscores), it can be used as a keyword argument [#pep484]_:: + + # x is positional-only + # y can be used positionally or as keyword argument + # z is keyword-only + def foo(__x, y, *, z): ... + +PEP 570 [#pep570]_ style positional-only parameters are currently not +supported. + +If an argument or return type is unannotated, per PEP 484 [#pep484]_ its +type is assumed to be ``Any``. It is preferred to leave unknown +types unannotated rather than explicitly marking them as ``Any``, as some +type checkers can optionally warn about unannotated arguments. + +If an argument has a literal or constant default value, it must match the implementation +and the type of the argument (if specified) must match the default value. +Alternatively, ``...`` can be used in place of any default value:: + + # The following arguments all have type Any. + def unannotated(a, b=42, c=...): ... + # The following arguments all have type int. + def annotated(a: int, b: int = 42, c: int = ...): ... + # The following default values are invalid and the types are unspecified. + def invalid(a: int = "", b: Foo = Foo()): ... + +For a class ``C``, the type of the first argument to a classmethod is +assumed to be ``Type[C]``, if unannotated. For other non-static methods, +its type is assumed to be ``C``:: + + class Foo: + def do_things(self): ... # self has type Foo + @classmethod + def create_it(cls): ... # cls has type Type[Foo] + @staticmethod + def utility(x): ... # x has type Any + +But:: + + _T = TypeVar("_T") + + class Foo: + def do_things(self: _T): ... # self has type _T + @classmethod + def create_it(cls: _T): ... # cls has type _T + +Using a function or method body other than the ellipsis literal is currently +unspecified. Stub authors may experiment with other bodies, but it is up to +individual type checkers how to interpret them. + + def foo(): ... # compatible + def bar(): pass # behavior undefined + +All variants of overloaded functions and methods must have an ``@overload`` +decorator:: + + @overload + def foo(x: str) -> str: ... + @overload + def foo(x: float) -> int: ... + +The following (which would be used in the implementation) is wrong in +type stubs:: + + @overload + def foo(x: str) -> str: ... + @overload + def foo(x: float) -> int: ... + def foo(x: Union[str, float]) -> Any: ... + +Aliases and NewType +------------------- + +Type checkers should accept module-level and class-level aliases, e.g.:: + + _IntList = list[int] + + class C: + def f(self) -> int: ... + g = f + +An alias to a type may contain type variables. As per PEP 484 [#pep484]_, +all type variables must be substituted when the alias is used:: + + _K = TypeVar("_K") + _V = TypeVar("_V") + _MyMap = Dict[str, Dict[_K, _V]] + + # either concrete types or other type variables can be substituted + def f(x: _MyMap[str, _V]) -> _V: ... + # explicitly substitute in Any rather than using a bare alias + def g(x: _MyMap[Any, Any]) -> Any: ... + +Otherwise, type variables in aliases follow the same rules as type variables in +generic class definitions. + +``typing.NewType`` is also supported in stubs. + +Decorators +---------- + +Type stubs may only use decorators defined in the ``typing`` module, plus a +fixed set of additional ones: + +* ``classmethod`` +* ``staticmethod`` +* ``property`` (including ``.setter``) +* ``abc.abstractmethod`` +* ``dataclasses.dataclass`` +* ``asyncio.coroutine`` (although ``async`` should be used instead) + +The behavior of other decorators should instead be incorporated into the types. +For example, for the following function:: + + import contextlib + @contextlib.contextmanager + def f(): + yield 42 + +the stub definition should be:: + + from contextlib import AbstractContextManager + def f() -> AbstractContextManager[int]: ... + +Version and Platform Checks +--------------------------- + +Type stubs for libraries that support multiple Python versions can use version +checks to supply version-specific type hints. Type stubs for different Python +versions should still conform to the most recent supported Python version's +syntax, as explain in the Syntax_ section above. + +Version checks are if-statements that use ``sys.version_info`` to determine the +current Python version. Version checks should only check against the ``major`` and +``minor`` parts of ``sys.version_info``. Type checkers are only required to +support the tuple-based version check syntax:: + + if sys.version_info >= (3,): + # Python 3-specific type hints. This tuple-based syntax is recommended. + else: + # Python 2-specific type hints. + + if sys.version_info >= (3, 5): + # Specific minor version features can be easily checked with tuples. + + if sys.version_info < (3,): + # This is only necessary when a feature has no Python 3 equivalent. + +Type stubs should avoid checking against ``sys.version_info.major`` +directly and should not use comparison operators other than ``<`` and ``>=``. + +No:: + + if sys.version_info.major >= 3: + # Semantically the same as the first tuple check. + + if sys.version_info[0] >= 3: + # This is also the same. + + if sys.version_info <= (2, 7): + # This does not work because e.g. (2, 7, 1) > (2, 7). + +Some type stubs also may need to specify type hints for different platforms. +Platform checks must be equality comparisons between ``sys.platform`` and the name +of a platform as a string literal: + +Yes:: + + if sys.platform == 'win32': + # Windows-specific type hints. + else: + # Posix-specific type hints. + +No:: + + if sys.platform.startswith('linux'): + # Not necessary since Python 3.3. + + if sys.platform in ['linux', 'cygwin', 'darwin']: + # Only '==' or '!=' should be used in platform checks. + +Version and platform comparisons can be chained using the ``and`` and ``or`` +operators:: + + if sys.platform == 'linux' and (sys.version_info < (3,) or sys,version_info >= (3, 7)): ... + +Enums +----- + +Enum classes are supported in stubs, regardless of the Python version targeted by +the stubs. + +Enum members may be specified just like other forms of assignments, for example as +``x: int``, ``x = 0``, or ``x = ...``. The first syntax is preferred because it +allows type checkers to correctly type the ``.value`` attribute of enum members, +without providing unnecessary information like the runtime value of the enum member. + +Additional properties on enum members should be specified with ``@property``, so they +do not get interpreted by type checkers as enum members. + +Yes:: + + from enum import Enum + + class Color(Enum): + RED: int + BLUE: int + @property + def rgb_value(self) -> int: ... + + class Color(Enum): + # discouraged; type checkers will not understand that Color.RED.value is an int + RED = ... + BLUE = ... + @property + def rgb_value(self) -> int: ... + +No:: + + from enum import Enum + + class Color(Enum): + RED: int + BLUE: int + rgb_value: int # no way for type checkers to know that this is not an enum member + +Unsupported Features +-------------------- + +Currently, positional-only argument syntax (PEP 570 [#pep570]_), +unions using the pipe operator (``|``) (PEP 604 [#pep604]_), +``ParamSpec`` (PEP 612 [#pep612]_), and ``TypeAlias`` (PEP 613 [#pep613]_) +are not supported by all type +checkers and should not be used in stubs. + +Type Stub Content +================= + +This section documents best practices on what elements to include or +leave out of type stubs. + +Public Interface +---------------- + +Stubs should include the complete public interface (classes, functions, +constants, etc.) of the module they cover, but it is not always +clear exactly what is part of the interface. + +The following should always be included: + +* All objects listed in the module's documentation. +* All objects included in ``__all__`` (if present). + +Other objects may be included if they are not prefixed with an underscore +or if they are being used in practice. (See the next section.) + +Undocumented Objects +-------------------- + +Undocumented objects may be included as long as they are marked with a comment +of the form ``# undocumented``. + +Example:: + + def list2cmdline(seq: Sequence[str]) -> str: ... # undocumented + +Such undocumented objects are allowed because omitting objects can confuse +users. Users who see an error like "module X has no attribute Y" will +not know whether the error appeared because their code had a bug or +because the stub is wrong. Although it may also be helpful for a type +checker to point out usage of private objects, we usually prefer false +negatives (no errors for wrong code) over false positives (type errors +for correct code). In addition, even for private objects a type checker +can be helpful in pointing out that an incorrect type was used. + +``__all__`` +------------ + +A type stub should contain an ``__all__`` variable if and only if it also +present at runtime. In that case, the contents of ``__all__`` should be +identical in the stub and at runtime. If the runtime dynamically adds +or removes elements (for example if certain functions are only available on +some platforms), include all possible elements in the stubs. + +Stub-Only Objects +----------------- + +Definitions that do not exist at runtime may be included in stubs to aid in +expressing types. Sometimes, it is desirable to make a stub-only class available +to a stub's users - for example, to allow them to type the return value of a +public method for which a library does not provided a usable runtime type:: + + from typing import Protocol + + class Readable(Protocol): + def read(self) -> str: ... + + def get_reader() -> Readable: ... + +Structural Types +---------------- + +As seen in the example with ``Readable`` in the previous section, a common use +of stub-only objects is to model types that are best described by their +structure. These objects are called protocols [#pep544]_, and it is encouraged +to use them freely to describe simple structural types. + +Incomplete Stubs +---------------- + +Partial stubs can be useful, especially for larger packages, but they should +follow the following guidelines: + +* Included functions and methods should list all arguments, but the arguments + can be left unannotated. +* Do not use ``Any`` to mark unannotated arguments or return values. +* Partial classes should include a ``__getattr__()`` method marked with an + ``# incomplete`` comment (see example below). +* Partial modules (i.e. modules that are missing some or all classes, + functions, or attributes) should include a top-level ``__getattr__()`` + function marked with an ``# incomplete`` comment (see example below). +* Partial packages (i.e. packages that are missing one or more sub-modules) + should have a ``__init__.pyi`` stub that is marked as incomplete (see above). + A better alternative is to create empty stubs for all sub-modules and + mark them as incomplete individually. + +Example of a partial module with a partial class ``Foo`` and a partially +annotated function ``bar()``:: + + def __getattr__(name: str) -> Any: ... # incomplete + + class Foo: + def __getattr__(self, name: str) -> Any: # incomplete + x: int + y: str + + def bar(x: str, y, *, z=...): ... + +Attribute Access +---------------- + +Python has several methods for customizing attribute access: ``__getattr__``, +``__getattribute__``, ``__setattr__``, and ``__delattr__``. Of these, +``__getattr__`` and ``__setattr___`` should sometimes be included in stubs. + +In addition to marking incomplete definitions, ``__getattr__`` should be +included when a class or module allows any name to be accessed. For example, consider +the following class:: + + class Foo: + def __getattribute__(self, name): + return self.__dict__.setdefault(name) + +An appropriate stub definition is:: + + from typing import Any, Optional + class Foo: + def __getattr__(self, name: str) -> Optional[Any]: ... + +Note that only ``__getattr__``, not ``__getattribute__``, is guaranteed to be +supported in stubs. + +On the other hand, ``__getattr__`` should be omitted even if the source code +includes it, if only limited names are allowed. For example, consider this class:: + + class ComplexNumber: + def __init__(self, n): + self._n = n + def __getattr__(self, name): + if name in ("real", "imag"): + return getattr(self._n, name) + raise AttributeError(name) + +In this case, the stub should list the attributes individually:: + + class ComplexNumber: + @property + def real(self) -> float: ... + @property + def imag(self) -> float: ... + def __init__(self, n: complex) -> None: ... + +``__setattr___`` should be included when a class allows any name to be set and +restricts the type. For example:: + + class IntHolder: + def __setattr__(self, name, value): + if isinstance(value, int): + return super().__setattr__(name, value) + raise ValueError(value) + +A good stub definition would be:: + + class IntHolder: + def __setattr__(self, name: str, value: int) -> None: ... + +``__delattr__`` should not be included in stubs. + +Finally, even in the presence of ``__getattr__`` and ``__setattr__``, it is +still recommended to separately define known attributes. + +Constants +--------- + +When the value of a constant is important, annotate it using ``Literal`` +instead of its type. + +Yes:: + + TEL_LANDLINE: Literal["landline"] + TEL_MOBILE: Literal["mobile"] + DAY_FLAG: Literal[0x01] + NIGHT_FLAG: Literal[0x02] + +No:: + + TEL_LANDLINE: str + TEL_MOBILE: str + DAY_FLAG: int + NIGHT_FLAG: int + +Documentation or Implementation +------------------------------- + +Sometimes a library's documented types will differ from the actual types in the +code. In such cases, type stub authors should use their best judgment. Consider +these two examples:: + + def print_elements(x): + """Print every element of list x.""" + for y in x: + print(y) + + def maybe_raise(x): + """Raise an error if x (a boolean) is true.""" + if x: + raise ValueError() + +The implementation of ``print_elements`` takes any iterable, despite the +documented type of ``list``. In this case, annotate the argument as +``Iterable[Any]``, to follow this PEP's style recommendation of preferring +abstract types. + +For ``maybe_raise``, on the other hand, it is better to annotate the argument as +``bool`` even though the implementation accepts any object. This guards against +common mistakes like unintentionally passing in ``None``. + +If in doubt, consider asking the library maintainers about their intent. + +Style Guide +=========== + +The recommendations in this section are aimed at type stub authors +who wish to provide a consistent style for type stubs. Type checkers +should not reject stubs that do not follow these recommendations, but +linters can warn about them. + +Stub files should generally follow the Style Guide for Python Code (PEP 8) +[#pep8]_. There are a few exceptions, outlined below, that take the +different structure of stub files into account and are aimed to create +more concise files. + +Maximum Line Length +------------------- + +Type stubs should be limited to 130 characters per line. + +Blank Lines +----------- + +Do not use empty lines between functions, methods, and fields, except to +group them with one empty line. Use one empty line around classes, but do not +use empty lines between body-less classes, except for grouping. + +Yes:: + + def time_func() -> None: ... + def date_func() -> None: ... + + def ip_func() -> None: ... + + class Foo: + x: int + y: int + def __init__(self) -> None: ... + + class MyError(Exception): ... + class AnotherError(Exception): ... + +No:: + + def time_func() -> None: ... + + def date_func() -> None: ... # do no leave unnecessary empty lines + + def ip_func() -> None: ... + + + class Foo: # leave only one empty line above + x: int + class MyError(Exception): ... # leave an empty line between the classes + +Module Level Attributes +----------------------- + +Do not use an assignment for module-level attributes. + +Yes:: + + CONST: Literal["const"] + x: int + +No:: + + CONST = "const" + x: int = 0 + y: float = ... + z = 0 # type: int + a = ... # type: int + +Classes +------- + +Classes without bodies should use the ellipsis literal ``...`` in place +of the body on the same line as the class definition. + +Yes:: + + class MyError(Exception): ... + +No:: + + class MyError(Exception): + ... + class AnotherError(Exception): pass + +Instance attributes and class variables follow the same recommendations as +module level attributes: + +Yes:: + + class Foo: + c: ClassVar[str] + x: int + +No:: + + class Foo: + c: ClassVar[str] = "" + d: ClassVar[int] = ... + x = 4 + y: int = ... + +Functions and Methods +--------------------- + +Use the same argument names as in the implementation, because +otherwise using keyword arguments will fail. Of course, this +does not apply to positional-only arguments, which are marked with a double +underscore. + +Use the ellipsis literal ``...`` in place of actual default argument +values. Use an explicit ``Optional`` annotation instead of +a ``None`` default. + +Yes:: + + def foo(x: int = ...) -> None: ... + def bar(y: Optional[str] = ...) -> None: ... + +No:: + + def foo(x: int = 0) -> None: ... + def bar(y: str = None) -> None: ... + def baz(z: Optional[str] = None) -> None: ... + +Do not annotate ``self`` and ``cls`` in method definitions, except when +referencing a type variable. + +Yes:: + + _T = TypeVar("_T") + class Foo: + def bar(self) -> None: ... + @classmethod + def create(cls: type[_T]) -> _T: ... + +No:: + + class Foo: + def bar(self: Foo) -> None: ... + @classmethod + def baz(cls: type[Foo]) -> int: ... + +The bodies of functions and methods should consist of only the ellipsis +literal ``...`` on the same line as the closing parenthesis and colon. + +Yes:: + + def to_int1(x: str) -> int: ... + def to_int2( + x: str, + ) -> int: ... + +No:: + + def to_int1(x: str) -> int: + return int(x) + def to_int2(x: str) -> int: + ... + def to_int3(x: str) -> int: pass + +Private Definitions +------------------- + +Type variables, type aliases, and other definitions that should not +be used outside the stub should be marked as private by prefixing them +with an underscore. + +Yes:: + + _T = TypeVar("_T") + _DictList = dict[str, list[Optional[int]]] + +No:: + + T = TypeVar("T") + DictList = dict[str, list[Optional[int]]] + +Language Features +----------------- + +Use the latest language features available as outlined +in the Syntax_ section, even for stubs targeting older Python versions. +Do not use quotes around forward references and do not use ``__future__`` +imports. + +Yes:: + + class Py35Class: + x: int + forward_reference: OtherClass + class OtherClass: ... + +No:: + + class Py35Class: + x = 0 # type: int + forward_reference: 'OtherClass' + class OtherClass: ... + +Types +----- + +Generally, use ``Any`` when a type cannot be expressed appropriately +with the current type system or using the correct type is unergonomic. + +Use ``float`` instead of ``Union[int, float]``. +Use ``None`` instead of ``Literal[None]``. +For argument types, +use ``bytes`` instead of ``Union[bytes, memoryview, bytearray]``. + +Use ``Text`` in stubs that support Python 2 when something accepts both +``str`` and ``unicode``. Avoid using ``Text`` in stubs or branches for +Python 3 only. + +Yes:: + + if sys.version_info < (3,): + def foo(s: Text) -> None: ... + else: + def foo(s: str, *, i: int) -> None: ... + def bar(s: Text) -> None: ... + +No:: + + if sys.version_info < (3,): + def foo(s: unicode) -> None: ... + else: + def foo(s: Text, *, i: int) -> None: ... + +For arguments, prefer protocols and abstract types (``Mapping``, +``Sequence``, ``Iterable``, etc.). If an argument accepts literally any value, +use ``object`` instead of ``Any``. + +For return values, prefer concrete types (``list``, ``dict``, etc.) for +concrete implementations. The return values of protocols +and abstract base classes must be judged on a case-by-case basis. + +Yes:: + + def map_it(input: Iterable[str]) -> list[int]: ... + def create_map() -> dict[str, int]: ... + def to_string(o: object) -> str: ... # accepts any object + +No:: + + def map_it(input: list[str]) -> list[int]: ... + def create_map() -> MutableMapping[str, int]: ... + def to_string(o: Any) -> str: ... + +Maybe:: + + class MyProto(Protocol): + def foo(self) -> list[int]: ... + def bar(self) -> Mapping[str]: ... + +Avoid ``Union`` return types, since they require ``isinstance()`` checks. +Use ``Any`` if necessary. + +Use built-in generics instead of the aliases from ``typing``. + +Yes:: + + from collections.abc import Iterable + + def foo(x: type[MyClass]) -> list[str]: ... + def bar(x: Iterable[str]) -> None: ... + +No:: + + from typing import Iterable, List, Type + + def foo(x: Type[MyClass]) -> List[str]: ... + def bar(x: Iterable[str]) -> None: ... + +NamedTuple and TypedDict +------------------------ + +Use the class-based syntax for ``typing.NamedTuple`` and +``typing.TypedDict``, following the Classes section of this style guide. + +Yes:: + + from typing import NamedTuple, TypedDict + class Point(NamedTuple): + x: float + y: float + + class Thing(TypedDict): + stuff: str + index: int + +No:: + + from typing import NamedTuple, TypedDict + Point = NamedTuple("Point", [('x', float), ('y', float)]) + Thing = TypedDict("Thing", {'stuff': str, 'index': int}) + +Existing Tools +============== + +Type Checkers +------------- + +* mypy [#mypy]_, the reference implementation for type checkers. + Supports Python 2 and 3. +* pyre [#pyre]_, written in OCaml and optimized for performance. + Supports Python 3 only. +* pyright [#pyright]_, a type checker that emphasizes speed. Supports Python 3 + only. +* pytype [#pytype]_, checks and infers types for unannotated code. + Supports Python 2 and 3. + +Development Environments +------------------------ + +* PyCharm [#pycharm]_, an IDE that supports type stubs both for type + checking and code completion. +* Visual Studio Code [#vscode]_, a code editor that supports type + checking using mypy, pyright, or the Pylance [#pylance]_ extension. + +Linters and Formatters +---------------------- + +* black [#black]_, a code formatter with support for type stub files. +* flake8-pyi [#flake8-pyi]_, a plugin for the flake8 linter [#flake8]_ that adds support for + type stubs. + +References +========== + +PEPs +---- + +.. [#pep8] PEP 8 -- Style Guide for Python Code, van Rossum et al. (https://www.python.org/dev/peps/pep-0008/) +.. [#pep484] PEP 484 -- Type Hints, van Rossum et al. (https://www.python.org/dev/peps/pep-0484) +.. [#pep492] PEP 492 -- Coroutines with async and await syntax, Selivanov (https://www.python.org/dev/peps/pep-0492/) +.. [#pep526] PEP 526 -- Syntax for Variable Annotations, Gonzalez et al. (https://www.python.org/dev/peps/pep-0526) +.. [#pep544] PEP 544 -- Protocols: Structural Subtyping, Levkivskyi et al. (https://www.python.org/dev/peps/pep-0544) +.. [#pep561] PEP 561 -- Distributing and Packaging Type Information, Smith (https://www.python.org/dev/peps/pep-0561) +.. [#pep570] PEP 570 -- Python Positional-Only Parameters, Hastings et al. (https://www.python.org/dev/peps/pep-0570) +.. [#pep585] PEP 585 -- Type Hinting Generics In Standard Collections, Langa (https://www.python.org/dev/peps/pep-0585) +.. [#pep604] PEP 604 -- Allow writing union types as X | Y, Prados and Moss (https://www.python.org/dev/peps/pep-0604) +.. [#pep612] PEP 612 -- Parameter Specification Variables, Mendoza (https://www.python.org/dev/peps/pep-0612) +.. [#pep613] PEP 613 -- Explicit Type Aliases, Zhu (https://www.python.org/dev/peps/pep-0613) +.. [#pep3107] PEP 3107 -- Function Annotations, Winter and Lownds (https://www.python.org/dev/peps/pep-3107) + +Type Checkers +------------- + +.. [#mypy] mypy -- Optional Static Typing for Python (http://www.mypy-lang.org/) +.. [#pyre] Pyre -- A performant type-checker for Python 3 (https://pyre-check.org/) +.. [#pyright] pyright -- Static type checker for Python (https://github.com/microsoft/pyright) +.. [#pytype] pytype -- A static analyzer for Python code (https://github.com/google/pytype) + +IDEs with Typing Support +------------------------ + +.. [#pycharm] PyCharm -- The Python IDE for Professional Developers (https://www.jetbrains.com/pycharm/) +.. [#pylance] Pylance -- Fast, feature-rich language support for Python (https://github.com/microsoft/pylance-release) +.. [#vscode] Visual Studio Code -- Code Editing. Redefined (https://code.visualstudio.com/) + +Other Resources +--------------- + +.. [#black] black -- The uncompromising code formatter (https://black.readthedocs.io/) +.. [#flake8] Flake8 -- Your Tool For Style Guide Enforcement (http://flake8.pycqa.org/) +.. [#flake8-pyi] flake8-pyi -- A plugin for Flake8 that provides specializations for type hinting stub files (https://github.com/ambv/flake8-pyi) +.. [#typeshed] typeshed -- Collection of library stubs for Python, with static types (https://github.com/python/typeshed) +.. [#ast] ast -- Abstract Syntax Trees, Python standard library module (https://docs.python.org/3/library/ast.html) +.. [#typed_ast] typed_ast -- Fork of CPython's ast module (https://pypi.org/project/typed-ast/) + +Copyright +========= + +This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. From 82b3940a59d173b210b4d525d0921b25f43669b6 Mon Sep 17 00:00:00 2001 From: Sebastian Rittau Date: Mon, 23 Aug 2021 19:56:11 +0200 Subject: [PATCH 09/18] Ignore venv and README when building docs (#843) * Ignore venv and README when building docs This suppresses warnings during the docs build if the venv directory exists. * Remove html static path from Sphinx config Fixes another Sphinx warning --- docs/conf.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/conf.py b/docs/conf.py index 881c068d..f870c7be 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -36,7 +36,7 @@ # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. # This pattern also affects html_static_path and html_extra_path. -exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] +exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', 'venv', 'README.rst'] # -- Options for HTML output ------------------------------------------------- @@ -49,4 +49,4 @@ # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". -html_static_path = ['_static'] +html_static_path = [] From 9e1e4478010f02a57ebe8e2ad6ccf227679dd2ad Mon Sep 17 00:00:00 2001 From: Sebastian Rittau Date: Mon, 23 Aug 2021 23:20:19 +0200 Subject: [PATCH 10/18] Move the typing tools section to the index (#847) * Enable the intersphinx extension for linking to Python documentation. * Use inline links instead of footnotes for links to projects and modules. * Mention that typed_ast is available on PyPI. --- docs/conf.py | 3 +++ docs/index.rst | 34 +++++++++++++++++++++++++++ docs/stubs.rst | 63 ++++---------------------------------------------- 3 files changed, 42 insertions(+), 58 deletions(-) diff --git a/docs/conf.py b/docs/conf.py index f870c7be..f16401ba 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -50,3 +50,6 @@ # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = [] + +extensions = ['sphinx.ext.intersphinx'] +intersphinx_mapping = {'python': ('https://docs.python.org/3', None)} diff --git a/docs/index.rst b/docs/index.rst index 97ab6bdf..7a764910 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -18,3 +18,37 @@ Indices and tables * :ref:`genindex` * :ref:`search` + +Typing-related Tools +==================== + +Type Checkers +------------- + +* `mypy `_, the reference implementation for type + checkers. Supports Python 2 and 3. +* `pyre `_, written in OCaml and optimized for + performance. Supports Python 3 only. +* `pyright `_, a type checker that + emphasizes speed. Supports Python 3 only. +* `pytype `_, checks and infers types for + unannotated code. Supports Python 2 and 3. + +Development Environments +------------------------ + +* `PyCharm `_, an IDE that supports + type stubs both for type checking and code completion. +* `Visual Studio Code `_, a code editor that + supports type checking using mypy, pyright, or the + `Pylance `_ + extension. + +Linters and Formatters +---------------------- + +* `black `_, a code formatter with support for + type stub files. +* `flake8-pyi `_, a plugin for the + `flake8 `_ linter that adds support for type + stubs. diff --git a/docs/stubs.rst b/docs/stubs.rst index 5e4a3681..8b888048 100644 --- a/docs/stubs.rst +++ b/docs/stubs.rst @@ -71,16 +71,18 @@ Python 3.8, although type checker authors are encouraged to support it. Stubs are treated as if ``from __future__ import annotations`` is enabled. In particular, built-in generics and forward references can be used. -Starting with Python 3.8, the ast_ module from the standard library supports +Starting with Python 3.8, the :py:mod:`ast` module from the standard library supports all syntax features required by this PEP. Older Python versions can use the -typed_ast_ package, which also supports Python 3.7 syntax and ``# type`` +`typed_ast `_ package from the +Python Package Index, which also supports Python 3.7 syntax and ``# type`` comments. Distribution ============ Type stubs can be distributed with or separately from the implementation; -see PEP 561 [#pep561]_ for more information. The typeshed_ project +see PEP 561 [#pep561]_ for more information. The +`typeshed `_ project includes stubs for Python's standard library and several third-party packages. These are usually distributed with type checkers and do not require separate installation. @@ -989,36 +991,6 @@ No:: Point = NamedTuple("Point", [('x', float), ('y', float)]) Thing = TypedDict("Thing", {'stuff': str, 'index': int}) -Existing Tools -============== - -Type Checkers -------------- - -* mypy [#mypy]_, the reference implementation for type checkers. - Supports Python 2 and 3. -* pyre [#pyre]_, written in OCaml and optimized for performance. - Supports Python 3 only. -* pyright [#pyright]_, a type checker that emphasizes speed. Supports Python 3 - only. -* pytype [#pytype]_, checks and infers types for unannotated code. - Supports Python 2 and 3. - -Development Environments ------------------------- - -* PyCharm [#pycharm]_, an IDE that supports type stubs both for type - checking and code completion. -* Visual Studio Code [#vscode]_, a code editor that supports type - checking using mypy, pyright, or the Pylance [#pylance]_ extension. - -Linters and Formatters ----------------------- - -* black [#black]_, a code formatter with support for type stub files. -* flake8-pyi [#flake8-pyi]_, a plugin for the flake8 linter [#flake8]_ that adds support for - type stubs. - References ========== @@ -1038,31 +1010,6 @@ PEPs .. [#pep613] PEP 613 -- Explicit Type Aliases, Zhu (https://www.python.org/dev/peps/pep-0613) .. [#pep3107] PEP 3107 -- Function Annotations, Winter and Lownds (https://www.python.org/dev/peps/pep-3107) -Type Checkers -------------- - -.. [#mypy] mypy -- Optional Static Typing for Python (http://www.mypy-lang.org/) -.. [#pyre] Pyre -- A performant type-checker for Python 3 (https://pyre-check.org/) -.. [#pyright] pyright -- Static type checker for Python (https://github.com/microsoft/pyright) -.. [#pytype] pytype -- A static analyzer for Python code (https://github.com/google/pytype) - -IDEs with Typing Support ------------------------- - -.. [#pycharm] PyCharm -- The Python IDE for Professional Developers (https://www.jetbrains.com/pycharm/) -.. [#pylance] Pylance -- Fast, feature-rich language support for Python (https://github.com/microsoft/pylance-release) -.. [#vscode] Visual Studio Code -- Code Editing. Redefined (https://code.visualstudio.com/) - -Other Resources ---------------- - -.. [#black] black -- The uncompromising code formatter (https://black.readthedocs.io/) -.. [#flake8] Flake8 -- Your Tool For Style Guide Enforcement (http://flake8.pycqa.org/) -.. [#flake8-pyi] flake8-pyi -- A plugin for Flake8 that provides specializations for type hinting stub files (https://github.com/ambv/flake8-pyi) -.. [#typeshed] typeshed -- Collection of library stubs for Python, with static types (https://github.com/python/typeshed) -.. [#ast] ast -- Abstract Syntax Trees, Python standard library module (https://docs.python.org/3/library/ast.html) -.. [#typed_ast] typed_ast -- Fork of CPython's ast module (https://pypi.org/project/typed-ast/) - Copyright ========= From b4ea885106d49b29d666975e0fa0957e5bcd9317 Mon Sep 17 00:00:00 2001 From: Sebastian Rittau Date: Mon, 23 Aug 2021 23:29:44 +0200 Subject: [PATCH 11/18] Build the docs when a PR is created (#848) This only triggers on pr, not on pushes, since there will be a separate workflow for pushed that builds and uploads the documentation. --- .github/workflows/build-docs.yml | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 .github/workflows/build-docs.yml diff --git a/.github/workflows/build-docs.yml b/.github/workflows/build-docs.yml new file mode 100644 index 00000000..806f8331 --- /dev/null +++ b/.github/workflows/build-docs.yml @@ -0,0 +1,26 @@ +name: Build the documentation + +on: + pull_request: + +permissions: + contents: read + +jobs: + build: + + name: Build documentation + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@v2 + - name: Set up Python + uses: actions/setup-python@v2 + with: + python-version: 3.9 + - name: Install dependencies + run: | + python -m pip install --upgrade pip + pip install -r docs/requirements.txt + - name: Build the documentation + run: make -C docs html From f5bc93aea3a6e13bcd860e2998d1c08299819a94 Mon Sep 17 00:00:00 2001 From: Sebastian Rittau Date: Tue, 24 Aug 2021 05:47:32 +0200 Subject: [PATCH 12/18] Link to the typing module docs (#849) --- docs/index.rst | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/docs/index.rst b/docs/index.rst index 7a764910..670c7824 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -1,8 +1,3 @@ -.. typing documentation master file, created by - sphinx-quickstart on Mon May 24 16:43:52 2021. - You can adapt this file completely to your liking, but it should at least - contain the root `toctree` directive. - Welcome to the Python Type System documentation! ================================================ @@ -10,6 +5,7 @@ Welcome to the Python Type System documentation! :maxdepth: 2 :caption: Contents: + typing Module Documentation stubs @@ -19,6 +15,7 @@ Indices and tables * :ref:`genindex` * :ref:`search` + Typing-related Tools ==================== From ce2ea207b0e5df921a820594e231858654c41aa3 Mon Sep 17 00:00:00 2001 From: Sebastian Rittau Date: Tue, 24 Aug 2021 08:08:17 +0200 Subject: [PATCH 13/18] Various improvements to the Type Stubs document (#846) * Rename "Abstract" to "Introduction" and remove fluff originally intended for the PEP. * "This PEP" -> "This document" * Remove note about the living document status. This is a given, not that it isn't a PEP anymore. * Third-party stubs are now distributed via PyPI. * Mention "# type: ignore[xxx]". * Fix indentation. * Reformulate sentence about cyclic imports. * Use type vars correctly in example. * Use new union syntax. * Remove "we" from sentence and slightly reformulate. * Clarify that "# incomplete" is mainly intended for stub authors. * Mention "X | Any" return type. Still needs to be explained. --- docs/stubs.rst | 65 ++++++++++++++++++++++++-------------------------- 1 file changed, 31 insertions(+), 34 deletions(-) diff --git a/docs/stubs.rst b/docs/stubs.rst index 8b888048..f06efe96 100644 --- a/docs/stubs.rst +++ b/docs/stubs.rst @@ -4,18 +4,11 @@ Type Stubs ********** -Abstract -======== - -Optional type hints were introduced to the Python language in PEP 484 -[#pep484]_, based on the function annotation syntax from PEP 3107 -[#pep3107]_. Static type checkers can use type hints to prevent bugs, -documentation tools can automatically add type information, -and IDEs can offer improved autocompletion and support safer refactorings. +Introduction +============ -PEP 484 also introduced *type stubs*, also called *stub files*, -that provide type information for untyped Python packages and modules. Type -stubs serve multiple purposes: +*type stubs*, also called *stub files*, provide type information for untyped +Python packages and modules. Type stubs serve multiple purposes: * They are the only way to add type information to extension modules. * They can provide type information for packages that do not wish to @@ -28,27 +21,24 @@ stubs serve multiple purposes: API of a package, without including the implementation or private members. -This PEP aims to give guidance to both authors of type stubs and developers +This document aims to give guidance to both authors of type stubs and developers of type checkers and other tools. It describes the constructs that can be used safely in type stubs, suggests a style guide for them, and lists constructs that type checkers are expected to support. -Type stubs that only use constructs described in this PEP should work with -all type checkers that also follow this PEP. +Type stubs that only use constructs described in this document should work with +all type checkers that also follow this document. Type stub authors can elect to use additional constructs, but must be prepared that some type checkers will not parse them as expected. -A type checker that conforms to this PEP will parse a type stub that only uses +A type checker that conforms to this document will parse a type stub that only uses constructs described here without error and will not interpret any construct in a contradictory manner. However, type checkers are not required to implement checks for all these constructs, and can elect to ignore unsupported ones. Additionally type checkers -can support constructs not described in this PEP and tool authors are +can support constructs not described in this document and tool authors are encouraged to experiment with additional features. -This PEP is intended as a living document and will be updated as new -features are supported or best practices evolve. - Syntax ====== @@ -84,8 +74,10 @@ Type stubs can be distributed with or separately from the implementation; see PEP 561 [#pep561]_ for more information. The `typeshed `_ project includes stubs for Python's standard library and several third-party -packages. These are usually distributed with type checkers and do not -require separate installation. +packages. The stubs for the standard library are usually distributed with type checkers and do not +require separate installation. Stubs for third-party libraries are +available on the `Python Package Index `_. A stub package for +a library called ``widget`` will be called ``types-widget``. Supported Constructs ==================== @@ -121,7 +113,9 @@ Two kinds of structured comments are accepted: declaring that the variable has type ``X``. However, PEP 526-style [#pep526]_ variable annotations are preferred over type comments. * A ``# type: ignore`` comment at the end of any line, which suppresses all type - errors in that line. + errors in that line. The type checker mypy supports suppressing certain + type errors by using ``# type: ignore[error-type]``. This is not supported + by other type checkers and should not be used in stubs. Imports ------- @@ -168,13 +162,13 @@ By default, ``from foo import *`` imports all names in ``foo`` that do not begin with an underscore. When ``__all__`` is defined, only those names specified in ``__all__`` are imported:: - __all__ = ['public_attr', '_private_looking_public_attr'] + __all__ = ['public_attr', '_private_looking_public_attr'] public_attr: int _private_looking_public_attr: int private_attr: int -Type checkers can handle cyclic imports in stub files. +Type checkers support cyclic imports in stub files. Module Level Attributes ----------------------- @@ -269,7 +263,7 @@ Alternatively, ``...`` can be used in place of any default value:: def invalid(a: int = "", b: Foo = Foo()): ... For a class ``C``, the type of the first argument to a classmethod is -assumed to be ``Type[C]``, if unannotated. For other non-static methods, +assumed to be ``type[C]``, if unannotated. For other non-static methods, its type is assumed to be ``C``:: class Foo: @@ -284,9 +278,9 @@ But:: _T = TypeVar("_T") class Foo: - def do_things(self: _T): ... # self has type _T + def do_things(self: _T) -> _T: ... # self has type _T @classmethod - def create_it(cls: _T): ... # cls has type _T + def create_it(cls: _T) -> _T: ... # cls has type _T Using a function or method body other than the ellipsis literal is currently unspecified. Stub authors may experiment with other bodies, but it is up to @@ -310,7 +304,7 @@ type stubs:: def foo(x: str) -> str: ... @overload def foo(x: float) -> int: ... - def foo(x: Union[str, float]) -> Any: ... + def foo(x: str | float) -> Any: ... Aliases and NewType ------------------- @@ -512,8 +506,8 @@ Such undocumented objects are allowed because omitting objects can confuse users. Users who see an error like "module X has no attribute Y" will not know whether the error appeared because their code had a bug or because the stub is wrong. Although it may also be helpful for a type -checker to point out usage of private objects, we usually prefer false -negatives (no errors for wrong code) over false positives (type errors +checker to point out usage of private objects, false negatives (no errors for +wrong code) are preferable over false positives (type errors for correct code). In addition, even for private objects a type checker can be helpful in pointing out that an incorrect type was used. @@ -580,6 +574,9 @@ annotated function ``bar()``:: def bar(x: str, y, *, z=...): ... +The ``# incomplete`` comment is mainly intended as a reminder for stub +authors, but can be used by tools to flag such items. + Attribute Access ---------------- @@ -899,10 +896,10 @@ Types Generally, use ``Any`` when a type cannot be expressed appropriately with the current type system or using the correct type is unergonomic. -Use ``float`` instead of ``Union[int, float]``. +Use ``float`` instead of ``int | float``. Use ``None`` instead of ``Literal[None]``. For argument types, -use ``bytes`` instead of ``Union[bytes, memoryview, bytearray]``. +use ``bytes`` instead of ``bytes | memoryview | bytearray``. Use ``Text`` in stubs that support Python 2 when something accepts both ``str`` and ``unicode``. Avoid using ``Text`` in stubs or branches for @@ -949,8 +946,8 @@ Maybe:: def foo(self) -> list[int]: ... def bar(self) -> Mapping[str]: ... -Avoid ``Union`` return types, since they require ``isinstance()`` checks. -Use ``Any`` if necessary. +Avoid union return types, since they require ``isinstance()`` checks. +Use ``Any`` or ``X | Any`` if necessary. Use built-in generics instead of the aliases from ``typing``. From c7c30c894e671a4175d7fd3ed1054a013b5853a9 Mon Sep 17 00:00:00 2001 From: Sebastian Rittau Date: Tue, 24 Aug 2021 08:40:48 +0200 Subject: [PATCH 14/18] Link to discussion forums (#852) --- docs/index.rst | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/index.rst b/docs/index.rst index 670c7824..0f03af18 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -16,6 +16,13 @@ Indices and tables * :ref:`search` +Discussions and Support +======================= + +* `User help forum `_ +* `User chat on Gitter `_ +* `Developer mailing list `_ + Typing-related Tools ==================== From 651cb585f1db980ac111a11fc40f1ebc4a8ff719 Mon Sep 17 00:00:00 2001 From: Sebastian Rittau Date: Tue, 24 Aug 2021 23:44:46 +0200 Subject: [PATCH 15/18] Rename the documentation (#854) Use correct heading level --- docs/index.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/index.rst b/docs/index.rst index 0f03af18..dba85dd8 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -1,5 +1,6 @@ -Welcome to the Python Type System documentation! -================================================ +************************* +Static Typing with Python +************************* .. toctree:: :maxdepth: 2 From 7d47c9fb37fdb4317831ae6e92d9324e7c92a8f4 Mon Sep 17 00:00:00 2001 From: Sebastian Rittau Date: Wed, 25 Aug 2021 15:51:17 +0200 Subject: [PATCH 16/18] Rework README (#853) * Rename to "Static Typing for Python". * Add "Documentation and Support" section. * Update what the repository is used for. --- README.md | 37 ++++++++++++++++++++++++++++--------- 1 file changed, 28 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index a1df5e21..aa1531d1 100644 --- a/README.md +++ b/README.md @@ -1,21 +1,40 @@ [![Chat at https://gitter.im/python/typing](https://badges.gitter.im/python/typing.svg)](https://gitter.im/python/typing?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) -PEP 484: Type Hints -=================== +Static Typing for Python +======================== -This GitHub repo is used for three separate things: +Documentation and Support +------------------------- -- The issue tracker is used to discuss PEP-level type system issues. - However, - [typing-sig](https://mail.python.org/mailman3/lists/typing-sig.python.org/) - is more appropriate these days. +The documentation for Python's static typing can be found at +[typing.readthedocs.io](https://typing.readthedocs.io/). You can get +help either in our [support forum](/python/typing/discussions) or +chat with us on (Gitter)[https://gitter.im/python/typing]. + +Improvements to the type system should be discussed on the +[typing-sig](https://mail.python.org/mailman3/lists/typing-sig.python.org/) +mailing list, although the [issues](/python/typing/issues) in this +repository contain some historic discussions. + +Repository Content +------------------ + +This GitHub repo is used for several things: - A backport of the `typing` module for older Python versions (2.7 and - 3.4) is maintained here. Note that the canonical source lives + 3.4) is maintained in the [src directory](./src). + Note that the canonical source lives [upstream](https://github.com/python/cpython/blob/master/Lib/typing.py) in the CPython repo. -- The `typing_extensions` module lives here. +- The `typing_extensions` module lives in the + [typing\_extensions](./typing_extensions) directory. + +- The documentation at [typing.readthedocs.io](https://typing.readthedocs.io/) + is maintained in the [docs directory](./docs). + +- A [discussion forum](/python/typing/discussions) for typing-related user + help is hosted here. Workflow -------- From 01b0368690ff9a367d6320cb161c4104fbb1af7a Mon Sep 17 00:00:00 2001 From: Sebastian Rittau Date: Wed, 25 Aug 2021 16:12:19 +0200 Subject: [PATCH 17/18] Linkfixes (#857) --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index aa1531d1..860cc567 100644 --- a/README.md +++ b/README.md @@ -8,12 +8,12 @@ Documentation and Support The documentation for Python's static typing can be found at [typing.readthedocs.io](https://typing.readthedocs.io/). You can get -help either in our [support forum](/python/typing/discussions) or -chat with us on (Gitter)[https://gitter.im/python/typing]. +help either in our [support forum](https://github.com/python/typing/discussions) or +chat with us on [Gitter](https://gitter.im/python/typing). Improvements to the type system should be discussed on the [typing-sig](https://mail.python.org/mailman3/lists/typing-sig.python.org/) -mailing list, although the [issues](/python/typing/issues) in this +mailing list, although the [issues](https://github.com/python/typing/issues) in this repository contain some historic discussions. Repository Content @@ -33,7 +33,7 @@ This GitHub repo is used for several things: - The documentation at [typing.readthedocs.io](https://typing.readthedocs.io/) is maintained in the [docs directory](./docs). -- A [discussion forum](/python/typing/discussions) for typing-related user +- A [discussion forum](https://github.com/python/typing/discussions) for typing-related user help is hosted here. Workflow From 7d2fae84d7592fdf04380fbbe2d3e667ecedbcf6 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Sun, 29 Aug 2021 08:52:30 -0700 Subject: [PATCH 18/18] prepare release 3.10.0.1 (#863) --- typing_extensions/setup.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/typing_extensions/setup.py b/typing_extensions/setup.py index 7b1611cb..ca537456 100644 --- a/typing_extensions/setup.py +++ b/typing_extensions/setup.py @@ -9,7 +9,7 @@ 'to install the typing package.\n') exit(1) -version = '3.10.0.0' +version = '3.10.0.1' description = 'Backported and Experimental Type Hints for Python 3.5+' long_description = '''\ Typing Extensions -- Backported and Experimental Type Hints for Python