Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upAdd typing extensions #443
Conversation
|
I don't think you need all these old versions bundled in tests. I think we should just properly configure Travis (it supports Python micro releases like 3.5.2 etc. (like in mypy's |
|
@ilevkivskyi I guess my only concern with that is that it becomes harder to do sanity checks locally -- having to constantly push to run tests seems a bit annoying to me. |
You can still keep these copies locally (without tracking them in git). But I would prefer a properly configured |
124768e
to
6928977
| T_contra = typing.TypeVar('T_contra', contravariant=True) # Ditto contravariant. | ||
|
|
||
|
|
||
| def _gorg(a): |
ilevkivskyi
Jun 27, 2017
Collaborator
Note that _gorg and _geqv will be removed soon after Python 3.6.2 is released, see PR #439.
Note that _gorg and _geqv will be removed soon after Python 3.6.2 is released, see PR #439.
Michael0x2a
Jun 27, 2017
•
Author
Contributor
Sounds good -- once this pull request gets accepted, I can prepare another pull request to add those changes and leave it pending just like #439 (or whatever other course of action you think is best).
Sounds good -- once this pull request gets accepted, I can prepare another pull request to add those changes and leave it pending just like #439 (or whatever other course of action you think is best).
ilevkivskyi
Jul 7, 2017
Collaborator
or whatever other course of action you think is best
OK, let's just not forget about this.
or whatever other course of action you think is best
OK, let's just not forget about this.
|
Concerning the test failure, we decided to stop supporting Python 3.2 (it reached EoL a year ago), so that you can just skip testing on Python 3.2. |
|
Update: I removed Once that's accepted, I can rebase, but this pull request should otherwise be ready for review (?). Edit: lol, sniped |
6928977
to
15cbbfc
|
I can review this, but I am not sure I will be able to review before week-end. |
|
Sorry for waiting with this, the PR looks very good. Thanks for work! I just have mostly minor comments. In general I am eager to merge this soon, so that we can put stuff from |
| author='Guido van Rossum, Jukka Lehtosalo, Lukasz Langa, Michael Lee', | ||
| author_email='jukka.lehtosalo@iki.fi', | ||
| # TODO: Change URL | ||
| url='https://github.com/michael0x2a/typing_extensions', |
ilevkivskyi
Jul 7, 2017
Collaborator
The link should be to the main typing repo.
The link should be to the main typing repo.
Michael0x2a
Jul 7, 2017
Author
Contributor
Whoops, forgot to change that.
Whoops, forgot to change that.
| ''' | ||
|
|
||
| classifiers = [ | ||
| 'Development Status :: 5 - Production/Stable', |
ilevkivskyi
Jul 7, 2017
Collaborator
This is far from stable :-) I would put alpha here.
This is far from stable :-) I would put alpha here.
Michael0x2a
Jul 7, 2017
Author
Contributor
Done!
Done!
| src/test_typing.py, | ||
| typing_extensions/src_py2/test_typing_extensions.py, | ||
| typing_extensions/src_py3/test_typing_extensions.py, | ||
| # test_data should contain an exact snapshot of the standard |
ilevkivskyi
Jul 7, 2017
Collaborator
Do we still need this? I thought we agreed that we could leave this out and just use updated Travis config.
Do we still need this? I thought we agreed that we could leave this out and just use updated Travis config.
Michael0x2a
Jul 7, 2017
Author
Contributor
Same thing -- forgot to remove it.
Same thing -- forgot to remove it.
| @@ -14,5 +18,8 @@ install: | ||
|
|
||
| script: | ||
| - export PYTHONPATH=`python -c "import sys; print('python2' if sys.version.startswith('2') else 'src')"`; py.test $PYTHONPATH | ||
| - python setup.py install; |
ilevkivskyi
Jul 7, 2017
Collaborator
I am just curious, why this was not necessary before and is necessary now?
I am just curious, why this was not necessary before and is necessary now?
Michael0x2a
Jul 7, 2017
Author
Contributor
The problem is that the typing_extensions depends on typing being installed for Python 3.4 and below since we try and re-export as much as possible from the typing module. That said, I've tried adjusting travis so it should install typing only for Python 3.4 and below -- we'll see if I did it correctly once the tests are done.
The problem is that the typing_extensions depends on typing being installed for Python 3.4 and below since we try and re-export as much as possible from the typing module. That said, I've tried adjusting travis so it should install typing only for Python 3.4 and below -- we'll see if I did it correctly once the tests are done.
| To run tests, navigate into the appropriate source directory and run | ||
| `test_typing_extensions.py`. You will also need to install the latest | ||
| version of `typing` if you are using a version of Python that does not | ||
| include `typing` as a part of the standard library. |
ilevkivskyi
Jul 7, 2017
Collaborator
Maybe we could add a requirements.txt or test_requirements.txt for this?
Maybe we could add a requirements.txt or test_requirements.txt for this?
Michael0x2a
Jul 7, 2017
Author
Contributor
I added it to the test_requirements.txt file for the typing repo. If you prefer, I can leave that one alone and add a separate test_requirements.txt and requirements.txt file to the typing_extensions folder.
Since typing_extensions is basically a separate project, maybe it makes sense for it to have its own requirement files, but it does feel a little redundant...
I added it to the test_requirements.txt file for the typing repo. If you prefer, I can leave that one alone and add a separate test_requirements.txt and requirements.txt file to the typing_extensions folder.
Since typing_extensions is basically a separate project, maybe it makes sense for it to have its own requirement files, but it does feel a little redundant...
ilevkivskyi
Jul 18, 2017
Collaborator
I would actually prefer separate requirements for typing_extensions.
I would actually prefer separate requirements for typing_extensions.
|
|
||
|
|
||
| if __name__ == '__main__': | ||
| # main(argv=[sys.argv[0]] + sys.argv[2:]) |
ilevkivskyi
Jul 7, 2017
Collaborator
Is this comment necessary? Why do we need this pattern?
Is this comment necessary? Why do we need this pattern?
Michael0x2a
Jul 7, 2017
Author
Contributor
This was a holdover from the code I previously had for running tests, which we removed. I removed the comment here + in the Python 2 tests.
This was a holdover from the code I previously had for running tests, which we removed. I removed the comment here + in the Python 2 tests.
| try: | ||
| import collections.abc as collections_abc | ||
| except ImportError: | ||
| import collections as collections_abc # Fallback for PY3.2. |
ilevkivskyi
Jul 7, 2017
Collaborator
Again, this is not needed.
Again, this is not needed.
Michael0x2a
Jul 7, 2017
Author
Contributor
Also removed.
Also removed.
| _generic_new = typing._generic_new | ||
| else: | ||
| def _generic_new(base_cls, cls, *args, **kwargs): | ||
| return base_cls.__new__(cls, *args, **kwargs) |
ilevkivskyi
Jul 7, 2017
Collaborator
I am not 100% sure this is a safe substitution. Do you have a test for this? IIRC, the point was that __init__ was not called at instantiation of subscripted generics.
I am not 100% sure this is a safe substitution. Do you have a test for this? IIRC, the point was that __init__ was not called at instantiation of subscripted generics.
Michael0x2a
Jul 7, 2017
Author
Contributor
I don't know if I necessarily have tests for specifically this -- I was mostly copying what the typing module was already doing and assumed the behavior was correct. For example, in Python 3.5.0 - 3.5.2, the List type was defined like this:
class List(list, MutableSequence[T], extra=list):
def __new__(cls, *args, **kwds):
if _geqv(cls, List):
raise TypeError("Type List cannot be instantiated; "
"use list() instead")
return list.__new__(cls, *args, **kwds)
In Python 3.5.3+, it was defined like this:
class List(list, MutableSequence[T], extra=list):
__slots__ = ()
def __new__(cls, *args, **kwds):
if _geqv(cls, List):
raise TypeError("Type List cannot be instantiated; "
"use list() instead")
return _generic_new(list, cls, *args, **kwds)
I'm also not sure if this substitution is 100% correct, but if it's broken, it'll at least be consistently broken in the same way the rest of the typing module was in Python 3.5.0 - 3.5.2. Defining the function this way also helped me reduce redundancy since I can define things like Deque just once instead of twice.
I don't know if I necessarily have tests for specifically this -- I was mostly copying what the typing module was already doing and assumed the behavior was correct. For example, in Python 3.5.0 - 3.5.2, the List type was defined like this:
class List(list, MutableSequence[T], extra=list):
def __new__(cls, *args, **kwds):
if _geqv(cls, List):
raise TypeError("Type List cannot be instantiated; "
"use list() instead")
return list.__new__(cls, *args, **kwds)In Python 3.5.3+, it was defined like this:
class List(list, MutableSequence[T], extra=list):
__slots__ = ()
def __new__(cls, *args, **kwds):
if _geqv(cls, List):
raise TypeError("Type List cannot be instantiated; "
"use list() instead")
return _generic_new(list, cls, *args, **kwds)I'm also not sure if this substitution is 100% correct, but if it's broken, it'll at least be consistently broken in the same way the rest of the typing module was in Python 3.5.0 - 3.5.2. Defining the function this way also helped me reduce redundancy since I can define things like Deque just once instead of twice.
ilevkivskyi
Jul 18, 2017
Collaborator
be consistently broken in the same way the rest of the typing module was in Python 3.5.0 - 3.5.2
OK, but then please mention this in a docstring/comment.
be consistently broken in the same way the rest of the
typingmodule was in Python 3.5.0 - 3.5.2
OK, but then please mention this in a docstring/comment.
| 'NewType', | ||
| 'overload', | ||
| 'Text', | ||
| 'TYPE_CHECKING', |
ilevkivskyi
Jul 7, 2017
Collaborator
Again, I am not sure about GenericMeta (see comment above).
Again, I am not sure about GenericMeta (see comment above).
| T_contra = typing.TypeVar('T_contra', contravariant=True) # Ditto contravariant. | ||
|
|
||
|
|
||
| def _gorg(a): |
ilevkivskyi
Jul 7, 2017
Collaborator
or whatever other course of action you think is best
OK, let's just not forget about this.
or whatever other course of action you think is best
OK, let's just not forget about this.
|
Actually, I have one more suggestion. Maybe edit description to reflect the fact that experimental things that are going to appear in |
|
@ilevkivskyi Thanks for the review! I'll start going through your comments and making fixes in a moment. |
|
You will have to wait until after the release unless @JukkaL merges this
despite the code freeze.
…On Jul 7, 2017 8:47 AM, "Michael Lee" ***@***.***> wrote:
@ilevkivskyi <https://github.com/ilevkivskyi> Thanks for the review! I'll
start going through your comments and making fixes in a moment.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#443 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ACwrMvzaBjfKPMxJS4jdNXcCf6T9eISxks5sLlMkgaJpZM4OF__7>
.
|
| TYPING_V3 = sys.version_info >= (3, 5, 3) | ||
| TYPING_V4 = sys.version_info >= (3, 6, 1) | ||
| TYPING_LATEST = sys.version_info[:3] < (3, 5, 0) | ||
| TYPING_V2 = TYPING_LATEST or sys.version_info[:3] >= (3, 5, 1) |
JelleZijlstra
Jul 7, 2017
Contributor
I think this would be clearer as TYPING_3_5_1 or similar, because "v2" is not a terminology used elsewhere.
I think this would be clearer as TYPING_3_5_1 or similar, because "v2" is not a terminology used elsewhere.
Michael0x2a
Jul 7, 2017
Author
Contributor
Ok, done.
Ok, done.
|
@ilevkivskyi -- Ok, I should be ready for a second round of review. I tried adding links to relevant pull requests and issues for the typing version-specific if statements, but found it quite hard to do so in general, so unfortunately most of them are still undocumented :(. |
See latest comment in python/typing#443 for justification.
|
Just as an FYI, I decided to remove In order to correctly backport that type, I basically need to inject
I think this is potentially repairable at least at typechecking-time with some very careful surgery on the typing stubs, but doing that seems questionable to me. It also wouldn't fix any runtime (More specifically, Python 3's typing stubs has |
This pull request adds a 'typing_exensions' subproject to 'typing'. The 'typing_extensions' module backports any new additions to 'typing' for Python 3.5+ users who are using older versions of 'typing' that were bundled with their standard library (and so can't update to the latest versions). See #435 for motivation and additional context.
In retrospect, attempting to backport typing.Collection was a mistake.
It's difficult in practice to use it in a consistently meaningful way
because the typing module deliberately does not have types like Sequence
or List subclass anything resembling the Collections type for older
versions of Python.
This means that the following code will not typecheck on Python 2
and on older versions of Python 3:
from mypy_extensions import Collection
def foo(x):
# type: Collection[int] -> int
return sum(x)
foo([1, 2, 3])
...because `List` is not defined to be a subclass of `Collection` in
Python 2 and Python 3.x - 3.5 at runtime and within the typing stubs
in typeshed.
I think this is potentially repairable with some careful surgery on
the typing stubs, but doing that seems highly questionable to me,
so I'm thinking attempting to backport `Collection` is out-of-scope
for this pull request and should be handled later (or potentially
never).
Incorporates change made by #439, which was recently merged.
eb24e8f
to
11c0fcf
|
@ilevkivskyi -- as an update, I incorporated the changes you just pushed to master earlier today (removing |
|
@Michael0x2a thanks for continuing to work on this! I will try to make the second review today or tomorrow. |
|
@Michael0x2a |
|
Concerning |
|
OK, just three more minor comments. |
|
|
||
| The ``typing_extensions`` module contains both backports of changes made to | ||
| the ``typing`` module since Python 3.5.0 as well as experimental types that | ||
| will be eventually added to the ``typing`` module once stabilized. |
ilevkivskyi
Jul 18, 2017
Collaborator
I think you forgot to mention that it is provisional until Python 3.7.
I think you forgot to mention that it is provisional until Python 3.7.
| self.assertIsSubclass(collections.Counter, Counter) | ||
| self.assertIs(type(Counter()), collections.Counter) | ||
| self.assertIs(type(Counter[T]()), collections.Counter) | ||
| self.assertIs(type(Counter[int]()), collections.Counter) |
ilevkivskyi
Jul 18, 2017
Collaborator
I actually wanted to also test that Counter.__bases__ are already erased (before instantiation).
I actually wanted to also test that Counter.__bases__ are already erased (before instantiation).
| import typing_extensions | ||
| import collections.abc as collections_abc | ||
|
|
||
| TYPING_LATEST = sys.version_info[:3] < (3, 5, 0) |
ilevkivskyi
Jul 18, 2017
Collaborator
I am not sure I follow the logic here, shouldn't this be TYPING_OLDEST? Or this is the version of typing you "emulate"? Could you please add a comment here?
I am not sure I follow the logic here, shouldn't this be TYPING_OLDEST? Or this is the version of typing you "emulate"? Could you please add a comment here?
- Reworded README/description in setup.py to include note about typing's provisional status ending in Python 3.7. - Made the README/description in setup.py a little shorter in general - Added some explanatory comments - Added a test to check Counter.__bases__ is erased in Python 2.7
|
@ilevkivskyi -- ok, I've addressed your new comments and am ready for the 3rd round of review! Regarding your comment regarding That said, I want to make sure I actually understand both comments correctly -- are the tests I added what you were looking for, or should I need to add in more checks there? |
|
Maybe there are some small things left but it will be easier to fix them in subsequent smaller PRs, so that I am going to merge this soon. I think the next step is to sumbit |
|
@ilevkivskyi stubs for typeshed can be found here: python/typeshed#1471 I'll work on making changes to mypy itself next. iirc it contains some special-casing and tests specifically for mypy_extensions so I'll probably start by duplicating a subset of those for typing_extensions. |
See latest comment in python/typing#443 for justification.
See latest comment in python/typing#443 for justification.

Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.

This pull request adds a
typing_extensionssubproject totyping. Thetyping_extensionsmodule backports any new additions totypingfor Python 3.5+ users who are using older versions oftypingthat were bundled with their standard library (and so can't update to the latest versions).See #435 for more motivation and additional context.