From 42850905f1769026065279272910fdab9f2f3fa2 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 22 Oct 2020 15:33:26 -0700 Subject: [PATCH 01/33] Simplest example: Return a constant --- Doc/howto/descriptor.rst | 47 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index b792b6c6ab77f2..02ad7be6117097 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -7,6 +7,53 @@ Descriptor HowTo Guide .. Contents:: +Preview +^^^^^^^ + +Descriptors let objects customize attribute lookup, storage, and deletion. + +This howto guide has three major sections. It starts with a "primer" that gives a basic overview. The second section explains a complete, practical descriptor example. The third section provides a more technical tutorial that goes into the detailed mechanics of how descriptors work. + +Primer +^^^^^^ + +In this primer, we start with most basic possible example and then we'll add new capabilities one by one. + +Super simple example: A descriptor that returns a constant +---------------------------------------------------------- + +The :class:`Ten` is a descriptor that always returns a constant ``10``:: + + + class Ten: + + def __get__(self, obj, objtype=None): + return 10 + +To use the descriptor, it must be stored as a class variable in another class:: + + class A: + x = 5 # Regular attribute + y = Ten() # Descriptor + +An interactive session shows the difference between normal attribute lookup and descriptor lookup:: + + >>> a = A() # Make an instance of class A + >>> a.x # Normal attribute lookup + 5 + >>> a.y # Descriptor lookup + 10 + +In the ``a.x`` attribute lookup, the dot operator finds the value ``5`` stored in the class dictionary. In the ``a.y`` descriptor lookup, the dot operator calls the :meth:`get()` on the descriptor. That method returns ``10``. Note that the value ``10`` is not stored in either the class dictionary or instance dictionary. The value ``10`` is computed on demand. + + + + + +Technical Tutorial +^^^^^^^^^^^^^^^^^^ + + Abstract -------- From fc4865a92014db2b69a300d1f9780a6a0e1ff637 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 22 Oct 2020 17:11:19 -0700 Subject: [PATCH 02/33] Add directory size example of a dynamic descriptor --- Doc/howto/descriptor.rst | 53 +++++++++++++++++++++++++++++++++------- 1 file changed, 44 insertions(+), 9 deletions(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 02ad7be6117097..451ef8425c6be7 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -19,35 +19,70 @@ Primer In this primer, we start with most basic possible example and then we'll add new capabilities one by one. -Super simple example: A descriptor that returns a constant ----------------------------------------------------------- +Simple example: A descriptor that returns a constant +---------------------------------------------------- -The :class:`Ten` is a descriptor that always returns a constant ``10``:: +The :class:`Ten` class is a descriptor that always returns a constant ``10``:: class Ten: - def __get__(self, obj, objtype=None): return 10 To use the descriptor, it must be stored as a class variable in another class:: class A: - x = 5 # Regular attribute - y = Ten() # Descriptor + x = 5 # Regular class attribute + y = Ten() # Descriptor An interactive session shows the difference between normal attribute lookup and descriptor lookup:: - >>> a = A() # Make an instance of class A - >>> a.x # Normal attribute lookup + >>> a = A() # Make an instance of class A + >>> a.x # Normal attribute lookup 5 - >>> a.y # Descriptor lookup + >>> a.y # Descriptor lookup 10 In the ``a.x`` attribute lookup, the dot operator finds the value ``5`` stored in the class dictionary. In the ``a.y`` descriptor lookup, the dot operator calls the :meth:`get()` on the descriptor. That method returns ``10``. Note that the value ``10`` is not stored in either the class dictionary or instance dictionary. The value ``10`` is computed on demand. +This example shows how a simple descriptor works, but it isn't very useful. For retrieving constants, normal attribute lookup would almost always be better. + +In the next section, we'll create something most useful, a dynamic lookup. + +Dynamic lookups +--------------- + +Interesting descriptors typically run computations instead of doing lookups:: + import os + + class DirectorySize: + + def __get__(self, obj, objtype=None): + return len(os.listdir(obj.dirname)) + + class Directory: + + size = DirectorySize() # Descriptor + + def __init__(self, dirname): + self.dirname = dirname # Regular instance attribute + +An interactive session shows that the lookup is dynamic — it computes different, updated answers each time:: + + >>> g = Directory('games') + >>> s = Directory('songs') + >>> g.size # The games directory has three files + 3 + >>> os.system('touch games/newfile') # Add a fourth file to the directory + 0 + >>> g.size: + 4 + >>> s.size # The songs directory has twenty files + 20 + +Besides showing how descriptors can run computations, this example also reveals the purpose of the parameters to :meth:`__get__`. The *self* parameter is *size*, an instance of *DirectorySize*. The *obj* parameter is either *g* or *s*, an instance of *Directory*. It is *obj* parameter that lets the :meth:`__get__` method learn the target directory. The *objtype* parameter is the class *Directory*. Technical Tutorial From 83d72407f63c18d3c6d6d8de13955bbb96cec3a2 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 22 Oct 2020 18:03:56 -0700 Subject: [PATCH 03/33] Add the managed attributes example --- Doc/howto/descriptor.rst | 63 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 451ef8425c6be7..f5eecab392db49 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -26,6 +26,7 @@ The :class:`Ten` class is a descriptor that always returns a constant ``10``:: class Ten: + def __get__(self, obj, objtype=None): return 10 @@ -85,6 +86,68 @@ An interactive session shows that the lookup is dynamic — it computes differen Besides showing how descriptors can run computations, this example also reveals the purpose of the parameters to :meth:`__get__`. The *self* parameter is *size*, an instance of *DirectorySize*. The *obj* parameter is either *g* or *s*, an instance of *Directory*. It is *obj* parameter that lets the :meth:`__get__` method learn the target directory. The *objtype* parameter is the class *Directory*. +Managed attributes +------------------ + +A popular use for descriptors is managing access to instance data. The descriptor is assigned to a public attribute while the actual data is stored as a private attribute in the instance dictionary. The descriptor's :meth:`__get__` and :meth:`__set__` methods are triggered when the public attribute is accessed. + +In the following example, *age* is the public attribute and *_age* is the private attribute. When the public attribute is accessed, the descriptor logs the lookup or update:: + + import logging + + logging.basicConfig(level=logging.INFO) + + class LoggedAccess: + + def __get__(self, obj, objtype=None): + result = object.__getattribute__(obj, '_age') + logging.info('Accessing %r giving %r', 'age', result) + return result + + def __set__(self, obj, value): + logging.info('Updating %r to %r', 'age', value) + object.__setattr__(obj, '_age', 10) + + class Person: + + age = LoggedAccess() # Descriptor + + def __init__(self, name, age): + self.name = name # Regular instance attribute + self.age = age # Calls the descriptor + + def birthday(self): + self.age += 1 + + +An interactive session shows that all access to the managed attribute *age* is logged, but that the regular attribute *name* is not logged: + + >>> mary = Person('Mary M', 30) # __init__() triggers the descriptor + INFO:root:Updating 'age' to 30 + >>> dave = Person('David D', 40) # So, the initial age update is logged + INFO:root:Updating 'age' to 40 + + >>> vars(mary) # The actual data is in private attributes + {'name': 'Mary M', '_age': 10} + >>> vars(dave) + {'name': 'David D', '_age': 10} + + >>> mary.age # Accesses the data and logs the lookup + INFO:root:Accessing 'age' giving 10 + 10 + >>> mary.birthday() # Updates are logged as well + INFO:root:Accessing 'age' giving 10 + INFO:root:Updating 'age' to 11 + + >>> dave.name # Regular attribute lookup isn't logged + 'David D' + >>> dave.age # Only the managed attribute is logged + INFO:root:Accessing 'age' giving 10 + 10 + +One major issue with this example is the private name *_age* is hardwired in the *LoggedAccess* class. That means that each instance can only have one logged attribute and that its name is unchangeable. In the next example, we'll fix that problem. + + Technical Tutorial ^^^^^^^^^^^^^^^^^^ From 6e1bae3715b2ac33c2ad24a94d58ce759d2ddfd7 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 22 Oct 2020 18:15:17 -0700 Subject: [PATCH 04/33] Add stub sections --- Doc/howto/descriptor.rst | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index f5eecab392db49..e356db0b1798ba 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -147,6 +147,17 @@ An interactive session shows that all access to the managed attribute *age* is l One major issue with this example is the private name *_age* is hardwired in the *LoggedAccess* class. That means that each instance can only have one logged attribute and that its name is unchangeable. In the next example, we'll fix that problem. +Customized Names +---------------- + +XXX: Add the set_name example + + +Complete Practical Example +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +XXX: Add the validators example + Technical Tutorial ^^^^^^^^^^^^^^^^^^ @@ -219,6 +230,11 @@ To make a read-only data descriptor, define both :meth:`__get__` and called. Defining the :meth:`__set__` method with an exception raising placeholder is enough to make it a data descriptor. +Automatic Name Notification +--------------------------- + +XXX: Explain how set_name works. + Invoking Descriptors -------------------- From 5cd1b8d35d21e0836a89ca6e967c532401097481 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 22 Oct 2020 19:01:39 -0700 Subject: [PATCH 05/33] Fix hardwired value --- Doc/howto/descriptor.rst | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index e356db0b1798ba..8a01870290ef77 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -106,7 +106,7 @@ In the following example, *age* is the public attribute and *_age* is the privat def __set__(self, obj, value): logging.info('Updating %r to %r', 'age', value) - object.__setattr__(obj, '_age', 10) + object.__setattr__(obj, '_age', value) class Person: @@ -128,22 +128,22 @@ An interactive session shows that all access to the managed attribute *age* is l INFO:root:Updating 'age' to 40 >>> vars(mary) # The actual data is in private attributes - {'name': 'Mary M', '_age': 10} + {'name': 'Mary M', '_age': 30} >>> vars(dave) - {'name': 'David D', '_age': 10} + {'name': 'David D', '_age': 40} >>> mary.age # Accesses the data and logs the lookup - INFO:root:Accessing 'age' giving 10 - 10 + INFO:root:Accessing 'age' giving 30 + 30 >>> mary.birthday() # Updates are logged as well - INFO:root:Accessing 'age' giving 10 - INFO:root:Updating 'age' to 11 + INFO:root:Accessing 'age' giving 30 + INFO:root:Updating 'age' to 31 >>> dave.name # Regular attribute lookup isn't logged 'David D' >>> dave.age # Only the managed attribute is logged - INFO:root:Accessing 'age' giving 10 - 10 + INFO:root:Accessing 'age' giving 40 + 40 One major issue with this example is the private name *_age* is hardwired in the *LoggedAccess* class. That means that each instance can only have one logged attribute and that its name is unchangeable. In the next example, we'll fix that problem. From e8db25463a2970c85760d9ef6866d2ef0d8ecc97 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 22 Oct 2020 19:36:18 -0700 Subject: [PATCH 06/33] Add the set_name example. --- Doc/howto/descriptor.rst | 81 ++++++++++++++++++++++++++++++++++------ 1 file changed, 69 insertions(+), 12 deletions(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 8a01870290ef77..74778ec0c9f8ac 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -97,7 +97,7 @@ In the following example, *age* is the public attribute and *_age* is the privat logging.basicConfig(level=logging.INFO) - class LoggedAccess: + class LoggedAgeAccess: def __get__(self, obj, objtype=None): result = object.__getattribute__(obj, '_age') @@ -110,11 +110,11 @@ In the following example, *age* is the public attribute and *_age* is the privat class Person: - age = LoggedAccess() # Descriptor + age = LoggedAgeAccess() # Descriptor def __init__(self, name, age): - self.name = name # Regular instance attribute - self.age = age # Calls the descriptor + self.name = name # Regular instance attribute + self.age = age # Calls the descriptor def birthday(self): self.age += 1 @@ -122,26 +122,26 @@ In the following example, *age* is the public attribute and *_age* is the privat An interactive session shows that all access to the managed attribute *age* is logged, but that the regular attribute *name* is not logged: - >>> mary = Person('Mary M', 30) # __init__() triggers the descriptor + >>> mary = Person('Mary M', 30) # __init__() triggers the descriptor INFO:root:Updating 'age' to 30 - >>> dave = Person('David D', 40) # So, the initial age update is logged + >>> dave = Person('David D', 40) # So, the initial age update is logged INFO:root:Updating 'age' to 40 - >>> vars(mary) # The actual data is in private attributes + >>> vars(mary) # The actual data is in private attributes {'name': 'Mary M', '_age': 30} >>> vars(dave) {'name': 'David D', '_age': 40} - >>> mary.age # Accesses the data and logs the lookup + >>> mary.age # Accesses the data and logs the lookup INFO:root:Accessing 'age' giving 30 30 - >>> mary.birthday() # Updates are logged as well + >>> mary.birthday() # Updates are logged as well INFO:root:Accessing 'age' giving 30 INFO:root:Updating 'age' to 31 - >>> dave.name # Regular attribute lookup isn't logged + >>> dave.name # Regular attribute lookup isn't logged 'David D' - >>> dave.age # Only the managed attribute is logged + >>> dave.age # Only the managed attribute is logged INFO:root:Accessing 'age' giving 40 40 @@ -150,7 +150,64 @@ One major issue with this example is the private name *_age* is hardwired in the Customized Names ---------------- -XXX: Add the set_name example +When a class uses descriptors, it can inform each descriptor about what variable name was used. + +In this example, the :class:`Person` class has two descriptor instances, *name* and *age*. When the :class:`Person` class is defined, it makes a callback to :meth:`__set_name__` in *LoggedAccess* so that the field names can be recorded, giving each descriptor its own *public_name* and *private_name*:: + + import logging + + logging.basicConfig(level=logging.INFO) + + class LoggedAccess: + + def __set_name__(self, owner, name): + self.public_name = name + self.private_name = f'_{name}' + + def __get__(self, obj, objtype=None): + result = object.__getattribute__(obj, self.private_name) + logging.info('Accessing %r giving %r', self.public_name, result) + return result + + def __set__(self, obj, value): + logging.info('Updating %r to %r', self.public_name, value) + object.__setattr__(obj, self.private_name, value) + + class Person: + + name = LoggedAccess() # First descriptor + age = LoggedAccess() # Second descriptor + + def __init__(self, name, age): + self.name = name # Calls the first descriptor + self.age = age # Calls the second descriptor + + def birthday(self): + self.age += 1 + +An interactive session shows that the :class:`Person` class has called :meth:`__set_name__` so that the exact field names can be recorded:: + + >>> vars(vars(Person)['name']) + {'public_name': 'name', 'private_name': '_name'} + >>> vars(vars(Person)['age']) + {'public_name': 'age', 'private_name': '_age'} + +The new class now logs access to both *name* and *age*:: + + >>> pete = Person('Peter P', 10) + INFO:root:Updating 'name' to 'Peter P' + INFO:root:Updating 'age' to 10 + >>> kate = Person('Catherine C', 20) + INFO:root:Updating 'name' to 'Catherine C' + INFO:root:Updating 'age' to 20 + +The two *Person* instances contain only the private names:: + + >>> vars(pete) + {'_name': 'Peter P', '_age': 10} + >>> vars(kate) + {'_name': 'Catherine C', '_age': 20} + Complete Practical Example From 36247316f2279e372bbfe9035fb3ba468879af75 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 22 Oct 2020 20:17:35 -0700 Subject: [PATCH 07/33] Add the data validators complete practical example. --- Doc/howto/descriptor.rst | 107 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 106 insertions(+), 1 deletion(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 74778ec0c9f8ac..8d00467307de4a 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -213,7 +213,112 @@ The two *Person* instances contain only the private names:: Complete Practical Example ^^^^^^^^^^^^^^^^^^^^^^^^^^ -XXX: Add the validators example +In this example, we create a practical and powerful tool for locating notoriously hard to find data corruption bugs. + +A validator is a descriptor for managed attribute access. Prior to storing any data, it verifies that the new value meets various type and range restrictions. If those restrictions aren't met, it raises an exception and prevents data corruption at its source. + +This :class:`Validator` class is both an :term:`abstract base class` and a managed attribute descriptor:: + + from abc import ABC, abstractmethod + + class Validator(ABC): + + def __set_name__(self, owner, name): + self.private_name = f'_{name}' + + def __get__(self, obj, objtype=None): + return getattr(obj, self.private_name) + + def __set__(self, obj, value): + self.validate(value) + setattr(obj, self.private_name, value) + + @abstractmethod + def validate(self, value): + pass + +Custom validators need to subclass from :class:`Validator` and supply a :meth:`validate` method to test various restrictions as needed. + +Here are three practical data validation utilities: + +1) :class:`OneOf` verifies that a value is one of a restricted set of options. + +2) :class:`Number` verifies that a value is either an :class:`int` or :class:`float`. Optionally, it verifies that a value is between a given minimum or maximum. + +3) :class:`String` verifies that a value is a :class:`str`. Optionally, it validates a given minimum or maximum length. Optionally, it can test for another predicate as well. + +:: + + class OneOf(Validator): + + def __init__(self, *options): + self.options = set(options) + + def validate(self, value): + if value not in self.options: + raise ValueError(f'Expected {value!r} to be one of {self.options!r}') + + class Number(Validator): + + def __init__(self, minvalue=None, maxvalue=None): + self.minvalue = minvalue + self.maxvalue = maxvalue + + def validate(self, value): + if not isinstance(value, (int, float)): + raise TypeError(f'Expected {value!r} to be an int or float') + if self.minvalue is not None and value < self.minvalue: + raise ValueError( + f'Expected {value!r} to be at least {self.minvalue!r}' + ) + if self.maxvalue is not None and value > self.maxvalue: + raise ValueError + (f'Expected {value!r} to be no more than {self.maxvalue!r}' + ) + + class String(Validator): + + def __init__(self, minsize=None, maxsize=None, predicate=None): + self.minsize = minsize + self.maxsize = maxsize + self.predicate = predicate + + def validate(self, value): + if not isinstance(value, str): + raise TypeError(f'Expected {value!r} to be an str') + if self.minsize is not None and len(value) < self.minsize: + raise ValueError( + f'Expected {value!r} to be no smaller than {self.minsize!r}' + ) + if self.maxsize is not None and len(value) > self.maxsize: + raise ValueError( + f'Expected {value!r} to be no bigger than {self.maxsize!r}' + ) + if self.predicate is not None and not self.predicate(value): + raise ValueError( + f'Expected {self.predicate} to be true for {value!r}' + ) + +Here's how the data validators can be used in a real class:: + + class Component: + + name = String(minsize=3, maxsize=10, predicate=str.upper) + kind = OneOf('plastic', 'metal') + quantity = Number(minvalue=0) + + def __init__(self, name, kind, quantity): + self.name = name + self.kind = kind + self.quantity = quantity + +The descriptors invalid instances from being created:: + + Component('WIDGET', 'metal', 5) # Allowed. + Component('Widget', 'metal', 5) # Blocked: 'Widget' is not all uppercase + Component('WIDGET', 'metle', 5) # Blocked: 'metle' is misspelled + Component('WIDGET', 'metal', -5) # Blocked: -5 is negative + Component('WIDGET', 'metal', 'V') # Blocked: 'V' isn't a number Technical Tutorial From dab7c25a08bf6de2de6fa12cd7ff3b05235a7de5 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 22 Oct 2020 20:35:02 -0700 Subject: [PATCH 08/33] Minor touch-ups --- Doc/howto/descriptor.rst | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 8d00467307de4a..0c86bb5a57c522 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -7,18 +7,24 @@ Descriptor HowTo Guide .. Contents:: -Preview -^^^^^^^ Descriptors let objects customize attribute lookup, storage, and deletion. -This howto guide has three major sections. It starts with a "primer" that gives a basic overview. The second section explains a complete, practical descriptor example. The third section provides a more technical tutorial that goes into the detailed mechanics of how descriptors work. +This HowTo guide has three major sections: + +1) The "primer" gives a basic overview, moving gently from simple examples, adding one feature at a time. It is a great place to start. + +2) The second section shows a complete, practical descriptor example. If you already know the basics, start there. + +3) The third section provides a more technical tutorial that goes into the detailed mechanics of how descriptors work. Most people don't need this level of detal. + Primer ^^^^^^ In this primer, we start with most basic possible example and then we'll add new capabilities one by one. + Simple example: A descriptor that returns a constant ---------------------------------------------------- @@ -26,7 +32,6 @@ The :class:`Ten` class is a descriptor that always returns a constant ``10``:: class Ten: - def __get__(self, obj, objtype=None): return 10 @@ -48,7 +53,8 @@ In the ``a.x`` attribute lookup, the dot operator finds the value ``5`` stored i This example shows how a simple descriptor works, but it isn't very useful. For retrieving constants, normal attribute lookup would almost always be better. -In the next section, we'll create something most useful, a dynamic lookup. +In the next section, we'll create something more useful, a dynamic lookup. + Dynamic lookups --------------- @@ -147,6 +153,7 @@ An interactive session shows that all access to the managed attribute *age* is l One major issue with this example is the private name *_age* is hardwired in the *LoggedAccess* class. That means that each instance can only have one logged attribute and that its name is unchangeable. In the next example, we'll fix that problem. + Customized Names ---------------- @@ -209,7 +216,6 @@ The two *Person* instances contain only the private names:: {'_name': 'Catherine C', '_age': 20} - Complete Practical Example ^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -574,7 +580,7 @@ affect existing client code accessing the attribute directly. The solution is to wrap access to the value attribute in a property data descriptor:: class Cell: - . . . + ... def getvalue(self): "Recalculate the cell before returning value" self.recalc() @@ -601,7 +607,7 @@ non-data descriptors which return bound methods when they are invoked from an object. In pure Python, it works like this:: class Function: - . . . + ... def __get__(self, obj, objtype=None): "Simulate func_descr_get() in Objects/funcobject.c" if obj is None: @@ -732,7 +738,7 @@ is to create alternate class constructors. In Python 2.3, the classmethod Python equivalent is:: class Dict: - . . . + ... def fromkeys(klass, iterable, value=None): "Emulate dict_fromkeys() in Objects/dictobject.c" d = klass() From 95e1513691fff6555d40aca847c8836d9ae6b95e Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 22 Oct 2020 20:50:09 -0700 Subject: [PATCH 09/33] Add technical section for __set_name__ --- Doc/howto/descriptor.rst | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 0c86bb5a57c522..cd8ea89e31b646 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -398,11 +398,6 @@ To make a read-only data descriptor, define both :meth:`__get__` and called. Defining the :meth:`__set__` method with an exception raising placeholder is enough to make it a data descriptor. -Automatic Name Notification ---------------------------- - -XXX: Explain how set_name works. - Invoking Descriptors -------------------- @@ -466,6 +461,17 @@ Likewise, classes can turn-off descriptor invocation by overriding :meth:`__getattribute__()`. +Automatic Name Notification +--------------------------- + +Sometimes it desirable for a descriptor to know what class variable name it was assigned to. When a new class is created, the :class:`type` metaclass scans the dictionary of the new class. If any of the entries are descriptors and if they define :meth:`__set_name__`, that method is called with two arguments. The *owner* is the class where the descriptor is used, the *name* is class variable the descriptor was assigned to. + +The implementation details are in :c:func:`type_new()` and :c:func:`set_names()` in +:source:`Objects/typeobject.c`. + +Since the update logic is in :meth:`type.__new__`, notifications only take place at the time of class creation. If descriptors are added to the class afterwards, :meth:`__set_name__` will need to be called manually. + + Descriptor Example ------------------ From 65a0ab3b9b56c35ab1fcd2298f68660ce3a77dee Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 22 Oct 2020 20:58:13 -0700 Subject: [PATCH 10/33] Line wrap --- Doc/howto/descriptor.rst | 90 ++++++++++++++++++++++++++++++---------- 1 file changed, 67 insertions(+), 23 deletions(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index cd8ea89e31b646..2f9da3da76edb6 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -12,17 +12,22 @@ Descriptors let objects customize attribute lookup, storage, and deletion. This HowTo guide has three major sections: -1) The "primer" gives a basic overview, moving gently from simple examples, adding one feature at a time. It is a great place to start. +1) The "primer" gives a basic overview, moving gently from simple examples, + adding one feature at a time. It is a great place to start. -2) The second section shows a complete, practical descriptor example. If you already know the basics, start there. +2) The second section shows a complete, practical descriptor example. If you + already know the basics, start there. -3) The third section provides a more technical tutorial that goes into the detailed mechanics of how descriptors work. Most people don't need this level of detal. +3) The third section provides a more technical tutorial that goes into the + detailed mechanics of how descriptors work. Most people don't need this + level of detail. Primer ^^^^^^ -In this primer, we start with most basic possible example and then we'll add new capabilities one by one. +In this primer, we start with most basic possible example and then we'll add +new capabilities one by one. Simple example: A descriptor that returns a constant @@ -41,7 +46,8 @@ To use the descriptor, it must be stored as a class variable in another class:: x = 5 # Regular class attribute y = Ten() # Descriptor -An interactive session shows the difference between normal attribute lookup and descriptor lookup:: +An interactive session shows the difference between normal attribute lookup +and descriptor lookup:: >>> a = A() # Make an instance of class A >>> a.x # Normal attribute lookup @@ -49,9 +55,15 @@ An interactive session shows the difference between normal attribute lookup and >>> a.y # Descriptor lookup 10 -In the ``a.x`` attribute lookup, the dot operator finds the value ``5`` stored in the class dictionary. In the ``a.y`` descriptor lookup, the dot operator calls the :meth:`get()` on the descriptor. That method returns ``10``. Note that the value ``10`` is not stored in either the class dictionary or instance dictionary. The value ``10`` is computed on demand. +In the ``a.x`` attribute lookup, the dot operator finds the value ``5`` stored +in the class dictionary. In the ``a.y`` descriptor lookup, the dot operator +calls the :meth:`get()` on the descriptor. That method returns ``10``. Note +that the value ``10`` is not stored in either the class dictionary or instance +dictionary. The value ``10`` is computed on demand. -This example shows how a simple descriptor works, but it isn't very useful. For retrieving constants, normal attribute lookup would almost always be better. +This example shows how a simple descriptor works, but it isn't very useful. +For retrieving constants, normal attribute lookup would almost always be +better. In the next section, we'll create something more useful, a dynamic lookup. @@ -76,7 +88,8 @@ Interesting descriptors typically run computations instead of doing lookups:: def __init__(self, dirname): self.dirname = dirname # Regular instance attribute -An interactive session shows that the lookup is dynamic — it computes different, updated answers each time:: +An interactive session shows that the lookup is dynamic — it computes +different, updated answers each time:: >>> g = Directory('games') >>> s = Directory('songs') @@ -89,15 +102,26 @@ An interactive session shows that the lookup is dynamic — it computes differen >>> s.size # The songs directory has twenty files 20 -Besides showing how descriptors can run computations, this example also reveals the purpose of the parameters to :meth:`__get__`. The *self* parameter is *size*, an instance of *DirectorySize*. The *obj* parameter is either *g* or *s*, an instance of *Directory*. It is *obj* parameter that lets the :meth:`__get__` method learn the target directory. The *objtype* parameter is the class *Directory*. +Besides showing how descriptors can run computations, this example also +reveals the purpose of the parameters to :meth:`__get__`. The *self* +parameter is *size*, an instance of *DirectorySize*. The *obj* parameter is +either *g* or *s*, an instance of *Directory*. It is *obj* parameter that +lets the :meth:`__get__` method learn the target directory. The *objtype* +parameter is the class *Directory*. Managed attributes ------------------ -A popular use for descriptors is managing access to instance data. The descriptor is assigned to a public attribute while the actual data is stored as a private attribute in the instance dictionary. The descriptor's :meth:`__get__` and :meth:`__set__` methods are triggered when the public attribute is accessed. +A popular use for descriptors is managing access to instance data. The +descriptor is assigned to a public attribute while the actual data is stored +as a private attribute in the instance dictionary. The descriptor's +:meth:`__get__` and :meth:`__set__` methods are triggered when the public +attribute is accessed. -In the following example, *age* is the public attribute and *_age* is the private attribute. When the public attribute is accessed, the descriptor logs the lookup or update:: +In the following example, *age* is the public attribute and *_age* is the +private attribute. When the public attribute is accessed, the descriptor logs +the lookup or update:: import logging @@ -126,7 +150,8 @@ In the following example, *age* is the public attribute and *_age* is the privat self.age += 1 -An interactive session shows that all access to the managed attribute *age* is logged, but that the regular attribute *name* is not logged: +An interactive session shows that all access to the managed attribute *age* is +logged, but that the regular attribute *name* is not logged: >>> mary = Person('Mary M', 30) # __init__() triggers the descriptor INFO:root:Updating 'age' to 30 @@ -151,15 +176,22 @@ An interactive session shows that all access to the managed attribute *age* is l INFO:root:Accessing 'age' giving 40 40 -One major issue with this example is the private name *_age* is hardwired in the *LoggedAccess* class. That means that each instance can only have one logged attribute and that its name is unchangeable. In the next example, we'll fix that problem. +One major issue with this example is the private name *_age* is hardwired in +the *LoggedAccess* class. That means that each instance can only have one +logged attribute and that its name is unchangeable. In the next example, +we'll fix that problem. Customized Names ---------------- -When a class uses descriptors, it can inform each descriptor about what variable name was used. +When a class uses descriptors, it can inform each descriptor about what +variable name was used. -In this example, the :class:`Person` class has two descriptor instances, *name* and *age*. When the :class:`Person` class is defined, it makes a callback to :meth:`__set_name__` in *LoggedAccess* so that the field names can be recorded, giving each descriptor its own *public_name* and *private_name*:: +In this example, the :class:`Person` class has two descriptor instances, +*name* and *age*. When the :class:`Person` class is defined, it makes a +callback to :meth:`__set_name__` in *LoggedAccess* so that the field names can +be recorded, giving each descriptor its own *public_name* and *private_name*:: import logging @@ -192,7 +224,8 @@ In this example, the :class:`Person` class has two descriptor instances, *name* def birthday(self): self.age += 1 -An interactive session shows that the :class:`Person` class has called :meth:`__set_name__` so that the exact field names can be recorded:: +An interactive session shows that the :class:`Person` class has called +:meth:`__set_name__` so that the exact field names can be recorded:: >>> vars(vars(Person)['name']) {'public_name': 'name', 'private_name': '_name'} @@ -219,11 +252,16 @@ The two *Person* instances contain only the private names:: Complete Practical Example ^^^^^^^^^^^^^^^^^^^^^^^^^^ -In this example, we create a practical and powerful tool for locating notoriously hard to find data corruption bugs. +In this example, we create a practical and powerful tool for locating +notoriously hard to find data corruption bugs. -A validator is a descriptor for managed attribute access. Prior to storing any data, it verifies that the new value meets various type and range restrictions. If those restrictions aren't met, it raises an exception and prevents data corruption at its source. +A validator is a descriptor for managed attribute access. Prior to storing +any data, it verifies that the new value meets various type and range +restrictions. If those restrictions aren't met, it raises an exception and +prevents data corruption at its source. -This :class:`Validator` class is both an :term:`abstract base class` and a managed attribute descriptor:: +This :class:`Validator` class is both an :term:`abstract base class` and a +managed attribute descriptor:: from abc import ABC, abstractmethod @@ -243,7 +281,8 @@ This :class:`Validator` class is both an :term:`abstract base class` and a manag def validate(self, value): pass -Custom validators need to subclass from :class:`Validator` and supply a :meth:`validate` method to test various restrictions as needed. +Custom validators need to subclass from :class:`Validator` and supply a +:meth:`validate` method to test various restrictions as needed. Here are three practical data validation utilities: @@ -464,10 +503,15 @@ Likewise, classes can turn-off descriptor invocation by overriding Automatic Name Notification --------------------------- -Sometimes it desirable for a descriptor to know what class variable name it was assigned to. When a new class is created, the :class:`type` metaclass scans the dictionary of the new class. If any of the entries are descriptors and if they define :meth:`__set_name__`, that method is called with two arguments. The *owner* is the class where the descriptor is used, the *name* is class variable the descriptor was assigned to. +Sometimes it desirable for a descriptor to know what class variable name it +was assigned to. When a new class is created, the :class:`type` metaclass +scans the dictionary of the new class. If any of the entries are descriptors +and if they define :meth:`__set_name__`, that method is called with two +arguments. The *owner* is the class where the descriptor is used, the *name* +is class variable the descriptor was assigned to. -The implementation details are in :c:func:`type_new()` and :c:func:`set_names()` in -:source:`Objects/typeobject.c`. +The implementation details are in :c:func:`type_new()` and +:c:func:`set_names()` in :source:`Objects/typeobject.c`. Since the update logic is in :meth:`type.__new__`, notifications only take place at the time of class creation. If descriptors are added to the class afterwards, :meth:`__set_name__` will need to be called manually. From 8e5f962d4d5d0ab1bfdfbaf96ccc0b51dd444fca Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 22 Oct 2020 21:02:27 -0700 Subject: [PATCH 11/33] Use the @-notation where possible --- Doc/howto/descriptor.rst | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 2f9da3da76edb6..3659cc20539bf2 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -631,11 +631,12 @@ to wrap access to the value attribute in a property data descriptor:: class Cell: ... + + @property def getvalue(self): "Recalculate the cell before returning value" self.recalc() return self._value - value = property(getvalue) Functions and Methods @@ -658,6 +659,7 @@ object. In pure Python, it works like this:: class Function: ... + def __get__(self, obj, objtype=None): "Simulate func_descr_get() in Objects/funcobject.c" if obj is None: @@ -745,9 +747,9 @@ Since staticmethods return the underlying function with no changes, the example calls are unexciting:: >>> class E: + ... @staticmethod ... def f(x): ... print(x) - ... f = staticmethod(f) ... >>> E.f(3) 3 @@ -771,9 +773,9 @@ argument list before calling the function. This format is the same for whether the caller is an object or a class:: >>> class E: + ... @classmethod ... def f(klass, x): ... return klass.__name__, x - ... f = classmethod(f) ... >>> print(E.f(3)) ('E', 3) @@ -789,13 +791,14 @@ Python equivalent is:: class Dict: ... + + @classmethod def fromkeys(klass, iterable, value=None): "Emulate dict_fromkeys() in Objects/dictobject.c" d = klass() for key in iterable: d[key] = value return d - fromkeys = classmethod(fromkeys) Now a new dictionary of unique keys can be constructed like this:: From 7f52aee1628589be18c4eaa5047d53583ab9bffa Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 22 Oct 2020 21:04:09 -0700 Subject: [PATCH 12/33] Modern style uses "cls" instead of "klass" --- Doc/howto/descriptor.rst | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 3659cc20539bf2..38f876a9d4aa12 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -709,7 +709,7 @@ patterns of binding functions into methods. To recap, functions have a :meth:`__get__` method so that they can be converted to a method when accessed as attributes. The non-data descriptor transforms an -``obj.f(*args)`` call into ``f(obj, *args)``. Calling ``klass.f(*args)`` +``obj.f(*args)`` call into ``f(obj, *args)``. Calling ``cls.f(*args)`` becomes ``f(*args)``. This chart summarizes the binding and its two most useful variants: @@ -722,7 +722,7 @@ This chart summarizes the binding and its two most useful variants: +-----------------+----------------------+------------------+ | staticmethod | f(\*args) | f(\*args) | +-----------------+----------------------+------------------+ - | classmethod | f(type(obj), \*args) | f(klass, \*args) | + | classmethod | f(type(obj), \*args) | f(cls, \*args) | +-----------------+----------------------+------------------+ Static methods return the underlying function without changes. Calling either @@ -774,8 +774,8 @@ for whether the caller is an object or a class:: >>> class E: ... @classmethod - ... def f(klass, x): - ... return klass.__name__, x + ... def f(cls, x): + ... return cls.__name__, x ... >>> print(E.f(3)) ('E', 3) @@ -793,9 +793,9 @@ Python equivalent is:: ... @classmethod - def fromkeys(klass, iterable, value=None): + def fromkeys(cls, iterable, value=None): "Emulate dict_fromkeys() in Objects/dictobject.c" - d = klass() + d = cls() for key in iterable: d[key] = value return d @@ -814,10 +814,10 @@ Using the non-data descriptor protocol, a pure Python version of def __init__(self, f): self.f = f - def __get__(self, obj, klass=None): - if klass is None: - klass = type(obj) + def __get__(self, obj, cls=None): + if cls is None: + cls = type(obj) def newfunc(*args): - return self.f(klass, *args) + return self.f(cls, *args) return newfunc From 48329d21646c4d4f02213ad64ad61a2ef6021109 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 22 Oct 2020 21:06:20 -0700 Subject: [PATCH 13/33] Fix typo --- Doc/howto/descriptor.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 38f876a9d4aa12..07d9df780b8602 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -357,7 +357,7 @@ Here's how the data validators can be used in a real class:: self.kind = kind self.quantity = quantity -The descriptors invalid instances from being created:: +The descriptors prevent invalid instances from being created:: Component('WIDGET', 'metal', 5) # Allowed. Component('Widget', 'metal', 5) # Blocked: 'Widget' is not all uppercase From 1583c25a6e6c673f2a52be9303a05678a53a94e7 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 22 Oct 2020 21:19:11 -0700 Subject: [PATCH 14/33] Missing colon --- Doc/howto/descriptor.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 07d9df780b8602..32113d5c9fa71b 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -151,7 +151,7 @@ the lookup or update:: An interactive session shows that all access to the managed attribute *age* is -logged, but that the regular attribute *name* is not logged: +logged, but that the regular attribute *name* is not logged:: >>> mary = Person('Mary M', 30) # __init__() triggers the descriptor INFO:root:Updating 'age' to 30 From 45466fcba90ef763026f08fc51568910b93ee19b Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 22 Oct 2020 21:52:02 -0700 Subject: [PATCH 15/33] Note false positive in the suspcious entry extension --- Doc/tools/susp-ignored.csv | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv index 99d6decc4ece17..e8606e145808ed 100644 --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -23,6 +23,8 @@ howto/curses,,:blue,"2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and 7:white. howto/curses,,:magenta,"2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and 7:white. The" howto/curses,,:cyan,"2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and 7:white. The" howto/curses,,:white,"2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and 7:white. The" +how/descriptors,,:INFO,"INFO:root:Updating" +how/descriptors,,:INFO,"INFO:root:Accessing" howto/instrumentation,,::,python$target:::function-entry howto/instrumentation,,:function,python$target:::function-entry howto/instrumentation,,::,python$target:::function-return From 31fb2306f4d7736ccc9f86e6c4dce440f320d815 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 22 Oct 2020 22:02:05 -0700 Subject: [PATCH 16/33] Note false positive in the suspcious entry extension --- Doc/tools/susp-ignored.csv | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv index e8606e145808ed..cb87b99e9f7015 100644 --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -23,8 +23,8 @@ howto/curses,,:blue,"2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and 7:white. howto/curses,,:magenta,"2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and 7:white. The" howto/curses,,:cyan,"2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and 7:white. The" howto/curses,,:white,"2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and 7:white. The" -how/descriptors,,:INFO,"INFO:root:Updating" -how/descriptors,,:INFO,"INFO:root:Accessing" +how/descriptors,,:root,"INFO:root:Updating" +how/descriptors,,:root,"INFO:root:Accessing" howto/instrumentation,,::,python$target:::function-entry howto/instrumentation,,:function,python$target:::function-entry howto/instrumentation,,::,python$target:::function-return From 58275e7f620f8e45336daef4ad0de450f8f59efb Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 22 Oct 2020 22:13:10 -0700 Subject: [PATCH 17/33] Note false positive in the suspcious entry extension --- Doc/tools/susp-ignored.csv | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv index cb87b99e9f7015..0e432541de809e 100644 --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -23,8 +23,12 @@ howto/curses,,:blue,"2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and 7:white. howto/curses,,:magenta,"2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and 7:white. The" howto/curses,,:cyan,"2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and 7:white. The" howto/curses,,:white,"2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and 7:white. The" -how/descriptors,,:root,"INFO:root:Updating" -how/descriptors,,:root,"INFO:root:Accessing" +how/descriptor,,:root,"INFO:root:Updating 'age' to 30" +how/descriptor,,:root,"INFO:root:Updating 'age' to 40" +how/descriptor,,:root,"INFO:root:Accessing 'age' giving 30" +how/descriptor,,:root,"INFO:root:Accessing 'age' giving 30" +how/descriptor,,:root,"INFO:root:Updating 'age' to 31" +how/descriptor,,:root,"INFO:root:Accessing 'age' giving 40" howto/instrumentation,,::,python$target:::function-entry howto/instrumentation,,:function,python$target:::function-entry howto/instrumentation,,::,python$target:::function-return From 80bfd0823b0fa359a450816a55e75c44a3c9db58 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 22 Oct 2020 22:31:38 -0700 Subject: [PATCH 18/33] Note false positive in the suspcious entry extension --- Doc/tools/susp-ignored.csv | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv index 0e432541de809e..b15fd32b357f03 100644 --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -23,12 +23,9 @@ howto/curses,,:blue,"2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and 7:white. howto/curses,,:magenta,"2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and 7:white. The" howto/curses,,:cyan,"2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and 7:white. The" howto/curses,,:white,"2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and 7:white. The" -how/descriptor,,:root,"INFO:root:Updating 'age' to 30" -how/descriptor,,:root,"INFO:root:Updating 'age' to 40" -how/descriptor,,:root,"INFO:root:Accessing 'age' giving 30" -how/descriptor,,:root,"INFO:root:Accessing 'age' giving 30" -how/descriptor,,:root,"INFO:root:Updating 'age' to 31" -how/descriptor,,:root,"INFO:root:Accessing 'age' giving 40" +howto/descriptor,,:root,"INFO:root" +howto/descriptor,,:Updating,"root:Updating" +howto/descriptor,,:Accessing,"root:Accessing" howto/instrumentation,,::,python$target:::function-entry howto/instrumentation,,:function,python$target:::function-entry howto/instrumentation,,::,python$target:::function-return From 112a2728c0e350a02687b8cd76be65aa67402c65 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Fri, 23 Oct 2020 00:42:32 -0700 Subject: [PATCH 19/33] Fix typos. Minor grammar edits. --- Doc/howto/descriptor.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 32113d5c9fa71b..1de8cf9e6018dc 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -33,7 +33,7 @@ new capabilities one by one. Simple example: A descriptor that returns a constant ---------------------------------------------------- -The :class:`Ten` class is a descriptor that always returns a constant ``10``:: +The :class:`Ten` class is a descriptor that always returns the constant ``10``:: class Ten: @@ -57,9 +57,9 @@ and descriptor lookup:: In the ``a.x`` attribute lookup, the dot operator finds the value ``5`` stored in the class dictionary. In the ``a.y`` descriptor lookup, the dot operator -calls the :meth:`get()` on the descriptor. That method returns ``10``. Note +calls the descriptor's :meth:`get()` method. That method returns ``10``. Note that the value ``10`` is not stored in either the class dictionary or instance -dictionary. The value ``10`` is computed on demand. +dictionary. Instead, the value ``10`` is computed on demand. This example shows how a simple descriptor works, but it isn't very useful. For retrieving constants, normal attribute lookup would almost always be @@ -97,7 +97,7 @@ different, updated answers each time:: 3 >>> os.system('touch games/newfile') # Add a fourth file to the directory 0 - >>> g.size: + >>> g.size 4 >>> s.size # The songs directory has twenty files 20 @@ -177,7 +177,7 @@ logged, but that the regular attribute *name* is not logged:: 40 One major issue with this example is the private name *_age* is hardwired in -the *LoggedAccess* class. That means that each instance can only have one +the *LoggedAgeAccess* class. That means that each instance can only have one logged attribute and that its name is unchangeable. In the next example, we'll fix that problem. @@ -257,7 +257,7 @@ notoriously hard to find data corruption bugs. A validator is a descriptor for managed attribute access. Prior to storing any data, it verifies that the new value meets various type and range -restrictions. If those restrictions aren't met, it raises an exception and +restrictions. If those restrictions aren't met, it raises an exception to prevents data corruption at its source. This :class:`Validator` class is both an :term:`abstract base class` and a From 1a899c8201e2240b53cbaa177e24b185aa1e6fa6 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Fri, 23 Oct 2020 00:47:16 -0700 Subject: [PATCH 20/33] Fix method name --- Doc/howto/descriptor.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 1de8cf9e6018dc..212cf5587f55b7 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -57,7 +57,7 @@ and descriptor lookup:: In the ``a.x`` attribute lookup, the dot operator finds the value ``5`` stored in the class dictionary. In the ``a.y`` descriptor lookup, the dot operator -calls the descriptor's :meth:`get()` method. That method returns ``10``. Note +calls the descriptor's :meth:`__get__()` method. That method returns ``10``. Note that the value ``10`` is not stored in either the class dictionary or instance dictionary. Instead, the value ``10`` is computed on demand. From b3934e85409f7771657ffa6e0e6794c6027d206c Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Fri, 23 Oct 2020 00:57:56 -0700 Subject: [PATCH 21/33] Fix SyntaxError and misspelled predicate name --- Doc/howto/descriptor.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 212cf5587f55b7..159537d079089c 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -317,8 +317,8 @@ Here are three practical data validation utilities: f'Expected {value!r} to be at least {self.minvalue!r}' ) if self.maxvalue is not None and value > self.maxvalue: - raise ValueError - (f'Expected {value!r} to be no more than {self.maxvalue!r}' + raise ValueError( + f'Expected {value!r} to be no more than {self.maxvalue!r}' ) class String(Validator): @@ -348,7 +348,7 @@ Here's how the data validators can be used in a real class:: class Component: - name = String(minsize=3, maxsize=10, predicate=str.upper) + name = String(minsize=3, maxsize=10, predicate=str.isupper) kind = OneOf('plastic', 'metal') quantity = Number(minvalue=0) @@ -503,7 +503,7 @@ Likewise, classes can turn-off descriptor invocation by overriding Automatic Name Notification --------------------------- -Sometimes it desirable for a descriptor to know what class variable name it +Sometimes it is desirable for a descriptor to know what class variable name it was assigned to. When a new class is created, the :class:`type` metaclass scans the dictionary of the new class. If any of the entries are descriptors and if they define :meth:`__set_name__`, that method is called with two From 90992bfc5b67dda9eefcaf5ed03f21f9ab1baa15 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Fri, 23 Oct 2020 01:11:32 -0700 Subject: [PATCH 22/33] Add references to and from the glossary --- Doc/glossary.rst | 3 ++- Doc/howto/descriptor.rst | 5 ++++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 4f0654b3254e4b..32aa12a200f636 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -301,7 +301,8 @@ Glossary including functions, methods, properties, class methods, static methods, and reference to super classes. - For more information about descriptors' methods, see :ref:`descriptors`. + For more information about descriptors' methods, see :ref:`descriptors` + or the :ref:`Descriptor How To Guide `. dictionary An associative array, where arbitrary keys are mapped to values. The diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 159537d079089c..bb3f0415deb7d9 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -1,3 +1,5 @@ +.. _descriptorhowto: + ====================== Descriptor HowTo Guide ====================== @@ -8,7 +10,8 @@ Descriptor HowTo Guide .. Contents:: -Descriptors let objects customize attribute lookup, storage, and deletion. +:term:`Decriptors ` let objects customize attribute lookup, + storage, and deletion. This HowTo guide has three major sections: From 2599cff41bd4c6655056a200219ebb08678d52c0 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Fri, 23 Oct 2020 01:18:21 -0700 Subject: [PATCH 23/33] Fix markup --- Doc/howto/descriptor.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index bb3f0415deb7d9..055aea004867d2 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -11,7 +11,7 @@ Descriptor HowTo Guide :term:`Decriptors ` let objects customize attribute lookup, - storage, and deletion. +storage, and deletion. This HowTo guide has three major sections: From 11e790ae95e26962d5f447ca9a8ee631842ffa1a Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Fri, 23 Oct 2020 06:42:44 -0700 Subject: [PATCH 24/33] Update Doc/howto/descriptor.rst Co-authored-by: Eduardo Orive Vinuesa --- Doc/howto/descriptor.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 055aea004867d2..a3cd4c7d801235 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -10,7 +10,7 @@ Descriptor HowTo Guide .. Contents:: -:term:`Decriptors ` let objects customize attribute lookup, +:term:`Descriptors ` let objects customize attribute lookup, storage, and deletion. This HowTo guide has three major sections: @@ -823,4 +823,3 @@ Using the non-data descriptor protocol, a pure Python version of def newfunc(*args): return self.f(cls, *args) return newfunc - From b0c435caee4a49b2ddfb6b8285ef174e9094d790 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Fri, 23 Oct 2020 07:31:24 -0700 Subject: [PATCH 25/33] Minor comment and variable name improvements --- Doc/howto/descriptor.rst | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index a3cd4c7d801235..14b4ffb74b478f 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -65,8 +65,7 @@ that the value ``10`` is not stored in either the class dictionary or instance dictionary. Instead, the value ``10`` is computed on demand. This example shows how a simple descriptor works, but it isn't very useful. -For retrieving constants, normal attribute lookup would almost always be -better. +For retrieving constants, normal attribute lookup would be better. In the next section, we'll create something more useful, a dynamic lookup. @@ -150,15 +149,15 @@ the lookup or update:: self.age = age # Calls the descriptor def birthday(self): - self.age += 1 + self.age += 1 # Calls both __get__() and __set__() An interactive session shows that all access to the managed attribute *age* is logged, but that the regular attribute *name* is not logged:: - >>> mary = Person('Mary M', 30) # __init__() triggers the descriptor + >>> mary = Person('Mary M', 30) # The initial age update is logged INFO:root:Updating 'age' to 30 - >>> dave = Person('David D', 40) # So, the initial age update is logged + >>> dave = Person('David D', 40) INFO:root:Updating 'age' to 40 >>> vars(mary) # The actual data is in private attributes @@ -166,7 +165,7 @@ logged, but that the regular attribute *name* is not logged:: >>> vars(dave) {'name': 'David D', '_age': 40} - >>> mary.age # Accesses the data and logs the lookup + >>> mary.age # Access the data and log the lookup INFO:root:Accessing 'age' giving 30 30 >>> mary.birthday() # Updates are logged as well @@ -207,9 +206,9 @@ be recorded, giving each descriptor its own *public_name* and *private_name*:: self.private_name = f'_{name}' def __get__(self, obj, objtype=None): - result = object.__getattribute__(obj, self.private_name) - logging.info('Accessing %r giving %r', self.public_name, result) - return result + value = object.__getattribute__(obj, self.private_name) + logging.info('Accessing %r giving %r', self.public_name, value) + return value def __set__(self, obj, value): logging.info('Updating %r to %r', self.public_name, value) @@ -228,7 +227,7 @@ be recorded, giving each descriptor its own *public_name* and *private_name*:: self.age += 1 An interactive session shows that the :class:`Person` class has called -:meth:`__set_name__` so that the exact field names can be recorded:: +:meth:`__set_name__` so that the field names would be recorded:: >>> vars(vars(Person)['name']) {'public_name': 'name', 'private_name': '_name'} From 0f6df85be8fb388856cd19288cb270c026aa7ff2 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Fri, 23 Oct 2020 08:16:28 -0700 Subject: [PATCH 26/33] Simplify examples. Add closing thoughts section. --- Doc/howto/descriptor.rst | 34 ++++++++++++++++++++++++++++------ 1 file changed, 28 insertions(+), 6 deletions(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 14b4ffb74b478f..17c6b6bc562fe1 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -132,13 +132,13 @@ the lookup or update:: class LoggedAgeAccess: def __get__(self, obj, objtype=None): - result = object.__getattribute__(obj, '_age') - logging.info('Accessing %r giving %r', 'age', result) - return result + value = obj._age + logging.info('Accessing %r giving %r', 'age', value) + return value def __set__(self, obj, value): logging.info('Updating %r to %r', 'age', value) - object.__setattr__(obj, '_age', value) + obj._age = value class Person: @@ -206,13 +206,13 @@ be recorded, giving each descriptor its own *public_name* and *private_name*:: self.private_name = f'_{name}' def __get__(self, obj, objtype=None): - value = object.__getattribute__(obj, self.private_name) + value = getattr(obj, self.private_name) logging.info('Accessing %r giving %r', self.public_name, value) return value def __set__(self, obj, value): logging.info('Updating %r to %r', self.public_name, value) - object.__setattr__(obj, self.private_name, value) + setattr(obj, self.private_name, value) class Person: @@ -251,6 +251,28 @@ The two *Person* instances contain only the private names:: {'_name': 'Catherine C', '_age': 20} +Closing thoughts +---------------- + +A :term:`descriptor` is any object that defines :meth:`__get__`, +:meth:`__set__`, and/or :meth:`__delete__`. + +A descriptor gets invoked by the dot operator during attribute lookup. If a +descriptor is accessed indirectly with ``vars(some_class)[descriptor_name]``, +it has no effect. + +Descriptors only work when used as class variables. When put in instances, +they have no effect. + +The main motivation for descriptors is that it lets objects control what +happens to them during dotted lookup. + +Descriptors are used throughout the language. It is how functions turn into +bound methods. Common tools like :func:`classmethod`, :func:`staticmethod`, +:func:`property`, and :func:`functools.cached_property` are all implemented as +descriptors. + + Complete Practical Example ^^^^^^^^^^^^^^^^^^^^^^^^^^ From 1e9482e7678200af2e0e2197ed6f6d0ff39cd40f Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Fri, 23 Oct 2020 08:46:43 -0700 Subject: [PATCH 27/33] Add more section headings --- Doc/howto/descriptor.rst | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 17c6b6bc562fe1..a56548e0a7f6a5 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -279,6 +279,10 @@ Complete Practical Example In this example, we create a practical and powerful tool for locating notoriously hard to find data corruption bugs. + +Validator class +--------------- + A validator is a descriptor for managed attribute access. Prior to storing any data, it verifies that the new value meets various type and range restrictions. If those restrictions aren't met, it raises an exception to @@ -308,6 +312,10 @@ managed attribute descriptor:: Custom validators need to subclass from :class:`Validator` and supply a :meth:`validate` method to test various restrictions as needed. + +Custom validators +----------------- + Here are three practical data validation utilities: 1) :class:`OneOf` verifies that a value is one of a restricted set of options. @@ -368,6 +376,10 @@ Here are three practical data validation utilities: f'Expected {self.predicate} to be true for {value!r}' ) + +Practical use +------------- + Here's how the data validators can be used in a real class:: class Component: @@ -393,6 +405,9 @@ The descriptors prevent invalid instances from being created:: Technical Tutorial ^^^^^^^^^^^^^^^^^^ +What follows is a more technical tutorial for the mechanics and details of how +descriptors work. + Abstract -------- From 512fa4b9bd43ef9beb7a242c04a50b565fbb3a69 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Fri, 23 Oct 2020 09:58:00 -0700 Subject: [PATCH 28/33] More wordsmithing --- Doc/howto/descriptor.rst | 31 ++++++++++++++++--------------- 1 file changed, 16 insertions(+), 15 deletions(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index a56548e0a7f6a5..624a4dd055c8fb 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -60,9 +60,9 @@ and descriptor lookup:: In the ``a.x`` attribute lookup, the dot operator finds the value ``5`` stored in the class dictionary. In the ``a.y`` descriptor lookup, the dot operator -calls the descriptor's :meth:`__get__()` method. That method returns ``10``. Note -that the value ``10`` is not stored in either the class dictionary or instance -dictionary. Instead, the value ``10`` is computed on demand. +calls the descriptor's :meth:`__get__()` method. That method returns ``10``. +Note that the value ``10`` is not stored in either the class dictionary or the +instance dictionary. Instead, the value ``10`` is computed on demand. This example shows how a simple descriptor works, but it isn't very useful. For retrieving constants, normal attribute lookup would be better. @@ -116,10 +116,10 @@ Managed attributes ------------------ A popular use for descriptors is managing access to instance data. The -descriptor is assigned to a public attribute while the actual data is stored -as a private attribute in the instance dictionary. The descriptor's -:meth:`__get__` and :meth:`__set__` methods are triggered when the public -attribute is accessed. +descriptor is assigned to a public attribute in the class dictionary while the +actual data is stored as a private attribute in the instance dictionary. The +descriptor's :meth:`__get__` and :meth:`__set__` methods are triggered when +the public attribute is accessed. In the following example, *age* is the public attribute and *_age* is the private attribute. When the public attribute is accessed, the descriptor logs @@ -160,7 +160,7 @@ logged, but that the regular attribute *name* is not logged:: >>> dave = Person('David D', 40) INFO:root:Updating 'age' to 40 - >>> vars(mary) # The actual data is in private attributes + >>> vars(mary) # The actual data is in a private attribute {'name': 'Mary M', '_age': 30} >>> vars(dave) {'name': 'David D', '_age': 40} @@ -227,7 +227,8 @@ be recorded, giving each descriptor its own *public_name* and *private_name*:: self.age += 1 An interactive session shows that the :class:`Person` class has called -:meth:`__set_name__` so that the field names would be recorded:: +:meth:`__set_name__` so that the field names would be recorded. Here +we call :func:`vars` to lookup the descriptor without triggering it:: >>> vars(vars(Person)['name']) {'public_name': 'name', 'private_name': '_name'} @@ -254,18 +255,18 @@ The two *Person* instances contain only the private names:: Closing thoughts ---------------- -A :term:`descriptor` is any object that defines :meth:`__get__`, -:meth:`__set__`, and/or :meth:`__delete__`. +A :term:`descriptor` is what we call any object that defines :meth:`__get__`, +:meth:`__set__`, or :meth:`__delete__`. -A descriptor gets invoked by the dot operator during attribute lookup. If a +Descriptors get invoked by the dot operator during attribute lookup. If a descriptor is accessed indirectly with ``vars(some_class)[descriptor_name]``, -it has no effect. +the descriptor instance is returned without invoking it. Descriptors only work when used as class variables. When put in instances, they have no effect. -The main motivation for descriptors is that it lets objects control what -happens to them during dotted lookup. +The main motivation for descriptors is to let objects control what happens +during dotted lookup. Descriptors are used throughout the language. It is how functions turn into bound methods. Common tools like :func:`classmethod`, :func:`staticmethod`, From 0dc5f723fb7aad280494581ab4426152bca9ce9a Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Fri, 23 Oct 2020 10:21:01 -0700 Subject: [PATCH 29/33] Wrap long lines --- Doc/howto/descriptor.rst | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 624a4dd055c8fb..041c2bdfe73315 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -321,9 +321,13 @@ Here are three practical data validation utilities: 1) :class:`OneOf` verifies that a value is one of a restricted set of options. -2) :class:`Number` verifies that a value is either an :class:`int` or :class:`float`. Optionally, it verifies that a value is between a given minimum or maximum. +2) :class:`Number` verifies that a value is either an :class:`int` or + :class:`float`. Optionally, it verifies that a value is between a given + minimum or maximum. -3) :class:`String` verifies that a value is a :class:`str`. Optionally, it validates a given minimum or maximum length. Optionally, it can test for another predicate as well. +3) :class:`String` verifies that a value is a :class:`str`. Optionally, it + validates a given minimum or maximum length. Optionally, it can test for + another predicate as well. :: @@ -553,7 +557,9 @@ is class variable the descriptor was assigned to. The implementation details are in :c:func:`type_new()` and :c:func:`set_names()` in :source:`Objects/typeobject.c`. -Since the update logic is in :meth:`type.__new__`, notifications only take place at the time of class creation. If descriptors are added to the class afterwards, :meth:`__set_name__` will need to be called manually. +Since the update logic is in :meth:`type.__new__`, notifications only take +place at the time of class creation. If descriptors are added to the class +afterwards, :meth:`__set_name__` will need to be called manually. Descriptor Example From 10b9cd4db8a9f6b22b34c610d8c086c66c8856f1 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Fri, 23 Oct 2020 10:36:01 -0700 Subject: [PATCH 30/33] Clarify the motivation and the caller/callee relationship --- Doc/howto/descriptor.rst | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 041c2bdfe73315..1df0ab82d0670f 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -265,8 +265,12 @@ the descriptor instance is returned without invoking it. Descriptors only work when used as class variables. When put in instances, they have no effect. -The main motivation for descriptors is to let objects control what happens -during dotted lookup. +The main motivation for descriptors is to provide a hook allowing objects +stored in class variables to control what happens during dotted lookup. + +Traditionally, the calling class controls what happens during lookup. +Descriptors invert that relationship and allow the data being looked-up to +have a say in the matter. Descriptors are used throughout the language. It is how functions turn into bound methods. Common tools like :func:`classmethod`, :func:`staticmethod`, From 12b0fa260155dc0500411e6ecd797b0c846af860 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Fri, 23 Oct 2020 11:55:49 -0700 Subject: [PATCH 31/33] Beautify technical tutorial sections --- Doc/howto/descriptor.rst | 54 ++++++++++++++++++++-------------------- 1 file changed, 27 insertions(+), 27 deletions(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 1df0ab82d0670f..de4a5661490803 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -543,7 +543,7 @@ The implementation details are in :c:func:`super_getattro()` in The details above show that the mechanism for descriptors is embedded in the :meth:`__getattribute__()` methods for :class:`object`, :class:`type`, and :func:`super`. Classes inherit this machinery when they derive from -:class:`object` or if they have a meta-class providing similar functionality. +:class:`object` or if they have a metaclass providing similar functionality. Likewise, classes can turn-off descriptor invocation by overriding :meth:`__getattribute__()`. @@ -583,7 +583,7 @@ descriptor is useful for monitoring just a few chosen attributes:: self.val = initval self.name = name - def __get__(self, obj, objtype): + def __get__(self, obj, objtype=None): print('Retrieving', self.name) return self.val @@ -591,11 +591,11 @@ descriptor is useful for monitoring just a few chosen attributes:: print('Updating', self.name) self.val = val - >>> class MyClass: - ... x = RevealAccess(10, 'var "x"') - ... y = 5 - ... - >>> m = MyClass() + class B: + x = RevealAccess(10, 'var "x"') + y = 5 + + >>> m = B() >>> m.x Retrieving var "x" 10 @@ -683,7 +683,7 @@ to wrap access to the value attribute in a property data descriptor:: ... @property - def getvalue(self): + def value(self): "Recalculate the cell before returning value" self.recalc() return self._value @@ -718,10 +718,10 @@ object. In pure Python, it works like this:: Running the interpreter shows how the function descriptor works in practice:: - >>> class D: - ... def f(self, x): - ... return x - ... + class D: + def f(self, x): + return x + >>> d = D() # Access through the class dictionary does not invoke __get__. @@ -766,7 +766,7 @@ This chart summarizes the binding and its two most useful variants: +-----------------+----------------------+------------------+ | Transformation | Called from an | Called from a | - | | Object | Class | + | | object | class | +=================+======================+==================+ | function | f(obj, \*args) | f(\*args) | +-----------------+----------------------+------------------+ @@ -796,11 +796,11 @@ It can be called either from an object or the class: ``s.erf(1.5) --> .9332`` o Since staticmethods return the underlying function with no changes, the example calls are unexciting:: - >>> class E: - ... @staticmethod - ... def f(x): - ... print(x) - ... + class E: + @staticmethod + def f(x): + print(x) + >>> E.f(3) 3 >>> E().f(3) @@ -822,15 +822,15 @@ Unlike static methods, class methods prepend the class reference to the argument list before calling the function. This format is the same for whether the caller is an object or a class:: - >>> class E: - ... @classmethod - ... def f(cls, x): - ... return cls.__name__, x - ... - >>> print(E.f(3)) - ('E', 3) - >>> print(E().f(3)) - ('E', 3) + class F: + @classmethod + def f(cls, x): + return cls.__name__, x + + >>> print(F.f(3)) + ('F', 3) + >>> print(F().f(3)) + ('F', 3) This behavior is useful whenever the function only needs to have a class From 8ca8c77e324cf6f013f04e3c67978a2f1e53d451 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Fri, 23 Oct 2020 12:13:32 -0700 Subject: [PATCH 32/33] Move comments of the the code and into the main text --- Doc/howto/descriptor.rst | 27 ++++++++++++++++----------- 1 file changed, 16 insertions(+), 11 deletions(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index de4a5661490803..1d84b6b63cc9a5 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -716,35 +716,40 @@ object. In pure Python, it works like this:: return self return types.MethodType(self, obj) -Running the interpreter shows how the function descriptor works in practice:: +Running the following in class in the interpreter shows how the function +descriptor works in practice:: class D: def f(self, x): return x - >>> d = D() +Access through the class dictionary does not invoke :meth:`__get__`. Instead, +it just returns the underlying function object:: - # Access through the class dictionary does not invoke __get__. - # It just returns the underlying function object. >>> D.__dict__['f'] - # Dotted access from a class calls __get__() which just returns - # the underlying function unchanged. +Dotted access from a class calls :meth:`__get__` which just returns the +underlying function unchanged:: + >>> D.f - # The function has a __qualname__ attribute to support introspection +The function has a :term:`qualified name` attribute to support introspection:: + >>> D.f.__qualname__ 'D.f' - # Dotted access from an instance calls __get__() which returns the - # function wrapped in a bound method object +Dotted access from an instance calls :meth:`__get__` which returns a bound +method object:: + + >>> d = D() >>> d.f > - # Internally, the bound method stores the underlying function and - # the bound instance. +Internally, the bound method stores the underlying function and the bound +instance:: + >>> d.f.__func__ >>> d.f.__self__ From cc22b88421e8042960a8d73c8dfc709cf9cacfe6 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Fri, 23 Oct 2020 12:37:00 -0700 Subject: [PATCH 33/33] Remove outdated references to Python 2 and new-style classes --- Doc/howto/descriptor.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 1d84b6b63cc9a5..4a53b9e6156922 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -450,10 +450,10 @@ Where this occurs in the precedence chain depends on which descriptor methods were defined. Descriptors are a powerful, general purpose protocol. They are the mechanism -behind properties, methods, static methods, class methods, and :func:`super()`. -They are used throughout Python itself to implement the new style classes -introduced in version 2.2. Descriptors simplify the underlying C-code and offer -a flexible set of new tools for everyday Python programs. +behind properties, methods, static methods, class methods, and +:func:`super()`. They are used throughout Python itself. Descriptors +simplify the underlying C code and offer a flexible set of new tools for +everyday Python programs. Descriptor Protocol @@ -839,8 +839,8 @@ for whether the caller is an object or a class:: This behavior is useful whenever the function only needs to have a class -reference and does not care about any underlying data. One use for classmethods -is to create alternate class constructors. In Python 2.3, the classmethod +reference and does not care about any underlying data. One use for +classmethods is to create alternate class constructors. The classmethod :func:`dict.fromkeys` creates a new dictionary from a list of keys. The pure Python equivalent is::