added stuff as suggested by Mark

python · iritkatriel · Jan 16, 2025 · Jan 5, 2025 · Jan 5, 2025 · Jan 15, 2025
commit 628a3a4059dec430b0c3a36cc9a942409089e01b
diff --git a/InternalDocs/generators.md b/InternalDocs/generators.md
@@ -5,12 +5,13 @@ Generators and Coroutines
 Generators
 ----------

-The implementation of generators in CPython consists of the builtin object type
-`PyGenObject` and bytecode instructions that operate on instances of this type.
+The implementation of generators in CPython consists of instances of `PyGenObject`
+and bytecode instructions that operate on instances of this type.

-A generator object executes in its own [`frame`](frames.md), like a function.
-The difference is that a function returns only once, while a generator
-"returns" to the caller every time it emits a new item with a
+A generator object is invoked in a [`frame`](frames.md), like a function.
+The difference is that a function returns to the calling frame only once,
+while a generator "returns" to the caller every time it emits a new item
+with a
 [`yield` expression](https://docs.python.org/dev/reference/expressions.html#yield-expressions).
 This is implemented by the
 [`YIELD_VALUE`](https://docs.python.org/dev/library/dis.html#opcode-YIELD_VALUE)
@@ -23,22 +24,65 @@ instruction pointer and stores the interpreter's exception state on the generato
 object. When the generator is resumed, this exception state is copied back to the
 interpreter state.

-The `frame` of a generator is embedded in the generator object struct (see
-`_PyGenObject_HEAD` in [`pycore_genobject.h`](../Include/internal/pycore_genobject.h)).
+The `frame` of a generator is embedded in the generator object struct as a
+[`_PyInterpreterFrame`](frames.md) (see `_PyGenObject_HEAD` in
+[`pycore_genobject.h`](../Include/internal/pycore_genobject.h)).
 This means that we can get the frame from the generator or the generator
 from the frame (see `_PyGen_GetGeneratorFromFrame` in the same file).
 Other fields of the generator struct include metadata (such as the name of
 the generator function) and runtime state information (such as whether its
 frame is executing, suspended, cleared, etc.).

+Generator Object Creation and Destruction
+-----------------------------------------
+
+The bytecode of a generator function begins with a
+[`RETURN_GENERATOR`](https://docs.python.org/dev/library/dis.html#opcode-RETURN_GENERATOR)
+instruction, which creates a generator object, along with its embedded frame.
+The generator's frame is initialized as a copy of the frame in which
+`RETURN_GENERATOR` is executing, but its `owner` field is overwritten to indicate
+that it is owned by a generator. Finally, `RETURN_GENERATOR` pushes the new generator
+object to the stack and returns to the caller of the generator function. When the
+generator is next resumed by [`gen_send_ex2()`](../Objects/genobject.c),
+`_PyEval_EvalFrame()` is called to continue executing the generator function,
+in the frame that is embedded in the generator object.
+
+When a generator object is destructed in [`gen_dealloc`](../Objects/genobject.c),
+its embedded `_PyInterpreterFrame` field may need to be preserved, if it is exposed
+to Python as part of a [`PyFrameObject`](frames.md#frame-objects). This is detected
+in [`_PyFrame_ClearExceptCode`](../Python/frame.c) by the fact that the interpreter
+frame's `frame_obj` field is set, and the frame object it points to has refcount
+greater than 1. If so, the `take_ownership()` function is called to create a new
+copy of the interpreter frame and transfer ownership of it from the generator to
+the frame object.
+
+Iteration
+---------
+
+The [`FOR_ITER`](https://docs.python.org/dev/library/dis.html#opcode-FOR_ITER)
+instruction calls `__next__` on the iterator which is on the top of the stack,
+and pushes the result to the stack. It has [`specializations`](adaptive.md)
+for a few common iterator types, including `FOR_ITER_GEN`, for iterating over
+a generator.
+
 Chained Generators
 ------------------

 A `yield from` expression creates a generator that efficiently yields the
 sequence created by another generator. This is implemented with the
 [`SEND` instruction](https://docs.python.org/dev/library/dis.html#opcode-SEND),
 which pushes the value of its arg to the stack of the generator's frame, sets
-the exception state on this frame, and resumes execution of the chained generator. 
+the exception state on this frame, and resumes execution of the chained generator.
+On return from `SEND`, the value at the top of the stack is sent back up
+the generator chain with a `YIELD_VALUE`. This sequence of `SEND` followed by
+`YIELD_VALUE` is repeated in a loop, until a `StopIteration` exception is
+raised to indicate that the generator has no more values to emit.
+
+The [`CLEANUP_THROW`](https://docs.python.org/dev/library/dis.html#opcode-CLEANUP_THROW)
+instruction is used to handle exceptions raised from the send-yield loop.
+Exceptions of type `StopIteration` is handled, their `value` field hold the
+value to be returned by the generator's `close()` function. Any other
+exception is re-raised by `CLEANUP_THROW`.

 Coroutines
 ----------