(From #2545)

We need to determine how users will interact with our iterators and what level of abstraction is expected.

Right now @dhermes has implemented an iterator that iterates through items but exposes page information by setting state on the iterator itself.

iterator = MyIterator(...)

for item in iterator:
     print(item)
     print(iterator.page)
     print(iterator.next_page_token)

I'm not the biggest fan of this for two reasons: it combines the iterable and the iterator interfaces, it merges the item-level and page-level abstractions.

In my opinion, we should work at well-defined abstraction levels and keep the iterable and iterator separate.

The first level is the page level:

iterator = MyIterator(...)

# You could also specify arguments to pages, such as page_token
for page in iterator.pages():
    print(page.number)
    print(page.items)
    print(page.next_page_token)

The second level yields items but also page context:

for item, current_page in iterator.items_by_page():
     print(item)
     print(page.number)
     print(page.next_page_token)

The final level yields only items and completely abstracts away the concept of pages:

for item in iterator.items():
    print(item)