Bug report

Bug description:

This hard-coded sanity check for HTTP response headers is no longer sufficient to fetch a Microsoft 365 page.

I do have a case open with Microsoft Support (TrackingID#2503130010002871), but it's not getting much traction as it's not causing problems with full web browsers.

Steps to reproduce:

import http.client
con = http.client.HTTPSConnection('outlook.office365.com')
con.request("GET", "/owa/example.edu")  # any domain seems to trigger
r = con.getresponse()

And that throws a HTTPException

>>> con = http.client.HTTPSConnection('outlook.office365.com')
>>> con.request("GET", "/owa/foo.bar")
>>> r = con.getresponse()
Traceback (most recent call last):
  File "<python-input-8>", line 1, in <module>
    r = con.getresponse()
  File "C:\Users\jmacdone\AppData\Local\Programs\Python\Python313-arm64\Lib\http\client.py", line 1428, in getresponse
    response.begin()
    ~~~~~~~~~~~~~~^^
  File "C:\Users\jmacdone\AppData\Local\Programs\Python\Python313-arm64\Lib\http\client.py", line 350, in begin
    self.headers = self.msg = parse_headers(self.fp)
                              ~~~~~~~~~~~~~^^^^^^^^^
  File "C:\Users\jmacdone\AppData\Local\Programs\Python\Python313-arm64\Lib\http\client.py", line 248, in parse_headers
    headers = _read_headers(fp)
  File "C:\Users\jmacdone\AppData\Local\Programs\Python\Python313-arm64\Lib\http\client.py", line 226, in _read_headers
    raise HTTPException("got more than %d headers" % _MAXHEADERS)
http.client.HTTPException: got more than 100 headers
>>>

It seems to be just spilling over with 101 headers. Though, not consistently. Presumably it depends upon which load balancer node is responding.

$ curl --silent -D - 'https://outlook.office365.com/owa/example.edu' | grep -E "^[a-zA-Z-]+: " | wc -l

returns with 96, 99, 101, etc. headers, depending on Microsoft's mood unknown factors.

For background, it's common to use https://outlook.com/example.edu as a domain hint ("smart link") to go directly to a tenant's identity provider and avoid the "Please provide your email address" step. We have a nagios check for that, which broke recently as the number of Set-Cookie: OpenIdConnect.token.[...] variants continues to grow.

CPython versions tested on:

3.13, 3.9, 3.11

Operating systems tested on:

Linux, Windows

Linked PRs

• gh-131724: Add a new max_response_headers param to HTTP/HTTPSConnection #136814