Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings
Discussion options

If response has application/json content type and empty body wreq attempts to parse it to json anyway.
So this:

r = await Client().get("...")  # empty body
await r.json()

Becomes this:
exceptions.DecodingError: is_decode error: wreq::Error { kind: Decode, source: Error("EOF while parsing a value", line: 1, column: 0) }

Is it intentional?

You must be logged in to vote

It seems that only aiohttp likes to return None when parsing an empty body results in an error.

Replies: 1 comment · 4 replies

Comment options

This is the error thrown by the upstream library, and it should behave in a consistent manner with reqwest.

https://github.com/seanmonstar/reqwest/blob/1f72916f5cdc30f6cb6c63038c89063795294d50/src/async_impl/response.rs#L270
https://github.com/0x676e67/wreq/blob/9b0fbd925bd64d6901f31018aa4bd4c9143dd13a/src/client/response.rs#L277

Give me an example that can be replicated. Maybe there is room for optimization in that case.

You must be logged in to vote
4 replies
@0x676e67
Comment options

In Python, the response body is actually automatically cached. This means that for reading operations other than the streaming body, you can repeat the reading process. If JSON fails, you can still try bytes or text.

Note: The upstream library doesn't verify the application/json header; it expects the user to assume it's there.

@0x676e67
Comment options

import asyncio
import sys
import threading
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path

EXAMPLES_DIR = Path(__file__).resolve().parent
sys.path = [entry for entry in sys.path if Path(entry or ".").resolve() != EXAMPLES_DIR]

import aiohttp
import httpx
import requests
import urllib3
import wreq


class Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        routes = {
            "/204": (204, b"", None),
            "/empty": (200, b"", "application/json"),
            "/space": (200, b"   \n\t", "application/json"),
            "/null": (200, b"null", "application/json"),
            "/json": (200, b'{"ok": true}', "application/json"),
        }

        status, body, content_type = routes.get(self.path, (404, b"not found", "text/plain"))
        self.send_response(status)
        if content_type is not None:
            self.send_header("Content-Type", content_type)
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        if body:
            self.wfile.write(body)

    def log_message(self, format, *args):
        return


def format_result(result):
    if isinstance(result, Exception):
        return f"{type(result).__name__}: {result}"
    return repr(result)


def run_requests(url):
    try:
        return requests.get(url, timeout=5).json()
    except Exception as exc:
        return exc


def run_httpx(url):
    try:
        return httpx.get(url, timeout=5).json()
    except Exception as exc:
        return exc


def run_urllib3(http, url):
    try:
        return http.request("GET", url, timeout=5.0).json()
    except Exception as exc:
        return exc


async def run_aiohttp(url):
    try:
        async with aiohttp.ClientSession() as session:
            async with session.get(url) as resp:
                return await resp.json()
    except Exception as exc:
        return exc


async def run_wreq(url):
    try:
        resp = await wreq.get(url)
        async with resp:
            return await resp.json()
    except Exception as exc:
        return exc


async def main():
    server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)
    thread = threading.Thread(target=server.serve_forever, daemon=True)
    thread.start()

    base_url = f"http://127.0.0.1:{server.server_port}"
    paths = ["/204", "/empty", "/space", "/null", "/json"]
    http = urllib3.PoolManager()

    try:
        for path in paths:
            url = f"{base_url}{path}"
            print(f"\n=== {path} ===")
            print("requests:", format_result(run_requests(url)))
            print("httpx:", format_result(run_httpx(url)))
            print("urllib3:", format_result(run_urllib3(http, url)))
            print("aiohttp:", format_result(await run_aiohttp(url)))
            print("wreq:", format_result(await run_wreq(url)))
    finally:
        http.clear()
        server.shutdown()
        server.server_close()
        thread.join(timeout=1)


if __name__ == "__main__":
    asyncio.run(main())
# Results

=== /204 ===
requests: JSONDecodeError: Expecting value: line 1 column 1 (char 0)
httpx: JSONDecodeError: Expecting value: line 1 column 1 (char 0)
urllib3: JSONDecodeError: Expecting value: line 1 column 1 (char 0)
aiohttp: ContentTypeError: 204, message='Attempt to decode JSON with unexpected mimetype: ', url='http://127.0.0.1:63698/204'
wreq: DecodingError: is_decode error: wreq::Error { kind: Decode, source: Error("EOF while parsing a value", line: 1, column: 0) }

=== /empty ===
requests: JSONDecodeError: Expecting value: line 1 column 1 (char 0)
httpx: JSONDecodeError: Expecting value: line 1 column 1 (char 0)
urllib3: JSONDecodeError: Expecting value: line 1 column 1 (char 0)
aiohttp: None
wreq: DecodingError: is_decode error: wreq::Error { kind: Decode, source: Error("EOF while parsing a value", line: 1, column: 0) }

=== /space ===
requests: JSONDecodeError: Expecting value: line 2 column 2 (char 5)
httpx: JSONDecodeError: Expecting value: line 2 column 2 (char 5)
urllib3: JSONDecodeError: Expecting value: line 2 column 2 (char 5)
aiohttp: None
wreq: DecodingError: is_decode error: wreq::Error { kind: Decode, source: Error("EOF while parsing a value", line: 2, column: 1) }

=== /null ===
requests: None
httpx: None
urllib3: None
aiohttp: None
wreq: None

=== /json ===
requests: {'ok': True}
httpx: {'ok': True}
urllib3: {'ok': True}
aiohttp: {'ok': True}
wreq: {'ok': True}
@0x676e67
Comment options

It seems that only aiohttp likes to return None when parsing an empty body results in an error.

Answer selected by somespecialone
@somespecialone
Comment options

Wow, thanks for comprehensive answer.

In Python, the response body is actually automatically cached.

And I already use it.
As there are any convention regardless http client verifying empty body before passing to a decoder, I guess it is just right to stay with current behavior.

Maybe there is room for optimization in that case.

I don't think there is any, it is up to user to verify if response is malformed (json content type and empty body).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
🙏
Q&A
Labels
None yet
2 participants
Morty Proxy This is a proxified and sanitized view of the page, visit original site.