I did this

(imported issued from https://marc.info/?l=git&m=171175679831214&w=2, authored by Jeff King (@peff))

I noticed some http-related failures in the test suite on my Debian
unstable system, which recently got an upgraded curl package. It looks
like it's related to cases where we use the remote-curl helper in
"connect" mode (i.e., protocol v2) and the http buffer is small
(requiring us to stream the data to curl). Besides just running t5551,
an easy reproduction is:

  [this works]
  $ git ls-remote https://git.kernel.org/pub/scm/git/git.git | wc -l
  1867

  [this doesn't]
  $ git -c http.postbuffer=65536 ls-remote https://git.kernel.org/pub/scm/git/git.git
  fatal: expected flush after ref listing

The error message comes from ls-remote itself, which was expecting a
FLUSH packet from the remote. Instead it gets the RESPONSE_END from
remote-curl (remember that in connect mode, remote-curl is just ferrying
bytes back and forth between ls-remote and the server).

It works with older versions of libcurl, but not 8.7.0 (or 8.7.1).
Bisecting in libcurl points to 9369c30 (lib: Curl_read/Curl_write
clarifications, 2024-02-15).

Running with GIT_TRACE_CURL=1 shows weirdness on the POST we send to
issue the ls-refs command. With older curl, I see this:

  => Send header: POST /pub/scm/git/git.git/git-upload-pack HTTP/1.1
  => Send header: Host: git.kernel.org
  => Send header: User-Agent: git/2.44.0.789.g252ee96bc5.dirty
  => Send header: Accept-Encoding: deflate, gzip
  => Send header: Content-Type: application/x-git-upload-pack-request
  => Send header: Accept: application/x-git-upload-pack-result
  => Send header: Git-Protocol: version=2
  => Send header: Transfer-Encoding: chunked
  => Send header:
  => Send data: 14..0014command=ls-refs...
  => Send data: 2a..002aagent=git/2.44.0.789.g252ee96bc5.dirty..
  [and so on until...]
  == Info: Signaling end of chunked upload via terminating chunk.

But with the broken version, I get:

  => Send header: POST /pub/scm/git/git.git/git-upload-pack HTTP/1.1
  => Send header: Host: git.kernel.org
  => Send header: User-Agent: git/2.44.0.789.g252ee96bc5.dirty
  => Send header: Accept-Encoding: deflate, gzip, br, zstd
  => Send header: Content-Type: application/x-git-upload-pack-request
  => Send header: Accept: application/x-git-upload-pack-result
  => Send header: Git-Protocol: version=2
  => Send header: Transfer-Encoding: chunked
  => Send header:
  => Send data, 0000000014 bytes (0x0000000e)
  => Send data: 4..0014..0....
  == Info: upload completely sent off: 14 bytes

So we only get the first 4 bytes, and then we quit (the double mention
of 14 is confusing, but I think it is both the size of the pkt-line
("command=ls-refs\n") but also the length of the 4-byte string when
framed with chunked transfer-encoding). Those 4 bytes are the first
thing returned by rpc_out(), which we use as our CURLOPT_READFUNCTION.

It's possible that we're doing something wrong with our read/write
function callbacks. But I don't see how; we say "here's 4 bytes", but
then we never get called again. It's like curl is giving up on trying to
read the post input early for some reason.

I'm not sure how to dig further. That commit is pretty big and scary. I
did check that the tip of master in curl.git is still affected (I'd
hoped maybe the 0-length write fixes in b30d694 would be related,
but that's not it).

Ideas?

I expected the following

No response

curl/libcurl version

8.7.0 and 8.7.1

operating system

Linux, but probably not relevant