Skip to content

Body cap: count bytes on the stream instead of rebuilding the response for unencoded bodies #170

Closed

Description

@lesnik512

With max_response_body_bytes set, the client reads the body through _accumulate_capped and then builds a new httpx2.Response from the buffered bytes (src/httpware/_internal/body_cap.py). The rebuild is what loses response attributes such as elapsed, and it adds about 7 µs per capped request.

For a response without Content-Encoding, the decoded bytes are the wire bytes. There we could wrap the public Response.stream in a stream that counts bytes and raises once the count passes the cap, then let httpx2 read it with its normal read(). The response object stays the one httpx2 built, so nothing has to be copied over. Encoded bodies keep the current rebuild path, which matches ADR-0009 (the cap counts decoded bytes).

A prototype measured 19.6 µs before and 12.7 µs after per capped request, with the cap still enforced.

The 0.22.0 body-cap patch (wave 2) carries elapsed, next_request and default_encoding over to the rebuilt response. This issue covers the follow-up that removes the rebuild for unencoded bodies.

How to confirm the gain: benchmark a capped get() against a MockTransport with an unencoded JSON body, before and after.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestneeds-triageMaintainer needs to evaluate this issue

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions