Repository navigation
fix(security): refuse internal addresses when fetching file URLs - #43
Merged
Merged
Conversation
Caller-supplied file URLs (image_url and friends) were fetched by the gateway with no address check, and fetch errors echoed the response body and socket error back to the caller. That let any API caller probe loopback ports, the cloud metadata endpoint and cluster services, and read the answers. File URLs now go through a dedicated client (public_url.rs) that: - allows only http/https; - resolves hostnames itself and drops non-public addresses (loopback, RFC 1918, link-local, CGNAT incl. 100.100.100.200, multicast, reserved, IPv6 ULA/link-local, v4-mapped and NAT64 forms), so DNS rebinding and internal hostnames fail at connect time; - re-checks every redirect hop (max 5) and never uses a proxy. Fetch errors now say only that the file could not be downloaded (or the HTTP status); details go to the gateway log. TENSORZERO_ALLOW_PRIVATE_FILE_URLS=1 turns the address checks off for local development.
URL files are downloaded with `public_url::client()`, so the shared `TensorzeroHttpClient` argument was ignored. Remove it from the trait, the impl and the one caller. Also mark the redirect test's local server spawn as an expected `disallowed_methods` exception.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The gateway downloads caller-supplied file URLs (
image_url, file blocks) itself. Before this change it ran no address check and copied the response body and socket error into the error it returned. So an API caller could make the gateway request loopback ports, the cloud metadata endpoint or cluster services, and read the result.File URLs now go through a dedicated client in
tensorzero-core/src/public_url.rs, which:httpandhttps;100.100.100.200), multicast and reserved ranges, IPv6 ULA and link-local, and the v4-mapped and NAT64 forms of all of these. A hostname that resolves only to internal addresses fails at connect time, which also covers DNS rebinding;localhostbefore sending anything, and re-checks every redirect hop (at most 5);Fetch errors now say only that the file could not be downloaded, or give the HTTP status. The details go to the gateway log at
warn.TENSORZERO_ALLOW_PRIVATE_FILE_URLS=1turns the address checks off, for local development against private hosts.Test plan
cargo check -p tensorzero-corepublic_url:localhost;169.254.169.254is not followed.providers::openai::tests::test_file_url_no_mime_type_fetch_and_encode,test_cannot_forward_file_urland theinput_filestests.