Query params should support list-of-strings values.

Hi,

I recently tried to use httpx instead of requests in my simple app that calls few API endpoints and suddenly it did not work.

The problem here is that httpx and requests process query parameters differently.

Here's a simple script to see how both libraries process params:

from httpx.models import QueryParams
from requests.models import RequestEncodingMixin

dict_params = {
    "documentIds": ["a55b2f894b90dd1213201471_25", "b54b2f894b90dd1213201471_9"]
}

httpx_params = str(QueryParams(dict_params))
print("httpx", httpx_params)

requests_params = RequestEncodingMixin._encode_params(dict_params)
print("requests", requests_params)

Results:

httpx documentIds=%5B%27a55b2f894b90dd1213201471_25%27%2C+%27b54b2f894b90dd1213201471_9%27%5D
requests documentIds=a55b2f894b90dd1213201471_25&documentIds=b54b2f894b90dd1213201471_9

If we try to parse the query string httpx produced, here's what we get:

from urllib.parse import parse_qs

httpx_params_parsed = parse_qs('documentIds=%5B%27a55b2f894b90dd1213201471_25%27%2C+%27b54b2f894b90dd1213201471_9%27%5D')
print(httpx_params_parsed )

Result:

{'documentIds': ["['a55b2f894b90dd1213201471_25', 'b54b2f894b90dd1213201471_9']"]}

Which is not what you would expect, I guess. At least, it works differently than in requests.

If it is an intended behavior, can you please recommend me the proper way of passing a list into the query params?

Thanks!

Here's a bit more "live" example:

import httpx
import requests

URL = "https://httpbin.org/get"
PARAMS = {"documentIds": ["a55b2f894b90dd1213201471_25", "b54b2f894b90dd1213201471_9"]}


r = requests.get(URL, params=PARAMS)
print("requests", r.json()["args"])

r = httpx.get(URL, params=PARAMS)
print("httpx", r.json()["args"])

Results:

requests {'documentIds': ['a55b2f894b90dd1213201471_25', 'b54b2f894b90dd1213201471_9']}
httpx {'documentIds': "['a55b2f894b90dd1213201471_25', 'b54b2f894b90dd1213201471_9']"}

@senpos

Hi @senpos! Thank you for reporting this.

I can confirm this is a bug: multivalued query params are encoded in the form of a Python list. I discovered it in #298 but didn't think about open an issue:

httpx/tests/compat/test_requests.py

Lines 1006 to 1019 in 39348dd

    
               @pytest.mark.httpx_bug 
        
               @pytest.mark.xfail( 
        
                   reason="list repr gets URL-encoded: '/get?test=%5B%27foo%27%2C+%27baz%27%5D'" 
        
               ) 
        
               def test_urlencoded_get_query_multivalued_param(self, httpbin): 
        
                   r = requests.get(httpbin("get"), params={"test": ["foo", "baz"]}) 
        
                   assert r.status_code == 200 
        
                   assert r.url == httpbin("get?test=foo&test=baz") 
        
               def test_form_encoded_post_query_multivalued_element(self, httpbin): 
        
                   r = requests.Request( 
        
                       method="POST", url=httpbin("post"), data=dict(test=["foo", "baz"]) 
        
                   ) 
        
                   assert r.content == b"test=foo&test=baz"

We should look at adding some requests compatibility tests for this if they exist. :)

Thanks for opening the issue!
Retitled to make this more clearly specific.

Navigation Menu

Repository navigation

Query params should support list-of-strings values. #366

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	@pytest.mark.httpx_bug
	@pytest.mark.xfail(
	reason="list repr gets URL-encoded: '/get?test=%5B%27foo%27%2C+%27baz%27%5D'"
	)
	def test_urlencoded_get_query_multivalued_param(self, httpbin):
	r = requests.get(httpbin("get"), params={"test": ["foo", "baz"]})
	assert r.status_code == 200
	assert r.url == httpbin("get?test=foo&test=baz")

	def test_form_encoded_post_query_multivalued_element(self, httpbin):
	r = requests.Request(
	method="POST", url=httpbin("post"), data=dict(test=["foo", "baz"])
	)
	assert r.content == b"test=foo&test=baz"

Query params should support list-of-strings values. #366

Description

Activity

senpos commented on Sep 21, 2019

florimondmanca commented on Sep 21, 2019

sethmlarson commented on Sep 23, 2019

lovelydinosaur commented on Sep 25, 2019

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions