Skip to content

Commit cfe4887

Browse files
authored
Merge pull request #331 from zero-sum-seattle/fix/324-async-proxy-support
fix: honor environment proxies for library-created async clients
2 parents 31b69b7 + ca0ef5f commit cfe4887

6 files changed

Lines changed: 524 additions & 4 deletions

File tree

‎docs/async.md‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -207,6 +207,22 @@ real application the client is typically created once, reused across calls,
207207
and closed by whatever code owns its lifecycle — the examples above show a
208208
few ways to run this, not the required shape of your application.
209209

210+
## Environment proxies
211+
212+
A library-created client (the default — no `client=` passed) honors
213+
`HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`, and `NO_PROXY` from the environment,
214+
the same variables a plain `httpx.AsyncClient()` discovers on its own.
215+
216+
An injected client keeps whatever proxy configuration its caller gave it —
217+
`httpx.AsyncClient()` reads those variables itself by default, or a caller
218+
may pass `trust_env=False` or an explicit `proxy=`/`mounts=` to opt out or
219+
override. The library does not add or remove proxy configuration on an
220+
injected client.
221+
222+
See [HTTP transport: async client environment
223+
proxies](http-transport.md#async-client-environment-proxies) for the full
224+
behavior.
225+
210226
## Documentation boundaries
211227

212228
- [README](../README.md) — installation and quick-start examples

‎docs/http-transport.md‎

Lines changed: 26 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -710,8 +710,31 @@ bodies.
710710

711711
The client has no default response cache.
712712

713-
## No async support
713+
## Async client environment proxies
714714

715-
The client remains synchronous.
715+
Library-created `AsyncMlb` / `AsyncMlbDataAdapter` clients honor
716+
`HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`, and `NO_PROXY` (any case), the same
717+
environment variables HTTPX itself discovers for a plain `httpx.AsyncClient()`.
716718

717-
Async support is not part of version 1.0.0.
719+
```text
720+
Library-created async client
721+
Reads HTTP_PROXY / HTTPS_PROXY / ALL_PROXY / NO_PROXY from the environment
722+
Routes matching requests through the proxy
723+
Applies the library retry policy to proxied and direct requests alike
724+
725+
Caller-injected async client
726+
Keeps exactly whatever transport and mounts its caller configured
727+
The library never reads proxy environment variables for it
728+
```
729+
730+
This mirrors [Session ownership](#session-ownership) on the sync side: the
731+
library only ever configures a client it created itself. See
732+
[async.md](async.md#custom-httpx-client) for injecting a client, including one
733+
configured with its own proxy settings.
734+
735+
## Scope of this document
736+
737+
The retry, timeout, User-Agent, and strict-HTTP behavior documented above
738+
apply to the synchronous `Mlb` client. For the asynchronous client, see
739+
[async.md](async.md); it shares this document's retry, timeout, and
740+
error-handling contract except where noted above.

‎mlbstatsapi/_async_transport.py‎

Lines changed: 45 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@
2525
import asyncio
2626

2727
from ._async_support import import_httpx
28+
from ._env_proxies import environment_proxy_map
2829
from .mlb_dataadapter import _build_user_agent, create_retry_policy
2930

3031
httpx = import_httpx()
@@ -158,8 +159,51 @@ def create_library_async_client() -> httpx.AsyncClient:
158159
library defaults are applied here, at creation, and only to clients the
159160
library creates. Passing headers to the constructor replaces just the
160161
User-Agent, so HTTPX's other default headers survive.
162+
163+
HTTPX only builds its own environment-proxy mounts when the caller leaves
164+
``transport=None`` (``allow_env_proxies = trust_env and transport is
165+
None`` in ``httpx.Client.__init__``). Passing ``transport=`` here, which
166+
is required to install the retry transport, would otherwise silently
167+
disable ``HTTP_PROXY`` / ``HTTPS_PROXY`` / ``ALL_PROXY`` / ``NO_PROXY``
168+
support for every library-created async client (issue #324). This
169+
rebuilds that discovery from the stdlib (see ``_env_proxies.py``) and
170+
passes it through HTTPX's public ``mounts=`` argument instead, wrapping
171+
every proxy transport in the same retry transport the direct path uses,
172+
so a request routed through a proxy still gets library retries.
173+
174+
Environment discovery always runs here, matching the ``trust_env=True``
175+
default a caller gets from a plain ``httpx.AsyncClient()``. Neither
176+
``AsyncMlb`` nor ``AsyncMlbDataAdapter`` exposes a ``trust_env`` toggle;
177+
a caller who needs one injects their own client instead, the same way
178+
they would opt into any other HTTPX-level setting this factory does not
179+
surface.
180+
181+
One retry policy instance is shared by the direct transport and every
182+
proxy transport, mirroring the sync side sharing one Session across the
183+
v1 and v1.1 adapters: retries are a property of the client, not of any
184+
one transport within it.
161185
"""
186+
retry_policy = create_retry_policy()
187+
direct = MlbAsyncRetryTransport(
188+
httpx.AsyncHTTPTransport(), retry_policy=retry_policy
189+
)
190+
191+
mounts: dict[str, httpx.AsyncBaseTransport | None] = {}
192+
for pattern, proxy in environment_proxy_map().items():
193+
if proxy is None:
194+
# None tells HTTPX to fall back to client._transport for this
195+
# pattern (see AsyncClient._transport_for_url), i.e. bypass the
196+
# proxy rather than route through a second transport instance.
197+
# aclose() also skips a None mount, so this never gets closed
198+
# twice via both the direct transport and a mount entry.
199+
mounts[pattern] = None
200+
else:
201+
mounts[pattern] = MlbAsyncRetryTransport(
202+
httpx.AsyncHTTPTransport(proxy=proxy), retry_policy=retry_policy
203+
)
204+
162205
return httpx.AsyncClient(
163206
headers={"User-Agent": _build_user_agent()},
164-
transport=MlbAsyncRetryTransport(),
207+
transport=direct,
208+
mounts=mounts,
165209
)

‎mlbstatsapi/_env_proxies.py‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
"""Build an HTTPX-compatible proxy mount map from the environment.
2+
3+
HTTPX only discovers ``HTTP_PROXY`` / ``HTTPS_PROXY`` / ``ALL_PROXY`` /
4+
``NO_PROXY`` for itself when it builds its own transport, which happens only
5+
when the caller does not pass ``transport=`` (see ``allow_env_proxies =
6+
trust_env and transport is None`` in ``httpx.Client.__init__``). The async
7+
retry transport (``_async_transport.py``) always passes ``transport=``, so
8+
that discovery never runs, and environment proxy support silently disappears
9+
for library-created async clients (issue #324).
10+
11+
This module reimplements that discovery from the stdlib and hands the result
12+
to HTTPX's public ``mounts=`` argument instead, so the library stays off
13+
HTTPX's private ``httpx._utils.get_environment_proxies``. The parsing here
14+
intentionally mirrors that private function's semantics, verified against
15+
installed httpx 0.28.1. The differential test in
16+
``tests/test_async_env_proxies.py`` (``test_matches_stock_httpx_env_proxy_resolution``)
17+
is the drift alarm: it resolves the same URLs against a stock
18+
``httpx.AsyncClient()`` and against this module's output on every run, so a
19+
future httpx release changing ``NO_PROXY`` or proxy semantics fails that test
20+
instead of silently diverging.
21+
22+
No httpx import here: environment variables in, a plain ``dict`` out.
23+
"""
24+
25+
from __future__ import annotations
26+
27+
import ipaddress
28+
from urllib.request import getproxies
29+
30+
31+
def environment_proxy_map(*, trust_env: bool = True) -> dict[str, str | None]:
32+
"""Return an HTTPX ``mounts=``-shaped map of proxies from the environment.
33+
34+
Keys are URL patterns such as ``"https://"`` or ``"all://*mlb.com"``; a
35+
``None`` value means "bypass the proxy for this pattern" and is meaningful
36+
only when a broader pattern (from ``ALL_PROXY``) would otherwise match.
37+
"""
38+
if not trust_env:
39+
return {}
40+
41+
proxy_info = getproxies()
42+
mounts: dict[str, str | None] = {}
43+
44+
for scheme in ("http", "https", "all"):
45+
value = proxy_info.get(scheme)
46+
if value:
47+
mounts[f"{scheme}://"] = value if "://" in value else f"http://{value}"
48+
49+
no_proxy_hosts = [host.strip() for host in proxy_info.get("no", "").split(",")]
50+
for hostname in no_proxy_hosts:
51+
if hostname == "*":
52+
return {}
53+
elif hostname:
54+
if "://" in hostname:
55+
mounts[hostname] = None
56+
elif _is_ipv4(hostname):
57+
mounts[f"all://{hostname}"] = None
58+
elif _is_ipv6(hostname):
59+
mounts[f"all://[{hostname}]"] = None
60+
elif hostname.lower() == "localhost":
61+
mounts[f"all://{hostname}"] = None
62+
else:
63+
mounts[f"all://*{hostname}"] = None
64+
65+
return mounts
66+
67+
68+
def _is_ipv4(hostname: str) -> bool:
69+
try:
70+
ipaddress.IPv4Address(hostname.split("/")[0])
71+
except ValueError:
72+
return False
73+
return True
74+
75+
76+
def _is_ipv6(hostname: str) -> bool:
77+
try:
78+
ipaddress.IPv6Address(hostname.split("/")[0])
79+
except ValueError:
80+
return False
81+
return True

0 commit comments

Comments
 (0)