A self-hoster who sets AGENTA_API_URL=http://127.0.0.1:<port>/api gets that address handed
unchanged to a run that does not execute on their host. 127.0.0.1 there means the sandbox or
the container itself, never the operator's machine, so trace ingest cannot be reached and the
run's traces never arrive.
http://localhost:<port>/api works, because both copies of parse_url rewrite localhost to
host.docker.internal in bridge mode. 127.0.0.1 names the same interface and is an equally
natural thing to write, and neither copy rewrites it.
Found during the v0.114.4 release gate, while verifying the fix for a related localhost defect
(#6407). Not release-gating.
Verification, in about ten seconds
Run this against sdks/python/agenta/sdk/utils/helpers.py, in bridge mode:
parse_url("http://127.0.0.1:8080/api")
Expected, by analogy with the localhost case: http://host.docker.internal:8080/api.
Observed, on release/v0.114.4: http://127.0.0.1:8080/api, unchanged. It is unchanged in
bridge mode, in host mode, and with DOCKER_NETWORK_MODE unset.
The API's copy at api/oss/src/utils/helpers.py returns the same value, for a slightly different
reason. See "Two copies" below.
End-to-end reproduction
- Self-host with docker-compose, in the default bridge network mode.
- Set
AGENTA_API_URL=http://127.0.0.1:<port>/api, where <port> is the port the Agenta API
is published on. Bring the stack up.
- Confirm the address works from the host shell:
curl http://127.0.0.1:<port>/api/health
returns 200. This is why an operator picks it.
- Create an agent workflow and send one turn on a sandboxed run.
- Look for that turn's traces in the platform.
Expected: the traces arrive, exactly as they do when AGENTA_API_URL uses localhost.
Observed: the traces never arrive. The OTLP export targets
http://127.0.0.1:<port>/api/otlp/v1/traces, and in the process that performs the export
127.0.0.1 is that process's own loopback, so the connection fails.
Honest limit on this evidence. The rewrite gap is verified directly, by calling both copies
of parse_url. The end-to-end symptom was reasoned from the code path during the #6407 triage
and is not backed by a captured live error string. Whoever fixes this should capture the failure
once before the fix and once after, so the test asserts on the real shape.
Note that this is NOT the same failure as #6407. There, the mismatch between the configured base
and the rewritten endpoint made the runner drop the run's platform credential. Here both sides
carry the same raw 127.0.0.1, so the allowlist matches and the credential survives. The address
is simply not reachable from where the export happens.
Two copies of parse_url, and they have drifted
There are two functions named parse_url, and they are not the same function.
|
Path |
Rewrite test |
Scheme defaulting |
Hosts it rewrites |
| SDK |
sdks/python/agenta/sdk/utils/helpers.py:14 |
substring, "localhost" not in url and "0.0.0.0" not in url (:27) |
none |
localhost, 0.0.0.0 |
| API |
api/oss/src/utils/helpers.py:75 |
parsed hostname, membership in a set (:98-102) |
yes, :87-88 |
localhost, 0.0.0.0 |
The SDK's copy is the operative one for this bug. The endpoint on the wire is
ag.tracing.otlp_url, which the agent service derives through the SDK's copy
(sdks/python/agenta/sdk/utils/init.py:97-102, then :146). The API's copy shapes the service
URL a run is POSTed to (api/oss/src/core/workflows/service.py:685). The runner's own comment
already records which is which, at services/runner/src/tracing/otel.ts:375-379.
The drift between them is part of the problem. They are named the same, they mean the same thing
to a reader, and they answer differently. That drift is also the subject of a separate issue
about scheme-less input, which the API copy handles and the SDK copy does not.
Suggested implementation path
Three files. The third is the one that is easy to miss and expensive to miss.
1. The SDK copy: sdks/python/agenta/sdk/utils/helpers.py:14-47
Add 127.0.0.1 to the guard at :27 and to the replacement chain at :33-39.
2. The API copy: api/oss/src/utils/helpers.py:75-125
Add "127.0.0.1" to the is_local_runtime_host set at :98-102, and add a
.replace("127.0.0.1", "host.docker.internal") to both replacement chains, at :108-114 and at
:119-125.
Adding it to the set alone is a silent no-op. The set only decides whether the URL is a candidate
for rewriting; the chains do the rewriting, and they name localhost and 0.0.0.0 explicitly.
A change that touches only the set makes the function take the rewrite branch and then return
the input unchanged, which looks like a fix and is not one.
3. The runner allowlist: services/runner/src/tracing/otel.ts:364
BRIDGE_REWRITTEN_HOSTS is the runner's model of which hosts the platform rewrites. It holds
localhost and 0.0.0.0 today. Add 127.0.0.1 in the same change.
If this file is missed, the fix trades one bug for a worse one. The runner compares the
endpoint it is handed against configuredIngestBases(). Once the platform starts rewriting
127.0.0.1 to host.docker.internal, the raw configured base and the dispatched endpoint stop
matching, isAgentaIngest returns false, and the runner drops the run's platform credential.
That is exactly the failure #6407 fixed for localhost, reintroduced for 127.0.0.1.
The doc comment at services/runner/src/tracing/otel.ts:384-385 states the current premise in
writing: "127.0.0.1 earns no alias. Neither copy of parse_url rewrites it, so that deployment
already matches its own raw base and the bridge form is a pair the platform cannot produce."
This change makes that premise false, so update the comment along with the set.
4. Tests, one per copy
- SDK:
sdks/python/oss/tests/pytest/unit/test_helpers.py is a new file. There is no SDK test
for parse_url today, which is part of how the two copies drifted. Mirror the API's existing
cases as well as adding the new one, so the next drift is caught.
- API:
api/oss/tests/pytest/unit/test_helpers.py already covers the localhost cases. Add the
127.0.0.1 case beside them.
- Runner: extend the existing
configuredIngestBases tests so the 127.0.0.1 base emits its
bridge alias.
On ::1
Consider it, but do not add it as a string replacement. In a URL an IPv6 literal is bracketed, so
the host appears as [::1] and a naive .replace("::1", ...) corrupts the URL and can also
match inside a port or a path. If ::1 is worth supporting, do it in the API copy, which already
parses the hostname properly, and either rewrite the SDK copy to parse rather than substring-match
or leave ::1 out of the SDK and say so in a comment. Leaving it out entirely is a reasonable
answer for this issue.
On the "good first issue" label
Deliberately NOT labelled good first issue, despite each individual edit being a few lines.
The change spans two languages and three files, and it carries a cross-file invariant that is
invisible from any one of them: the set of hosts the platform rewrites must equal the set the
runner expects it to rewrite. A contributor who fixes the two parse_url copies and stops has
made the deployment worse, and nothing in those two files says so. Both traps are written out
above, so this is approachable, but it is not a first-day task.
A self-hoster who sets
AGENTA_API_URL=http://127.0.0.1:<port>/apigets that address handedunchanged to a run that does not execute on their host.
127.0.0.1there means the sandbox orthe container itself, never the operator's machine, so trace ingest cannot be reached and the
run's traces never arrive.
http://localhost:<port>/apiworks, because both copies ofparse_urlrewritelocalhosttohost.docker.internalin bridge mode.127.0.0.1names the same interface and is an equallynatural thing to write, and neither copy rewrites it.
Found during the v0.114.4 release gate, while verifying the fix for a related localhost defect
(#6407). Not release-gating.
Verification, in about ten seconds
Run this against
sdks/python/agenta/sdk/utils/helpers.py, in bridge mode:Expected, by analogy with the localhost case:
http://host.docker.internal:8080/api.Observed, on
release/v0.114.4:http://127.0.0.1:8080/api, unchanged. It is unchanged inbridge mode, in host mode, and with
DOCKER_NETWORK_MODEunset.The API's copy at
api/oss/src/utils/helpers.pyreturns the same value, for a slightly differentreason. See "Two copies" below.
End-to-end reproduction
AGENTA_API_URL=http://127.0.0.1:<port>/api, where<port>is the port the Agenta APIis published on. Bring the stack up.
curl http://127.0.0.1:<port>/api/healthreturns 200. This is why an operator picks it.
Expected: the traces arrive, exactly as they do when
AGENTA_API_URLuseslocalhost.Observed: the traces never arrive. The OTLP export targets
http://127.0.0.1:<port>/api/otlp/v1/traces, and in the process that performs the export127.0.0.1is that process's own loopback, so the connection fails.Honest limit on this evidence. The rewrite gap is verified directly, by calling both copies
of
parse_url. The end-to-end symptom was reasoned from the code path during the #6407 triageand is not backed by a captured live error string. Whoever fixes this should capture the failure
once before the fix and once after, so the test asserts on the real shape.
Note that this is NOT the same failure as #6407. There, the mismatch between the configured base
and the rewritten endpoint made the runner drop the run's platform credential. Here both sides
carry the same raw
127.0.0.1, so the allowlist matches and the credential survives. The addressis simply not reachable from where the export happens.
Two copies of parse_url, and they have drifted
There are two functions named
parse_url, and they are not the same function.sdks/python/agenta/sdk/utils/helpers.py:14"localhost" not in url and "0.0.0.0" not in url(:27)localhost,0.0.0.0api/oss/src/utils/helpers.py:75:98-102):87-88localhost,0.0.0.0The SDK's copy is the operative one for this bug. The endpoint on the wire is
ag.tracing.otlp_url, which the agent service derives through the SDK's copy(
sdks/python/agenta/sdk/utils/init.py:97-102, then:146). The API's copy shapes the serviceURL a run is POSTed to (
api/oss/src/core/workflows/service.py:685). The runner's own commentalready records which is which, at
services/runner/src/tracing/otel.ts:375-379.The drift between them is part of the problem. They are named the same, they mean the same thing
to a reader, and they answer differently. That drift is also the subject of a separate issue
about scheme-less input, which the API copy handles and the SDK copy does not.
Suggested implementation path
Three files. The third is the one that is easy to miss and expensive to miss.
1. The SDK copy:
sdks/python/agenta/sdk/utils/helpers.py:14-47Add
127.0.0.1to the guard at:27and to the replacement chain at:33-39.2. The API copy:
api/oss/src/utils/helpers.py:75-125Add
"127.0.0.1"to theis_local_runtime_hostset at:98-102, and add a.replace("127.0.0.1", "host.docker.internal")to both replacement chains, at:108-114and at:119-125.Adding it to the set alone is a silent no-op. The set only decides whether the URL is a candidate
for rewriting; the chains do the rewriting, and they name
localhostand0.0.0.0explicitly.A change that touches only the set makes the function take the rewrite branch and then return
the input unchanged, which looks like a fix and is not one.
3. The runner allowlist:
services/runner/src/tracing/otel.ts:364BRIDGE_REWRITTEN_HOSTSis the runner's model of which hosts the platform rewrites. It holdslocalhostand0.0.0.0today. Add127.0.0.1in the same change.If this file is missed, the fix trades one bug for a worse one. The runner compares the
endpoint it is handed against
configuredIngestBases(). Once the platform starts rewriting127.0.0.1tohost.docker.internal, the raw configured base and the dispatched endpoint stopmatching,
isAgentaIngestreturns false, and the runner drops the run's platform credential.That is exactly the failure #6407 fixed for
localhost, reintroduced for127.0.0.1.The doc comment at
services/runner/src/tracing/otel.ts:384-385states the current premise inwriting: "
127.0.0.1earns no alias. Neither copy ofparse_urlrewrites it, so that deploymentalready matches its own raw base and the bridge form is a pair the platform cannot produce."
This change makes that premise false, so update the comment along with the set.
4. Tests, one per copy
sdks/python/oss/tests/pytest/unit/test_helpers.pyis a new file. There is no SDK testfor
parse_urltoday, which is part of how the two copies drifted. Mirror the API's existingcases as well as adding the new one, so the next drift is caught.
api/oss/tests/pytest/unit/test_helpers.pyalready covers the localhost cases. Add the127.0.0.1case beside them.configuredIngestBasestests so the127.0.0.1base emits itsbridge alias.
On
::1Consider it, but do not add it as a string replacement. In a URL an IPv6 literal is bracketed, so
the host appears as
[::1]and a naive.replace("::1", ...)corrupts the URL and can alsomatch inside a port or a path. If
::1is worth supporting, do it in the API copy, which alreadyparses the hostname properly, and either rewrite the SDK copy to parse rather than substring-match
or leave
::1out of the SDK and say so in a comment. Leaving it out entirely is a reasonableanswer for this issue.
On the "good first issue" label
Deliberately NOT labelled
good first issue, despite each individual edit being a few lines.The change spans two languages and three files, and it carries a cross-file invariant that is
invisible from any one of them: the set of hosts the platform rewrites must equal the set the
runner expects it to rewrite. A contributor who fixes the two
parse_urlcopies and stops hasmade the deployment worse, and nothing in those two files says so. Both traps are written out
above, so this is approachable, but it is not a first-day task.