Skip to content

deps: bump aws-sdk-go to v1.55.5 (allow HTTPS container credential endpoints for CAIOS) - #8

Closed
wimarkuske wants to merge 1 commit into
coreweave:coreweavefrom
wimarkuske:fix/caios-https-cred-endpoint-sdk-bump
Closed

wimarkuske wants to merge 1 commit into
coreweave:coreweavefrom
wimarkuske:fix/caios-https-cred-endpoint-sdk-bump

Conversation

@wimarkuske

@wimarkuske wimarkuske commented Jul 22, 2026 •

Copy link
Copy Markdown

Problem

s5cmd cannot authenticate against CoreWeave CAIOS when credentials are sourced via the container credentials provider pointing at an HTTPS endpoint. A job configured with AWS_CONTAINER_CREDENTIALS_FULL_URI=https://api.coreweave.com/... fails at startup:

2026/07/22 22:50:32 Ignoring, HTTP credential provider invalid endpoint host,
"api.coreweave.com", only loopback hosts are allowed. <nil>
ERROR session: fetching region failed: NoCredentialProviders: no valid providers in chain.
ERROR "sync --concurrency=5 s3://src/* s3://dst/": NoCredentialProviders: no valid providers in chain.

Root cause

The fork is pinned to aws-sdk-go v1.44.298. In that version, aws/defaults/defaults.go localHTTPCredProvider applies the loopback-only host check to the credential endpoint regardless of URL scheme, so any non-loopback host — including a legitimate HTTPS token endpoint — is rejected and the whole provider chain collapses to NoCredentialProviders.

Later aws-sdk-go releases relaxed this: the host restriction (isAllowedHost, now loopback or ECS/EKS container IPs) is gated on parsed.Scheme == "http". HTTPS endpoints bypass the host allow-list, and AWS_CONTAINER_AUTHORIZATION_TOKEN[_FILE] support was added.

Change

  • Bump github.com/aws/aws-sdk-go v1.44.298 → v1.55.5 (last aws-sdk-go v1 release), go mod tidy + go mod vendor.
  • This mirrors the approach of unmerged upstream peak/s5cmd#769.

Verification

Built with go build -mod=vendor ./... (clean). Exercised defaults.RemoteCredProvider with AWS_CONTAINER_CREDENTIALS_FULL_URI set:

FULL_URI Result before (v1.44.298) Result after (v1.55.5)
https://api.coreweave.com/token rejected — only loopback hosts are allowed accepted (provider proceeds to call the endpoint)
http://api.coreweave.com/token rejected still rejected — only loopback/ecs/eks hosts are allowed

So the CAIOS HTTPS case is fixed while the plain-HTTP host restriction is preserved.

🤖 Generated with Claude Code


Release impact

Targets the coreweave branch. On merge, ci.yml push_dirty_tag cuts a new v2.3.0-<sha> tag, which triggers goreleaser.yml (binaries + .deb GitHub Release) and docker.yml (ghcr.io/coreweave/s5cmd:<tag> + :latest). Re-pull that image/deb in the CAIOS transfer job to pick up the fix.

…endpoints

s5cmd fails to authenticate against CoreWeave CAIOS when credentials are
sourced from AWS_CONTAINER_CREDENTIALS_FULL_URI=https://api.coreweave.com/...
The pinned aws-sdk-go v1.44.298 applies the loopback-only host check to the
credential endpoint regardless of scheme, so any non-loopback host is rejected:

  HTTP credential provider invalid endpoint host, "api.coreweave.com",
  only loopback hosts are allowed.
  NoCredentialProviders: no valid providers in chain.

aws-sdk-go relaxed this in later releases: the host restriction now only
applies to plain-http endpoints (aws/defaults/defaults.go now gates the
isAllowedHost check on parsed.Scheme == "http"), while https endpoints are
permitted. Bumping to v1.55.5 (the last aws-sdk-go v1 release) picks up that
change plus AWS_CONTAINER_AUTHORIZATION_TOKEN[_FILE] support.

Verified: with v1.55.5 an https FULL_URI is no longer rejected with the
loopback error, while an http FULL_URI to the same host still is.

Mirrors unmerged upstream PR peak#769.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@wimarkuske
wimarkuske force-pushed the fix/caios-https-cred-endpoint-sdk-bump branch from f3f5fa0 to 7316fee Compare July 22, 2026 23:11
@wimarkuske
wimarkuske changed the base branch from master to coreweave July 22, 2026 23:11
@AndresRCB

Copy link
Copy Markdown
Collaborator

Superseded by #9

@AndresRCB AndresRCB closed this Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants