feat!: allow passing a custom HTTP client to ApifyClient - #1058
Merged
Merged
Conversation
vdusek
added
the
t-tooling
Contributor
|
See more at https://github.com/apify/apify-client-js/actions/runs/36108249341#summary-107985616997 |
# Conflicts: # src/resource_clients/dataset.ts
# Conflicts: # docs/02_concepts/02_error-handling.md # docs/public-api/apify-client.api.md # src/apify_api_error.ts # src/apify_client.ts # src/base/resource_client.ts # src/http_client.ts # src/resource_clients/actor.ts # src/resource_clients/dataset.ts # src/resource_clients/key_value_store.ts # src/resource_clients/log.ts # src/resource_clients/request_queue.ts # src/resource_clients/run.ts # src/resource_clients/task.ts # test/client_timeouts.test.ts # test/http_client.test.ts
# Conflicts: # docs/04_upgrading/upgrading_v3.md # src/apify_api_error.ts # src/http_client.ts # src/resource_clients/dataset.ts
# Conflicts: # docs/04_upgrading/upgrading_v3.md # src/apify_api_error.ts # src/http_client.ts # src/interceptors.ts # src/utils.ts # test/http_client.test.ts
The configurable request compression from #1065 moves into the shared pipeline of the new `HttpClient` base, where the compression step lives now that `http_client.ts` and `interceptors.ts` are gone. `compression` becomes an `HttpClientOptions` field resolved in the base constructor, so `ApifyClient` forwards the option instead of resolving it, and a custom HTTP client gets the same compression as `AxiosHttpClient`. The new HTTP clients concept page moves to `08_http-clients.md`, since #1065 took the `07_` slot for HTTP compression.
janbuchar
reviewed
Sep 16, 2026
janbuchar
reviewed
Sep 16, 2026
janbuchar
reviewed
Sep 24, 2026
janbuchar
approved these changes
Sep 24, 2026
This was referenced Sep 25, 2026
vdusek
added a commit
that referenced
this pull request
Sep 25, 2026
Adds a regression test for #277: `start()` with a `Buffer` input and `contentType: 'application/json'` sends the raw bytes, and the server gets the decoded JSON back. The bug itself is already gone on `v3`. `serializeBody()` from #1058 passes binary and string bodies through before it reaches `JSON.stringify`, and axios no longer re-serializes strings since #1051 set `transformRequest: []`. With the Buffer branch disabled, the new test fails with the `{ type: 'Buffer', data: [...] }` payload from the issue. Closes #277 *✍️ Drafted by Claude Code*
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
ApifyClientsends its requests through an HTTP client you can replace, following apify-client-python (apify/apify-client-python#641). The abstractHttpClientholds the shared pipeline: header merging, body serialization, compression, retries with backoff, timeout growth, body parsing and theApifyApiErrorconversion. A transport implementssendRequest()and can overrideisTimeoutError(),isRetryableTransportError()andclose().AxiosHttpClientis the only built-in transport, andasync-retryis no longer a dependency.ApifyClient.withCustomHttpClient({ token, baseUrl, publicBaseUrl, httpClient })plugs a client in, as Python'swith_custom_http_clientdoes, since the retry and timeout options of the constructor configure the default client only.ApifyClientOptions.headersreplaces the fixed-header use ofrequestInterceptors.ApifyRequestConfig.signalaborts a call, which then rejects with the signal'sreason.HttpClientvalidates its retry and timeout options with the shapeApifyClientOptionsuses.Docs: an "HTTP clients" concept page, a "Build a custom HTTP client" guide over
fetchwith a section on reusing a Crawlee 4 client, the v3 upgrading guide and the regenerated api-extractor report. Tests drive acall()-only client and a hooks-only client overnode:httpthrough the pipeline.Issue
Closes #855
Follow-up: #1071
Breaking changes
requestInterceptorsmoved fromApifyClientOptionstoAxiosHttpClientOptions. Useheadersfor a fixed header, or pass anAxiosHttpClienttowithCustomHttpClient(). An interceptor now sees the body already serialized.ApifyRequestConfigandApifyResponseno longer extend the axios types.forceBufferbecameresponseType: 'buffer', andresponseTypetakes'parsed' | 'buffer' | 'stream'.InvalidResponseBodyError.responseis the transport'sHttpResponsewith the rawbody.ApifyApiError.httpMethodis uppercase (GET).Blob,File,FormDataor webReadableStreambody throws aTypeError. Read it into aBufferfirst.key[].timeoutMaxSecs, and retry delays are randomized.axios,httpAgentandhttpsAgentproperties moved fromHttpClienttoAxiosHttpClient, and the instance no longer carries the default headers.✍️ Drafted by Claude Code