Configuration to use Depot on GitLab Ci to avoid rate limiting with Maven Central. - #12222
Configuration to use Depot on GitLab Ci to avoid rate limiting with Maven Central.#12222AlexeyKuznetsov-DD wants to merge 3 commits into
Conversation
There was a problem hiding this comment.
More details
Proxy-disabled builds retain Maven Central, while proxy-enabled Muzzle directives still append their explicit framework repositories and Gradle builds retain the dedicated snapshot, Confluent, Spring, and Gradle repositories. The changed fallback behavior is consistent with making Depot authoritative for Central artifacts.
🤖 Datadog Autotest · Commit e3069e7 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
This comment has been minimized.
This comment has been minimized.
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
There was a problem hiding this comment.
A present-but-blank proxy value is now treated as authoritative, removing Maven Central from general Gradle and Muzzle resolution. Shell-based CI can materialize missing optional variables as empty strings, turning a clean build into an unrecoverable dependency-resolution failure.
🤖 Datadog Autotest · Commit 357e8a8 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
| } else { | ||
| val proxy = RemoteRepository.Builder("central-proxy", "default", mavenProxyUrl).build() | ||
| listOf(proxy, central) | ||
| listOf(proxy) |
There was a problem hiding this comment.
Treat blank proxy values as absent
CI or external builds that export an unset optional proxy variable as an empty string can fail all general dependency resolution before tests run.
Assertion details
- Input:
MAVEN_REPOSITORY_PROXYorORG_GRADLE_PROJECT_mavenRepositoryProxyis present but empty or whitespace-only, such as when shell setup exports an undefined variable. - Expected:
Normalize both proxy values before repository selection and treat blank or whitespace-only values as absent, retaining Maven Central. This requires coordinated changes in both changed files and coverage for blank environment/property values, so no single inline suggestion is complete. - Actual: Both changed paths use presence-only checks. An empty
MAVEN_REPOSITORY_PROXYnow produces only an empty proxy repository, while a present-but-empty Gradle property suppressesmavenCentral(); clean dependency resolution therefore has no usable general repository. Existing tests cover non-empty fixture proxies but not blank default-repository selection.
Was this helpful? React 👍 or 👎
🤖 Datadog Autotest · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
|
Closing this, since infra fixed network configuration that should prevent rate-limiting on Maven Central. |
What Does This Do
Makes the configured Maven repository proxy authoritative for general dependency resolution:
mavenRepositoryProxywhen configured and usemavenCentral()only when the proxy property is absent.MAVEN_REPOSITORY_PROXYwhen configured instead of querying both the proxy and Maven Central.GitLab CI already supplies these proxy settings, so CI resolves these dependencies through Depot. Local and external builds without the settings continue to use Maven Central.
Motivation
The main source of direct Maven Central traffic was the latest-dependency configurations that use dynamic versions such as
2.+,2.7.+, and2.3.20+.For a dynamic selector, Gradle must discover all matching versions. Gradle documentation explains that Gradle checks every defined repository for version metadata rather than stopping after the first repository responds. Consequently, declaring Depot first and Maven Central second did not make Central a passive fallback: Gradle requested
maven-metadata.xmlfrom both repositories for each dynamic dependency.Under parallel CI load, those direct requests can accumulate behind shared egress and eventually result in Maven Central rate limiting. A representative failure was:
Muzzle had the same issue for its version-range scans because its default repository list contained both the proxy and Maven Central.
This change keeps Depot as the general remote repository whenever the proxy is configured, so dynamic-version and Muzzle range lookups no longer query Maven Central alongside Depot through the general repository list.
Additional Notes
Validation:
org.glassfish.grizzly:grizzly-http-server:2.3.20+forlatestDepTestRuntimeClasspathwith the proxy configured; the resolver log showed Depot serving the metadata and no request to Maven Central../gradlew :buildSrc:test --tests datadog.gradle.plugin.muzzle.MuzzleMavenRepoUtilsTest— 4 tests passed../gradlew :buildSrc:spotlessCheckSpecialized repositories with explicit content filters are unchanged.
Contributor Checklist
type:and (comp:orinst:) labels in addition to any other useful labelsclose,fix, or any linking keywords when referencing an issueUse
solvesinstead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]