A non-2xx mediator response drops all winners instead of falling back to local ranking

[aram356]

Description

When an auction has a mediator ([auction] mediator = "adserver_mock") and the mediator answers with a non-2xx status, the auction delivers zero winners. The provider bids already collected are discarded. Every other mediator failure falls back to local ranking of those bids:

Mediator outcome	Result at a4e01eb
Budget exhausted before launch	warning, local ranking
Launch fails	warning, local ranking
Transport error while waiting	warning, local ranking
Response after the hard deadline	warning, local ranking
2xx with a body that is not JSON	warning, local ranking
Non-2xx status	zero winners, no fallback

The crate's auction README states the contract: when a mediator is configured, the orchestrator "falls back to local ranking when mediation cannot run" (crates/trusted-server-core/src/auction/README.md:89-92, added in #1016). The doc comment on collect_dispatched_auction says that on any error or partial failure it returns the best available result rather than propagating (crates/trusted-server-core/src/auction/orchestrator.rs:2092-2094). A test pins the fallback for transport failures (planned_executor_mediator_transport_failure_falls_back_locally, orchestrator.rs:5871). No test covers a non-2xx mediator response.

This is a bug rather than behavior to document, for three reasons. The README promises fallback. The non-2xx result contradicts every other failure mode in the table. And the non-2xx branch predates the fallback design: it has been in adserver_mock since the first auction orchestration commit (488af44, 2026-01-28). The parse-failure and launch or transport fallbacks came later, in #918 and #1016, and the non-2xx path was never aligned with them. The user guide's Error Handling list (docs/guide/auction-orchestration.md:850-858) does not say what happens when the mediator fails, so the fix also needs a doc line.

Steps to reproduce

Add to the mod tests block of crates/trusted-server-core/src/auction/orchestrator.rs. It uses the production plan-backed path (AuctionOrchestrator::from_plan, then run_auction, which dispatches and then calls collect_dispatched_auction) with the real AdServerMockProvider:

#[tokio::test]
async fn mediator_non_success_status_falls_back_to_local_ranking() {
    let http = Arc::new(StubHttpClient::new());
    http.push_response(
        200,
        serde_json::to_vec(&serde_json::json!({
            "seatbid": [{"seat": "provider-seat", "bid": [{
                "id": "provider", "impid": "fictional-slot", "price": 2.0,
                "adm": "<div>provider</div>", "w": 300, "h": 250
            }]}]
        }))
        .expect("should serialize provider response"),
    );
    http.push_response(503, b"mediator unavailable".to_vec());
    let backend = Arc::new(NamingBackend::new(BackendNamingPolicy::Axum));
    let services = build_services_with_backend_and_http_client(
        Arc::clone(&backend) as Arc<_>,
        Arc::clone(&http) as Arc<_>,
    );
    let plan = AuctionPlan::compile(planned_config(
        &[("provider-a", RoutingMode::AllEligible)],
        false,
    ))
    .expect("should compile planned auction");
    let mediator = AdServerMockProvider::new(AdServerMockConfig {
        enabled: true,
        endpoint: "https://mediator.example.com/mediate".to_string(),
        timeout_ms: 500,
        ..AdServerMockConfig::default()
    });
    let orchestrator = AuctionOrchestrator::from_plan(Arc::new(plan), Some(Arc::new(mediator)));
    let request = planned_request();
    let settings = create_test_settings();
    let inbound = http::Request::new(edgezero_core::body::Body::empty());
    let context = AuctionContext {
        settings: &settings,
        request: &inbound,
        timeout_ms: 777,
        transport_timeout_ms: 777,
        provider_responses: None,
        services: &services,
    };

    let result = orchestrator
        .run_auction(&request, &context)
        .await
        .expect("should complete the planned auction");

    assert_eq!(
        result
            .winning_bids
            .get("fictional-slot")
            .and_then(|bid| bid.bid_id.as_deref()),
        Some("provider"),
        "should fall back to local ranking when the mediator returns a non-2xx status"
    );
}

Run (the core test suite also runs natively on the host target; CI runs it with cargo test-fastly):

cargo test -p trusted-server-core --lib --target "$(rustc -vV | sed -n 's/host: //p')" \
  -- auction::orchestrator::tests::mediator_non_success_status_falls_back_to_local_ranking

Observed:

test auction::orchestrator::tests::mediator_non_success_status_falls_back_to_local_ranking ... FAILED
assertion `left == right` failed: should fall back to local ranking when the mediator returns a non-2xx status
  left: None
 right: Some("provider")

The same setup with four mediator outcomes (one provider bid at 2.0 each time):

PROBE mediator 200 with a winner   provider_bids=1 mediator_response=Some(adserver_mock Success, 1 bids) winners=1 winner=provider-a bid_id=Some("mediated") price=Some(2.5)
PROBE mediator 503                 provider_bids=1 mediator_response=Some(adserver_mock Error, 0 bids) winners=0 winner=none
PROBE mediator 200 invalid JSON    provider_bids=1 mediator_response=None winners=1 winner=provider-seat bid_id=Some("provider") price=Some(2.0)
PROBE mediator transport error     provider_bids=1 mediator_response=None winners=1 winner=provider-seat bid_id=Some("provider") price=Some(2.0)

Expected behavior

A non-2xx mediator response is a mediation failure. The orchestrator logs a warning and selects winners from the provider responses with local ranking and floors, as it does for launch, transport, deadline and parse failures.

Actual behavior

The orchestrator takes the mediator's error response as the final decision. winning_bids is empty, so neither POST /auction nor the publisher page path has a bid to deliver, even though provider bids were collected. The test-only AuctionOrchestratorHarness path behaves the same (a 503 gave mediator_response=Some(("adserver_mock", Error, 0)) winners=0).

Telemetry records a mediator provider row with status transport_error and zero bids, and the summary counts zero winners. The status is wrong because the error response carries no error_type, and provider_status defaults to transport_error (telemetry.rs:828-843). The fallback cases record no mediator row at all, because mediator_response is None (telemetry.rs:548-557).

Root cause

AdServerMockProvider::parse_response_inner (crates/trusted-server-core/src/integrations/adserver_mock.rs:366-369) turns a non-2xx status into a successful parse:

if !response.status().is_success() {
    log::warn!("AdServer Mock returned non-success: {}", response.status());
    return Ok(AuctionResponse::error("adserver_mock", response_time_ms));
}

This mirrors how bidder providers record an HTTP status error (auction/provider.rs:550-554). For a bidder that is correct: the error is one provider outcome and the others continue. For a mediator, collect_dispatched_auction treats any Some result as final (auction/orchestrator.rs:2475-2493) and falls back only on None, which it produces for launch errors (2465-2472), transport errors (2460-2463), late completion (2430-2436) and Err from the parser (2448-2456):

if let Some(mediator_response) = mediator_response {
    let winning = mediator_response
            .bids
            .iter()
            .filter_map(/* keep bids with a decoded price */)
            .collect();
    let winning = self.apply_floor_prices(winning, &floor_prices);
    (Some(mediator_response), winning)
} else {
    (None, self.select_winning_bids(&responses, &floor_prices))
}

AuctionResponse::error has no bids (auction/types.rs:392-400), so the winners map is empty.

Impact

• It affects only deployments that configure a mediator. The plan accepts only adserver_mock as the mediator ID (MOCK_MEDIATOR_ID at auction/plan.rs:20, checked at plan.rs:556-563), and the orchestrator builds the mediator only from that integration (auction/mod.rs:89-103). Its guide calls it "a development mediator" (docs/guide/integrations/adserver_mock.md:3), but it also tells operators to "configure an explicit endpoint for shared environments".
• In those environments, any 4xx or 5xx from the mediation endpoint (a restart, an overload, a bad deploy) removes all server-side winners for as long as the errors last. Pages render without server-side ads, and the telemetry status points at the transport rather than an HTTP error.
• The Define integrations crate split and ordered configuration #1194 draft spec assumes today's contract is that "mediator launch or parse failure continues to warn and fall back to local ranking". A future production mediator built on the current seam would inherit this gap.

Proposed fix

The smallest change is in adserver_mock.rs:366-369: return an error for a non-2xx status, so the orchestrator's existing parse-failure branch falls back:

if !response.status().is_success() {
    return Err(Report::new(TrustedServerError::Auction {
        message: format!("AdServer Mock returned HTTP {}", response.status()),
    }));
}

Applied to a copy of a4e01eb, this makes the regression test above pass: a 503 produces the local winner, and all 259 tests under auction:: and integrations::adserver_mock pass.

Two follow-ups are worth deciding in the same change:

• Where the policy lives. Alternatively, collect_dispatched_auction could treat a mediator response with status == BidStatus::Error as a failure and fall back. That protects against any mediator that reports errors as responses. Either way, the warning says "parse failed" for what is an HTTP status failure, so the log text should say which failure it was.
• Telemetry. After the fix, a failed mediation leaves no mediator row, the same as the other failure modes today. Recording a mediator row with error_type = "http_status" (reported as http_status_error) while still using local ranking would make mediator outages visible.

Also document the behavior: add a "Mediator failure" line to the Error Handling list in docs/guide/auction-orchestration.md and a sentence to docs/guide/integrations/adserver_mock.md.

Done when

• A non-2xx mediator response falls back to local ranking with a warning, and mediator_non_success_status_falls_back_to_local_ranking passes.
• The test-only AuctionOrchestratorHarness path behaves the same, with a matching test next to planned_executor_mediator_transport_failure_falls_back_locally.
• Telemetry for a failed mediation is decided and tested: either no mediator row, as for the other failures, or a row with an accurate status.
• docs/guide/auction-orchestration.md and docs/guide/integrations/adserver_mock.md describe mediator failure handling.
• CI gates pass, including cargo test-fastly.

Affected area

Ad serving (Equativ)

Version

main at a4e01eb

Related

• Add configuration-driven OpenRTB auction providers #1016 (merged): added launch and transport fallback on the collect path and the README contract.
• Add first-class APS OpenRTB integration #918 (merged): added the parse-failure fallback on the collect path.
• Define integrations crate split and ordered configuration #1194 (draft spec): states that mediator launch or parse failure falls back to local ranking.
• Integer truncation on bid dimensions in adserver_mock #420 (open) and Reject oversized bid dimensions instead of truncating them #1182 (open): another adserver_mock parsing fix (bid dimension truncation) in the same file. It does not touch the status branch.
• Clarify floor price enforcement authority across the auction pipeline #698 (closed): floor enforcement across the mediator and local paths. The fallback reuses select_winning_bids and apply_floor_prices, so floors still apply.