GTM script rewriter clobbers Next.js __NEXT_DATA__ rewrites and can truncate it

[aram356]

Description

When [integrations.nextjs] and [integrations.google_tag_manager] are both enabled, two script rewriters act on the text of <script id="__NEXT_DATA__">. Next.js matches script#__NEXT_DATA__ and GTM matches every script. Core gives each rewriter its own lol_html text handler. Every handler reads the original chunk text, and lol_html keeps only the last replacement. GTM is registered after Next.js, so whenever GTM rewrites __NEXT_DATA__, its output replaces the Next.js output.

GTM rewrites a script when the text contains googletagmanager.com or google-analytics.com. It also starts buffering a script when a text fragment ends with google or a longer prefix of those hosts. lol_html 2.9.0 hands long script text to the rewriters in fragments: the part inside the first input chunk (8 KiB in production), then 1 KiB pieces.

Verified at a4e01eb:

1. Origin URLs leak. If __NEXT_DATA__ contains a GTM or GA URL, the Next.js rewrite is lost. "href":"https://origin.example.com/reviews" stays in the page, while the GTM URL next to it is rewritten. This has been possible since Add Google Tag Manager first-party integration #262 added GTM's script rewriter next to the Next.js __NEXT_DATA__ rewriter.
2. __NEXT_DATA__ is truncated. If the first GTM or GA host appears after the first fragment, __NEXT_DATA__ is cut down to the fragments GTM buffered and is no longer valid JSON. A fragment that happens to end with google can do the same even when the page has no GTM or GA URL at all. Both rewriters have buffered fragments this way since Phase 3: Make script rewriters fragment-safe for streaming #591 landed on main with Streaming pipeline chunk-emitting #562.

Both symptoms were identical at 4c6d26a, the commit before #1135. #1135 added two related regressions with the same root cause, filed separately as #1206 (App Router pages with GTM URLs in RSC data fail to serve) and #1207 (__NEXT_DATA__ and inline GTM scripts get raw copies of fragments that end in _).

Steps to reproduce

Add this test to mod tests in crates/trusted-server-core/src/integrations/google_tag_manager.rs, next to fragmented_next_data_survives_with_gtm_enabled. The module already has make_settings, config_from_settings, create_html_processor and IntegrationRegistry in scope.

    /// `__NEXT_DATA__` that mentions a GTM URL must receive both the Next.js
    /// and the GTM rewrite, and must stay valid JSON at any chunk size.
    #[test]
    fn next_data_containing_gtm_url_keeps_nextjs_rewrite() {
        use crate::streaming_processor::{Compression, PipelineConfig, StreamingPipeline};
        use std::io::Cursor;

        let mut settings = make_settings();
        settings
            .integrations
            .insert_config(
                "google_tag_manager",
                &serde_json::json!({ "enabled": true, "container_id": "GTM-ABC123" }),
            )
            .expect("should update gtm config");
        settings
            .integrations
            .insert_config(
                "nextjs",
                &serde_json::json!({ "enabled": true, "rewrite_attributes": ["href", "link", "url"] }),
            )
            .expect("should update nextjs config");
        let registry = IntegrationRegistry::with_plan(
            &settings,
            Arc::new(
                crate::auction::compile_auction_plan(&settings)
                    .expect("should compile auction plan"),
            ),
        )
        .expect("should create registry");

        let small = r#"{"props":{"pageProps":{"href":"https://origin.example.com/reviews","gtm":"https://www.googletagmanager.com/gtm.js?id=GTM-ABC123"}}}"#.to_string();
        // The GTM URL sits after the first 8 KiB input chunk.
        let large = format!(
            r#"{{"props":{{"pageProps":{{"href":"https://origin.example.com/reviews","body":"{}","gtm":"https://www.googletagmanager.com/gtm.js?id=GTM-ABC123"}}}}}}"#,
            "lorem ipsum ".repeat(800)
        );
        let start_tag = r#"<script id="__NEXT_DATA__" type="application/json">"#;

        for (payload, chunk_size) in [(&small, 8192), (&small, 32), (&large, 8192)] {
            let html = format!("<html><body>{start_tag}{payload}</script></body></html>");
            let processor = create_html_processor(config_from_settings(&settings, &registry));
            let mut pipeline = StreamingPipeline::new(
                PipelineConfig {
                    input_compression: Compression::None,
                    output_compression: Compression::None,
                    chunk_size,
                },
                processor,
            );
            let mut output = Vec::new();
            pipeline
                .process(Cursor::new(html.as_bytes()), &mut output)
                .expect("should process HTML");
            let processed = String::from_utf8(output).expect("should produce UTF-8");

            let start = processed
                .find(start_tag)
                .expect("should keep __NEXT_DATA__")
                + start_tag.len();
            let end = start
                + processed[start..]
                    .find("</script>")
                    .expect("should close __NEXT_DATA__");
            let data: serde_json::Value = serde_json::from_str(&processed[start..end])
                .unwrap_or_else(|error| {
                    panic!(
                        "should keep __NEXT_DATA__ valid JSON (payload {} bytes, chunk {chunk_size}): {error}",
                        payload.len()
                    )
                });
            assert_eq!(
                data["props"]["pageProps"]["href"],
                "https://test.example.com/reviews",
                "should keep the Next.js rewrite (payload {} bytes, chunk {chunk_size})",
                payload.len()
            );
            assert_eq!(
                data["props"]["pageProps"]["gtm"],
                "/integrations/google_tag_manager/gtm.js?id=GTM-ABC123",
                "should keep the GTM rewrite (payload {} bytes, chunk {chunk_size})",
                payload.len()
            );
        }
    }

cargo test -p trusted-server-core --target <host-triple> --lib -- \
  next_data_containing_gtm_url_keeps_nextjs_rewrite

Observed at a4e01eb (the test also fails at 4c6d26a):

---- integrations::google_tag_manager::tests::next_data_containing_gtm_url_keeps_nextjs_rewrite stdout ----
assertion `left == right` failed: should keep the Next.js rewrite (payload 131 bytes, chunk 8192)
  left: String("https://origin.example.com/reviews")
 right: "https://test.example.com/reviews"

The test stops at its first case. I also ran a scratch probe that drives create_html_processor through StreamingPipeline, and the publisher entry point publisher::stream_publisher_body. Both paths gave the same results. None of these payloads contains an _, so the separate #1135 regression is not involved. With GTM and Next.js enabled:

Input	__NEXT_DATA__ at a4e01eb
The small payload above, 8192-byte chunks	{"props":{"pageProps":{"href":"https://origin.example.com/reviews","gtm":"/integrations/google_tag_manager/gtm.js?id=GTM-ABC123"}}}
Same payload, 32-byte chunks	w.googletagmanager.com/gtm.js?id=GTM-ABC123"}}} (47 bytes, not JSON)
9,157-byte payload, GTM URL after the first 8 KiB, 8192-byte chunks	1,043 bytes starting mid-string with amet lorem ipsum, not JSON
__NEXT_DATA__ with a GA URL (https://www.google-analytics.com/g/collect) instead of GTM	origin href kept, GA URL rewritten to /integrations/google_tag_manager/g/collect
11,154-byte payload with no GTM or GA URL, input chunk ending in google (inside a googleapis URL) at byte 8192	9,140 bytes, not JSON (EOF while parsing a string), origin href kept
Same, with the split at byte 16384 (19,346-byte payload)	2,048 bytes, not JSON
Every case above with only Next.js enabled (control)	valid JSON, href rewritten to https://test.example.com/reviews

Expected behavior

• When several rewriters match a script, the output is the same as running them one after another on its complete text, wherever chunk boundaries fall.
• __NEXT_DATA__ gets both rewrites (href on the publisher host, GTM URL on the first-party GTM path) and stays valid JSON.
• No script text is dropped or reordered.

Actual behavior

• The rewriter registered last wins. For a __NEXT_DATA__ that GTM rewrites, the Next.js rewrite is lost and origin URLs stay in the page.
• A fragmented __NEXT_DATA__ can lose everything before the fragment where GTM started buffering, which leaves invalid JSON.

Root cause

Handlers are registered independently. create_html_processor adds one lol_html text handler per script rewriter, and each handler applies its action straight to the shared chunk (html_processor.rs:664-694):

match rewriter.rewrite(text.as_str(), &ctx) {
    ScriptRewriteAction::Keep => {}
    ScriptRewriteAction::Replace(rewritten) => {
        text.replace(&rewritten, ContentType::Text);
    }
    ScriptRewriteAction::RemoveNode => {
        text.remove();
    }
}

Script rewriters are collected in registration order (registry.rs:889, inner.script_rewriters.extend(registration.script_rewriters)):

• nextjs comes before google_tag_manager in builders() (integrations/mod.rs:302-305 and 326-329).
• Next.js adds script#__NEXT_DATA__ and then script (nextjs/mod.rs:97-98), and GTM adds script (google_tag_manager.rs:1027-1029).
• The resulting order is [("nextjs", "script#__NEXT_DATA__"), ("nextjs", "script"), ("google_tag_manager", "script")], and lol_html runs the matching text handlers in that order for each chunk.

lol_html does not compose the mutations. Cargo.lock pins lol_html 2.9.0. There, TextChunk::as_str() returns the original chunk (&self.text), so every handler sees unmodified input. replace() calls MutationsInner::replace, which sets removed = true, clears any earlier replacement and stores the new one. remove() only sets removed = true. On output, a removed chunk emits its replacement. So for each chunk the last replace() wins, a remove() from one handler cannot cancel another handler's replace(), and no handler sees another handler's output.

lol_html fragments long script text. Its text decoder emits the text found in the first input chunk as one fragment. After that it decodes through a 1,024-byte buffer (DEFAULT_BUFFER_LEN in src/rewritable_units/text_decoder.rs), so the rest of a long script arrives in 1 KiB pieces.

Each rewriter buffers fragments on its own.

• NextJsNextDataRewriter::rewrite (nextjs/script_rewriter.rs:64-93, using capture_fragment in nextjs/rsc_stream.rs:89-144) returns RemoveNode for intermediate fragments while it buffers them, then Replace with the rewritten complete text on the last fragment.
• GTM's rewrite (google_tag_manager.rs:1031-1087) buffers a fragment and returns RemoveNode when might_contain_gtm_prefix (:319-334) matches the buffer or the fragment (:1045-1054). On the last fragment it returns Replace(Self::rewrite_gtm_urls(text)) when a marker from GTM_SCRIPT_MARKERS (:296) is present (:1073-1077), or Replace with the buffered text if it buffered anything (:1082-1083).

Both call replace on the last fragment, and GTM runs last. GTM's text is built from original fragments, so the Next.js rewrite is lost (symptom 1). If GTM began buffering after the first fragment, its text lacks the earlier fragments that Next.js suppressed (symptom 2).

GTM also stops adding fragments to its buffer once neither the buffer nor the new fragment "might match" (:1045-1056). It returns Keep for those fragments, so they are emitted ahead of the fragments it is still holding. That reorders script text even when GTM is the only rewriter. With GTM alone, an inline script whose first 8 KiB chunk ends in google and that spans four chunks came out with its third and fourth chunks before its first. This happens at both a4e01eb and 4c6d26a.

The existing guard covers only part of this. #562 added the might_contain_gtm_prefix gate so GTM returns Keep for scripts that cannot contain its hosts, and its comment names the __NEXT_DATA__ overlap (google_tag_manager.rs:1037-1044). The regression tests fragmented_next_data_survives_with_gtm_enabled and fragmented_next_data_with_trailing_g_survives_gtm use __NEXT_DATA__ without any GTM or GA URL.

Impact

Affected deployments enable both [integrations.nextjs] (off by default) and [integrations.google_tag_manager] and serve Next.js Pages Router pages.

• Origin leak. The Next.js integration rewrites __NEXT_DATA__ "to route traffic through first-party proxying" (docs/guide/integrations/nextjs.md:9). Next.js hydrates from __NEXT_DATA__, so with this bug, links and requests built from href, link and url props point at the origin host. Navigation leaves the publisher host, and with it the first-party context Trusted Server provides. The origin hostname is also exposed in the page source, and search engines that render the page can discover origin-host links. Every page whose __NEXT_DATA__ mentions a GTM or GA host gets this or the truncation below, for example when analytics settings or CMS embed code are passed as page props.
• Broken hydration. The Next.js 14.2.33 client reads the element with JSON.parse(document.getElementById("__NEXT_DATA__").textContent) (next/dist/client/index.js). Invalid JSON throws during client start-up, so the server-rendered HTML is shown but never becomes interactive, and client-side navigation does not work. This happens when the first GTM or GA URL sits past the first fragment boundary inside __NEXT_DATA__. It can also happen on pages with no GTM or GA URL, when a fragment ends in google.
• Likelihood. I measured this on 10 public Pages Router pages (the sample described in Regression from #1135: __NEXT_DATA__ is corrupted when script text splits after _ #1207). None had a GTM or GA URL inside __NEXT_DATA__, and 6 contained the substring google (2 to 35 times). To separate this bug from the _ regression, I applied that ticket's stopgap and ran the pages with GTM and Next.js at 32 alignments each, as identity and gzip bodies. 2 of 640 runs broke, both on the page with 32 occurrences. With the stopgap in place, the google path is the only GTM trigger left for pages with no GTM or GA URL. So in that sample this issue is uncommon. Pages that do pass a GTM or GA URL through page props hit symptom 1 or 2 on every request.

Proposed fix

Compose script rewriters in core, the way attribute rewriters already chain in IntegrationRegistry::rewrite_attribute (registry.rs:1045-1058). Keep lol_html for selector matching. Record which rewriters match the current <script> from per-selector element handlers. Then use a single text handler that pipes each chunk through the matching rewriters in order and applies one mutation:

let matched = Rc::new(RefCell::new(vec![false; script_rewriters.len()]));
{
    let matched = matched.clone();
    element_content_handlers.push(element!("script", move |_el| {
        matched.borrow_mut().iter_mut().for_each(|flag| *flag = false);
        Ok(())
    }));
}
for (index, script_rewriter) in script_rewriters.iter().enumerate() {
    let matched = matched.clone();
    element_content_handlers.push(element!(script_rewriter.selector(), move |_el| {
        matched.borrow_mut()[index] = true;
        Ok(())
    }));
}
element_content_handlers.push(text!("script", move |text| {
    let flags = matched.borrow().clone();
    let mut current: Option<String> = None;
    for (index, rewriter) in rewriters.iter().enumerate() {
        if !flags[index] {
            continue;
        }
        let ctx = IntegrationScriptContext { selector: rewriter.selector(), /* as today */ };
        match rewriter.rewrite(current.as_deref().unwrap_or(text.as_str()), &ctx) {
            ScriptRewriteAction::Keep => {}
            ScriptRewriteAction::Replace(rewritten) => current = Some(rewritten),
            ScriptRewriteAction::RemoveNode => current = Some(String::new()),
        }
    }
    match current {
        None => {}
        Some(value) if value.is_empty() => text.remove(),
        Some(value) => text.replace(&value, ContentType::Text),
    }
    Ok(())
}));

Each rewriter then works on the previous rewriter's output. A fragment that Next.js holds reaches GTM as empty text, and on the last fragment GTM receives Next.js's complete rewritten __NEXT_DATA__, so both rewrites apply. This assumes every script rewriter selector targets <script>, which is true today. A debug assertion on registration would keep it true.

In the same change, make GTM's buffering keep source order: once it holds a fragment, keep holding until the last one. At google_tag_manager.rs:1045-1046 that means !buf.is_empty() || might_contain_gtm_prefix(content).

I prototyped both changes in a scratch copy of a4e01eb:

• Every case in the table above came out as valid JSON with the Next.js rewrite, plus the GTM rewrite wherever a GTM or GA URL was present.
• GTM alone no longer reordered script text.
• Every existing test in a native cargo test -p trusted-server-core --lib run passed, and so did the test above.
• The same prototype resolves Regression from #1135: App Router pages with GTM URLs in RSC data fail to serve #1206 and Regression from #1135: __NEXT_DATA__ is corrupted when script text splits after _ #1207: their tests pass, the affected pages are served completely under Viceroy, and the real-page measurement drops to zero.

For __NEXT_DATA__, the order of GTM and Next.js inside the composition does not change the result once GTM's buffering keeps source order. Without the buffering fix, GTM-first order produced invalid JSON when a fragment ended in google. The order does matter for GTM URLs inside App Router flight data, which is discussed in #1206.

Alternatives considered:

• Making GTM skip __NEXT_DATA__ or JSON scripts fixes this issue only. The two Make body hold parser-aware and stream Next.js processing #1135 regressions involve the Next.js flight-data rewriter, which matches every <script>.
• Buffering each matched script completely and running the rewriters once on the full text is simpler. But it holds every matched script in memory until its end, including large flight scripts, which works against the bounded streaming that Make body hold parser-aware and stream Next.js processing #1135 added.

Compatibility: output changes only for scripts that more than one rewriter matches. No configuration changes.

Done when

• Script rewriters that match the same element are composed in core, and handler registration order no longer decides which rewrite survives.
• next_data_containing_gtm_url_keeps_nextjs_rewrite passes: both rewrites and valid JSON at 8192-byte and 32-byte chunks, and for a payload whose GTM URL is past the first 8 KiB.
• A test with a __NEXT_DATA__ fragment ending in google and no GTM or GA URL keeps valid JSON with the Next.js rewrite.
• GTM's buffering keeps source order: a GTM-only test with an inline script split after google across four or more chunks keeps its text in order.
• Existing GTM and Next.js fragment tests still pass.

Affected area

HTML processing / JS injection

Version

main at a4e01eb

Related

• Regression from #1135: App Router pages with GTM URLs in RSC data fail to serve #1206: a regression from Make body hold parser-aware and stream Next.js processing #1135 with the same root cause. App Router pages with GTM URLs in RSC data fail to serve.
• Regression from #1135: __NEXT_DATA__ is corrupted when script text splits after _ #1207: a regression from Make body hold parser-aware and stream Next.js processing #1135 with the same root cause. __NEXT_DATA__ is corrupted when script text splits after _.
• Add Google Tag Manager first-party integration #262: added GTM's script selector rewriter next to the Next.js script#__NEXT_DATA__ rewriter.
• Phase 3: Make script rewriters fragment-safe for full streaming #584 (closed): planned fragment accumulation for both rewriters and accepted "last text.replace() wins" for overlapping selectors.
• Phase 3: Make script rewriters fragment-safe for streaming #591, landed via Streaming pipeline chunk-emitting #562: implemented the accumulation and the might_contain_gtm_prefix gate for scripts without GTM content.
• Define integrations crate split and ordered configuration #1194 (open): integrations ordering spec. Its review found that script rewriters are last-replace-wins, so reordering only changes which rewrite is lost.