From 9ee8a639b63e1adc50a9d64452ae8098bd35de73 Mon Sep 17 00:00:00 2001 From: DJ Satoda Date: Wed, 15 Jul 2026 10:31:25 -0400 Subject: [PATCH 1/4] Clarify what RCA percentage change (lift) and contribution numbers mean --- docs/root-cause-analysis.mdx | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/root-cause-analysis.mdx b/docs/root-cause-analysis.mdx index 51fe5574..2c5bf1db 100644 --- a/docs/root-cause-analysis.mdx +++ b/docs/root-cause-analysis.mdx @@ -20,6 +20,10 @@ When you run an RCA, Mixpanel immediately launches a new Board and exposes the a Cards stream into the Board as each step completes, with a live progress banner at the top while the agent is still working. + + **What the numbers mean.** The percentage change (sometimes called "lift") compares the data point you flagged against a baseline, which is the average of the periods just before it: `(flagged value - baseline) / baseline`. So +30% means the flagged point came in 30% above the recent norm, and a negative number means it came in below. Each segment's contribution percentage is that segment's share of the total change: `(how much the segment moved) / (how much the whole metric moved)`. For example, if a metric dropped by 500 overall and one country accounts for 400 of that drop, that country shows an 80% contribution. + + ## Where to launch RCA From fe277267dafef8074309b3fa79eecc205cdd8b81 Mon Sep 17 00:00:00 2001 From: DJ Satoda Date: Wed, 15 Jul 2026 10:40:31 -0400 Subject: [PATCH 2/4] Add note explaining how RCA confidence is determined --- docs/root-cause-analysis.mdx | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/root-cause-analysis.mdx b/docs/root-cause-analysis.mdx index 2c5bf1db..3f510bb1 100644 --- a/docs/root-cause-analysis.mdx +++ b/docs/root-cause-analysis.mdx @@ -22,6 +22,8 @@ Cards stream into the Board as each step completes, with a live progress banner **What the numbers mean.** The percentage change (sometimes called "lift") compares the data point you flagged against a baseline, which is the average of the periods just before it: `(flagged value - baseline) / baseline`. So +30% means the flagged point came in 30% above the recent norm, and a negative number means it came in below. Each segment's contribution percentage is that segment's share of the total change: `(how much the segment moved) / (how much the whole metric moved)`. For example, if a metric dropped by 500 overall and one country accounts for 400 of that drop, that country shows an 80% contribution. + + **Confidence works differently.** Unlike the percentages above, confidence is not a calculated number. It's a label ("High," "Medium," or "Low") that the AI assigns to each finding based on its own read of how strong the evidence is. That read leans on two things: how unusual the change is versus normal fluctuation (the z-score), and how cleanly a single segment explains the move (a high contribution percentage). A big, clear change driven mostly by one segment earns higher confidence; a smaller change spread thinly across many segments, or one that looks like a normal seasonal pattern, earns lower confidence. ## Where to launch RCA From edf06babdb9464c7ea734c9e385dc85f06c2af76 Mon Sep 17 00:00:00 2001 From: DJ Satoda Date: Wed, 15 Jul 2026 10:41:49 -0400 Subject: [PATCH 3/4] Explain RCA numbers and confidence in one callout --- docs/root-cause-analysis.mdx | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/root-cause-analysis.mdx b/docs/root-cause-analysis.mdx index 3f510bb1..386d5092 100644 --- a/docs/root-cause-analysis.mdx +++ b/docs/root-cause-analysis.mdx @@ -21,9 +21,7 @@ When you run an RCA, Mixpanel immediately launches a new Board and exposes the a Cards stream into the Board as each step completes, with a live progress banner at the top while the agent is still working. - **What the numbers mean.** The percentage change (sometimes called "lift") compares the data point you flagged against a baseline, which is the average of the periods just before it: `(flagged value - baseline) / baseline`. So +30% means the flagged point came in 30% above the recent norm, and a negative number means it came in below. Each segment's contribution percentage is that segment's share of the total change: `(how much the segment moved) / (how much the whole metric moved)`. For example, if a metric dropped by 500 overall and one country accounts for 400 of that drop, that country shows an 80% contribution. - - **Confidence works differently.** Unlike the percentages above, confidence is not a calculated number. It's a label ("High," "Medium," or "Low") that the AI assigns to each finding based on its own read of how strong the evidence is. That read leans on two things: how unusual the change is versus normal fluctuation (the z-score), and how cleanly a single segment explains the move (a high contribution percentage). A big, clear change driven mostly by one segment earns higher confidence; a smaller change spread thinly across many segments, or one that looks like a normal seasonal pattern, earns lower confidence. + **What the numbers mean.** The percentage change (sometimes called "lift") compares the data point you flagged against a baseline, which is the average of the periods just before it: `(flagged value - baseline) / baseline`, so +30% means it came in 30% above the recent norm and a negative number means below. Each segment's contribution percentage is its share of the total change: `(how much the segment moved) / (how much the whole metric moved)`, so if a metric dropped by 500 overall and one country accounts for 400 of that, it shows an 80% contribution. Confidence works differently: it's not a calculated number but a label ("High," "Medium," or "Low") the AI assigns to each finding based on how unusual the change is versus normal fluctuation (the z-score) and how cleanly a single segment explains it, so a big, clear change driven by one segment earns higher confidence than a small change spread across many segments or one that looks seasonal. ## Where to launch RCA From e7428ffc5ed574608e6c0e61ab9f01e020246439 Mon Sep 17 00:00:00 2001 From: DJ Satoda Date: Thu, 16 Jul 2026 10:10:18 -0400 Subject: [PATCH 4/4] Add baseline window sizes, z-score threshold, and seasonality detail to RCA validation step --- docs/root-cause-analysis.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/root-cause-analysis.mdx b/docs/root-cause-analysis.mdx index 386d5092..a17dd2b9 100644 --- a/docs/root-cause-analysis.mdx +++ b/docs/root-cause-analysis.mdx @@ -13,7 +13,7 @@ Mixpanel's **AI-Powered Root Cause Analysis (RCA)** feature does that work for y When you run an RCA, Mixpanel immediately launches a new Board and exposes the agent's reasoning as it works: -1. **Validates the anomaly.** Confirms there's a real change to explain before going further, using a z-score to check the change is well outside normal day-to-day variation rather than routine noise or seasonality. +1. **Validates the anomaly.** Confirms there's a real change to explain before going further. RCA compares your flagged point against a baseline window that ends right before it, sized automatically from your chart's time grain (hourly uses roughly the prior 24 hours, daily ~14 days, weekly ~9 weeks, monthly ~6 months); if your chart shows a longer range than that, RCA uses your range instead. It then measures how many standard deviations your point sits from that window's average (the z-score), with 2 or more being the bar for a real anomaly rather than routine noise. A seasonality check compares like-for-like periods (same weekday for daily data, same hour-of-day for hourly) so normal recurring patterns don't get flagged. 2. **Selects properties to analyze.** An LLM agent picks which properties to break down on, combining properties Mixpanel suggests for the event with any you've prioritized in [Custom RCA Settings](#custom-rca-settings). It spreads picks across plausible cause categories (acquisition, audience, platform, data quality, user identity), filters out high-cardinality identifiers and near-duplicates, and factors in your project's business context. 3. **Runs breakdowns.** Executes the chosen breakdowns to see which segments actually moved. 4. **Writes an interpretation.** Fills the Board with contribution factor analysis (each segment's share of the total change), ranked dimensions, an AI-written explanation of the likely cause, a confidence level, and suggested next steps.