← Knowledge HubFROM BASIC TO ADVANCED

Root cause analysis: investigate a metric drop without guessing

Diagnose a metric drop before planning a fix. Begin with the anomaly, measurement quality and the affected population; execution is the supporting topic for mitigation and recovery.

Original PMMockr learning guide · Examples are illustrative, not company interview transcripts.

Before you start

Understand counts, rates and the difference between an event log and the customer outcome.

What you will learn

Localize the absolute contribution to a drop.
Keep competing hypotheses alive until evidence separates them.
Choose a reversible mitigation without pretending the root cause is proven.

Read the explanation, attempt the problem yourself, then compare the worked answer.

START WITH A QUESTION

Weekly successful document exports fell after a release. Diagnose the issue.

Original teaching scenario. Any figures are illustrative assumptions.

Scope and assumptions

Hypothetical counts: exports fell from 1,000 to 800 in comparable weeks.
One desktop version fell from 400 to 200; other cohorts stayed at 600.

How to tackle this question

Confirm the definition, timing and magnitude of the drop, then reconcile client events with independent server outcomes.
Quantify which cohort contributes most to the absolute loss. Predict a different observation for each competing hypothesis.
Request the check that best separates those hypotheses. Choose a safe mitigation and explain what remains unproven.

Try a short outline and spoken answer before comparing.

Reveal model solution and reasoning

Quantify and localize

The total loss is 200 exports, a 20% relative drop. The affected version also lost 200, accounting for the full observed decline. I would still verify event definitions, comparable user exposure and reporting lag.

Separate hypotheses

A missing completion event and a real export failure both fit the dashboard. I would compare stored export artifacts, successful download responses and client errors for the same cohort before choosing a cause.

Mitigate and validate

If artifacts and download responses also fall while errors rise, a real failure becomes more likely. I would pause that rollout or use the previous exporter when safe, then check affected and unaffected cohorts for recovery.

Interviewer follow-up

Export files are created normally, but the dashboard event is missing. What would you do?

I would repair tracking and annotate the reporting gap rather than launch a user-retention campaign. I would confirm downloads still work and avoid claiming a behavioral decline from a missing event.

Check your reasoning

Did I validate measurement before claiming a behavioral change?
Can the affected cohort explain the absolute loss?
Did I distinguish mitigation, correlation and evidence of cause?

Try a changed constraint

Create three competing hypotheses and specify one observation that would make each less likely.

Go deeper

A large percentage decline in a tiny cohort may explain little of the total; calculate absolute contribution before following that branch.

Recovery after rollback strengthens a release hypothesis but can be confounded by simultaneous changes; compare independent evidence and unaffected cohorts.

Start with the vocabulary

Clarification

A question that resolves an ambiguity which could change your recommendation. Clarify the goal, audience, scope, constraints and time horizon; do not ask a long list of questions just to follow a template.

“Are we improving the experience for first-time riders or increasing repeat rides?” These goals could lead to different products.

Common mistake: Asking about button colors before knowing which user problem matters.

Cohort and funnel

A cohort groups users by a shared starting event or characteristic. A funnel measures movement through ordered steps. Combining them helps locate where an experience breaks for a particular group.

Compare new Android users moving from signup to payment before and after a release, using the same observation window.

Common mistake: Comparing this week’s new users with last month’s mature users without accounting for age.

Guardrail metric

A measure that tells you whether improving your main objective is causing unacceptable harm elsewhere. Name the downside, how you measure it and what action a breach would trigger.

A faster checkout experiment can improve conversion while increasing refund requests. Track refund rate alongside conversion and agree a rollback threshold before the experiment.

Common mistake: Listing “quality” as a guardrail without an observable measure or decision rule.

Experiment

A structured test of a hypothesis with a comparison group, a primary metric and a decision rule. Think about assignment, sample size, interference and how long outcomes need to mature.

Randomize eligible accounts into old and new onboarding, and assess activation while monitoring support burden.

Common mistake: Stopping as soon as one dashboard turns green.

Hypothesis and diagnostic test

A possible explanation and an observation that can distinguish it from another explanation. A plausible story is not a confirmed cause.

If dashboard exports fall, compare saved files and download responses to distinguish broken tracking from a real export failure.

Common mistake: Listing many causes without saying what evidence would change your belief.

Denominator and observation window

The eligible population against which an event is counted, and the period allowed for that event to occur. Rates are comparable only when eligibility and maturity are comparable.

Activation within seven days counts mature new-team cohorts; teams created yesterday have not had the same opportunity to activate.

Common mistake: Calling a rate higher after silently excluding users who struggle.

Build your answer, layer by layer

This is a learning sequence, not a mandatory order for presenting every answer. Adapt to the actual question.

LAYER 01

Define the incident

Confirm the exact metric, baseline, current value, time window and scope. Distinguish a relative percentage change from percentage points.

Apply it: Ask whether conversion fell from 20% to 18% or by 2% relative.

Go deeper: Clarify whether the change is persistent, seasonal or within normal variation.

LAYER 02

Check the measurement

Verify events, dashboards, logging, denominator changes and reporting delay before assuming behavior changed.

Apply it: Compare raw completed purchases with the dashboard aggregate.

Go deeper: Explain what evidence would distinguish a logging defect from a real loss.

LAYER 03

Localize the break

Use journey stages and cohort cuts to find where the change concentrates. Follow the highest-information cut rather than mechanically requesting every split.

Apply it: Compare cart-to-payment and payment-to-confirmation across app versions.

Go deeper: Account for changes in traffic mix and cohort maturity.

LAYER 04

Build competing hypotheses

Separate internal changes, external conditions and user behavior. Keep several plausible causes alive until evidence differentiates them.

Apply it: Consider payment failures, price changes and a lower-intent acquisition campaign.

Go deeper: State what observation would make each hypothesis less likely.

LAYER 05

Prioritize diagnostic tests

Choose checks by expected information, speed and impact. Ask for a measurement that can separate hypotheses.

Apply it: Compare payment error rates by provider and version before requesting every server log.

Go deeper: Explain what you would do if the requested data is unavailable.

LAYER 06

Mitigate and validate the cause

Protect users while investigating. Distinguish a useful mitigation from proof of the underlying cause.

Apply it: Reroute a failing payment provider if evidence supports it, then compare recovery with unaffected cohorts.

Go deeper: Avoid claiming a rollback proves causality when other conditions also changed.

LAYER 07

Prevent recurrence

Recommend owners, monitoring and a follow-up test tied to the diagnosed failure. Close with confidence and remaining uncertainty.

Apply it: Add provider-level failure alerts and a tested fallback path.

Go deeper: Name the evidence needed before making a permanent fix.

Use observations to narrow the tree

Suppose successful exports fall from 1,000 to 800 in comparable windows. Android accounts for 180 lost exports and web for 20: prioritize Android because it contributes 90% of the observed loss. A large relative fall in a tiny cohort may matter less to the total, while a severe harm can still justify priority regardless of size. Reconcile client events with server completions before concluding that exports actually failed.

Choose the test that separates explanations

If server completions remain steady, event loss is plausible. If both fall after a client release, compare exposure, failure codes and unaffected versions. A timing coincidence does not rule out a backend outage or traffic-mix change. Predict the observation each explanation implies and request the smallest useful check. If a release plausibly causes ongoing harm, coordinate a safe rollback while continuing investigation; recovery is evidence, not final proof. Preserve a timeline and assign a follow-up owner.

Connect the reading to this case

Diagnosis asks what explains a change; measurement asks what success means. Keep the two tasks separate before choosing evidence or proposing a fix.

Aakash Gupta: PM metrics interview questions

Gmail user growth is up, but revenue is flat. How would you diagnose it?

Attempt this second problem before comparing. Behavioral answers must use a true experience.

Compare with PMMockr’s worked solution

Original public teaching answer; hypothetical figures and fictional experience are not results to claim as your own.

I would clarify the revenue definition and period before diagnosing. Is this paid-seat revenue in a hypothetical business model, advertising revenue, or an allocated accounting measure? I would not assume all Gmail usage generates revenue directly.

Assuming paid-seat revenue, decompose it into paying accounts, paid seats per account and realized revenue per seat. Reconcile revenue recognition, currency and billing changes first. Then compare user growth with growth in paid and free cohorts, upgrades, downgrades, cancellations and discounts.

If new users are mostly free while paid-seat retention is stable, flat revenue may reflect the acquisition mix rather than a broken checkout. If paid conversion fell in a specific segment, inspect the funnel and recent changes there. Prioritize the largest supported contributor, assign the next discriminating check, and propose a fix only after evidence distinguishes the hypotheses.

Pushback

Would you increase prices immediately?

No. First determine whether price, conversion, seat mix or recognition explains the result. A price change could mask churn or damage conversion without fixing the cause.

Common mistake

Users grew, so payment must be broken.

Why this answer is stronger

The answer jumps from correlation to a cause. The revision defines the revenue model and separates accounting, conversion, retention and mix.

From a weak answer to a defensible one

Weak: The graph dropped after a release, so the release caused it.

Stronger: I would reconcile the event with server outcomes, quantify the affected cohort’s loss, and compare exposed with unexposed versions.

Prepare your own answer

Create an observation table: hypothesis, predicted evidence, actual observation, next check and mitigation.

Close the guide and answer aloud. After reviewing, identify one missing decision and retry on a different problem.

Watch one interview

Google Product Manager Execution Interview: YouTube Watch Time Root Cause Analysis ↗

Exponent · Mock interview · YouTube

Watch with a task: Track which questions distinguish hypotheses. Write the evidence needed before accepting the candidate’s proposed cause.

Practice with a PMMockr worked answer

Open the practice problem and attempt it before comparing.

Compare after your attempt

Read the original worked answer, follow-up and feedback

Compare problem framing, assumptions, rejected options, evidence and response to pushback. Explain what evidence would settle a disagreement.

Turn the reading into practice

Easy: outline your answer and explain every term you use.
Medium: compare two options, defend one and name a downside.
Hard: handle uncertainty, conflicting goals and a changed assumption.

Complete three questions at each level in your selected learning path where available. Review model solutions through the question’s solution action after meeting its access requirements.

Choose a learning path →