Conversion in Android fell after a redesign. How would you investigate

Problem Statement Description

Product context: Google is a consumer technology, ads, AI, and cloud company; its products include Search, YouTube, Android, Maps, Gmail, Chrome, Google Play, Workspace, and Google Cloud. Android is Google's mobile operating system; its product ecosystem includes apps, Play services, notifications, permissions, device settings, OEM integrations, and developer APIs.

You are the PM responsible for a developer-facing Android experience at Google, such as a redesigned onboarding, setup, documentation, or publishing flow used by Android developers globally. Shortly after the redesign launched, the team observes that conversion has fallen compared with the prior experience.

Your task is to investigate the drop in a structured RCA interview setting. Assume the redesign may have affected UI, navigation, messaging, eligibility rules, platform behavior, or measurement. You should focus on understanding whether the decline is real, where it is happening, which users are affected, and what evidence would support or rule out likely causes.

The investigation should consider:

- How “conversion” is defined, including numerator, denominator, funnel start, funnel completion, and time window

- Whether instrumentation, event naming, tracking logic, or analytics pipelines changed during the redesign

- Segmentation by Android version, device type, geography, language, developer type, traffic source, account status, and app category

- Funnel-step analysis to identify where the largest drop-off emerged after launch

- Comparisons against control groups, pre-redesign baselines, staged rollout cohorts, and unaffected platforms or surfaces

- Hypotheses across UX friction, performance regressions, crashes, compatibility issues, policy or eligibility changes, copy changes, and backend dependencies

- Evidence needed to prioritize mitigation, such as logs, session replays, support tickets, crash reports, latency data, and qualitative feedback

- Prevention mechanisms for future redesign launches, including alerts, experiment design, rollout gates, and monitoring ownership

The goal is to demonstrate how you would separate a true product regression from a measurement artifact, localize the issue, evaluate competing hypotheses, and guide the team toward a data-backed mitigation path without jumping directly to a redesign reversal.