Conversion in Android fell after a redesign. How would you investigate

Problem Statement Description

Product context: Google is a consumer technology, ads, AI, and cloud company; its products include Search, YouTube, Android, Maps, Gmail, Chrome, Google Play, Workspace, and Google Cloud. Android is Google's mobile operating system; its product ecosystem includes apps, Play services, notifications, permissions, device settings, OEM integrations, and developer APIs.

Google has recently launched a redesign in an Android developer-facing experience, such as an onboarding, setup, documentation-to-action, or publishing funnel. Soon after launch, the team observes that conversion has fallen on Android. The drop may affect developers trying to complete a key action, such as signing up, configuring a project, downloading tools, accepting policy steps, or progressing through a Play/Android workflow.

Your task is to investigate the decline as an RCA problem, not to jump directly into redesign ideas. Assume the product operates at global scale, across many device types, OS versions, geographies, languages, traffic sources, and developer segments. The investigation should distinguish between a real user-behavior change, a measurement issue, rollout artifact, or an external factor unrelated to the redesign.

The experience should consider:

- Clear definition of “conversion,” including numerator, denominator, funnel entry point, completion event, and whether the metric changed during the redesign.

- Timing of the anomaly, including pre/post windows, rollout percentage, app or web version, experiment cells, and whether the decline aligns with the redesign launch.

- Segmentation by Android version, device class, geography, language, browser/app surface, new vs returning developers, account type, traffic source, and developer maturity.

- Instrumentation checks, including event firing, duplicate or missing logs, schema changes, tracking consent, attribution changes, and backend data delays.

- Funnel-step analysis to identify where users are dropping off and whether the redesign introduced new friction, latency, errors, policy confusion, or compatibility issues.

- Hypotheses that separate UX causes from technical causes, such as broken CTAs, slower load times, layout issues, authentication failures, API errors, or changed ranking/traffic mix.

- Evidence needed to confirm or reject each hypothesis, including logs, dashboards, session traces, experiment data, support tickets, qualitative feedback, and error monitoring.

- Immediate mitigation, rollback criteria, stakeholder communication, and longer-term prevention through monitoring, launch gates, and experimentation discipline.

The goal is to demonstrate a structured investigation plan that can quickly assess impact, isolate root cause, protect developers and the Android ecosystem, and guide an evidence-based decision on whether to rollback, patch, continue rollout, or conduct deeper analysis.