Retention for developers declined in Prime Video. What is your analysis plan

Problem Statement Description

Product context: Amazon is a commerce, logistics, media, devices, and cloud company; its products include Marketplace, Prime, Prime Video, Alexa devices, ads, fulfillment, and AWS. Prime Video is Amazon's streaming video service; its products include movies, series, live sports, rentals, channels, recommendations, downloads, and ad-supported viewing.

Prime Video has observed a meaningful decline in retention among developers who work with its ecosystem, such as partners or engineering teams using Prime Video APIs, SDKs, tooling, documentation, test environments, or content/workflow integration surfaces. The interview asks you to outline an analysis plan for diagnosing the decline, not to jump directly to a fix.

Frame the problem as an RCA for a developer-facing product surface within a large-scale media platform. These developers may be building, maintaining, or integrating experiences that affect content availability, playback, device support, publishing workflows, or operational tooling. A retention drop could reflect real dissatisfaction, workflow friction, technical reliability issues, changes in measurement, or shifts in developer mix.

Your response should show how you would clarify the metric, validate the data, segment the population, develop hypotheses, gather evidence, and decide what actions are needed. The analysis should account for Amazon-scale complexity: multiple device platforms, global markets, partner types, release cycles, operational dependencies, and high expectations for reliability.

The analysis plan should consider:

- How “developer retention” is defined, including the active developer action, time window, denominator, returning cohort, and whether retention is measured by account, user, team, partner, app, or integration.

- The anomaly framing: when the decline started, how large it is, whether it is sudden or gradual, and whether it differs from historical seasonality or expected release-cycle behavior.

- Instrumentation checks, including event logging changes, identity stitching, bot/test traffic, API key changes, dashboard logic, data freshness, and missing telemetry.

- Segmentation by developer type, geography, device platform, API/SDK version, partner tier, lifecycle stage, content workflow, acquisition channel, and tenure cohort.

- Hypotheses across product experience, documentation quality, onboarding friction, API reliability, sandbox or certification issues, support response times, policy changes, pricing/contract changes, and competing developer priorities.

- Evidence needed to confirm or reject hypotheses, such as funnel drop-offs, support tickets, latency/error rates, release notes, incident history, qualitative developer feedback, and cohort comparisons.

- Near-term mitigation and prevention thinking, including impact sizing, prioritization, ownership, communication with affected developers, and monitoring after any corrective action.

The goal is to present a rigorous, structured RCA plan that separates measurement artifacts from real behavioral decline, identifies the highest-likelihood drivers, quantifies business and developer impact, and creates a path toward evidence-based mitigation and long-term prevention.