Diagnose a sudden drop in conversion for content recommendation feed

Problem Statement Description

You are the product manager for a content recommendation feed used by on-call engineers to discover and act on relevant operational content, such as incident runbooks, alerts context, postmortems, troubleshooting guides, or recommended next steps. The feed has a defined conversion event, such as opening a recommended item, saving it, launching a workflow, or completing a recommended action. A sudden drop in conversion has been detected among on-call engineers.

Your task is to diagnose the issue before recommending fixes. Focus on how you would frame the anomaly, validate whether it is real, isolate where in the user journey the drop occurred, and form evidence-based hypotheses across product, data, ranking, user behavior, and operational factors.

This is an RCA-style interview question. You are not expected to jump to solutions immediately. The interviewer is looking for a structured investigation that distinguishes measurement issues from real user impact, narrows the affected surface area, and identifies what evidence would increase or decrease confidence in each hypothesis.

The experience should consider:

- The exact definition of “conversion,” including numerator, denominator, time window, and whether the event is tied to impressions, sessions, users, or recommendations.

- Segmentation by on-call cohort, team, geography, device, surface, alert severity, incident state, tenure, and traffic source.

- Funnel breakdown from feed exposure to recommendation impression, click/open, engagement, and final conversion.

- Instrumentation checks for logging gaps, schema changes, delayed pipelines, duplicate events, bot/internal traffic, and experiment assignment issues.

- Recent changes to ranking models, content supply, permissions, personalization, UI, notification triggers, or incident-management integrations.

- External context such as incident volume, seasonality, work schedules, paging load, or changes in engineer behavior during outages.

- Evidence needed to prioritize hypotheses, including dashboards, logs, experiment data, qualitative feedback, and comparison against unaffected segments.

- Immediate mitigation and prevention considerations once the root cause is understood, including monitoring, alerting, and ownership.

The goal is to show how you would lead a calm, rigorous diagnosis of a sudden conversion drop in a high-trust operational workflow, separating symptoms from root causes and ensuring any eventual fix is based on reliable evidence rather than assumptions.