Retention for marketplace sellers declined in Reels. What is your analysis plan

Problem Statement Description

Product context: Meta is a social technology company; its products include Facebook, Instagram, WhatsApp, Messenger, Threads, Quest, creator tools, and ads.

Meta has observed a decline in retention among marketplace sellers using Reels. These sellers may be individuals, small businesses, or local merchants who use Reels to showcase products, drive discovery, generate buyer inquiries, and convert interest into Marketplace activity. The issue is not simply that overall Reels engagement changed, but that a specific seller segment is returning less often or continuing their seller-related Reels behavior at a lower rate.

Your task is to lay out an analysis plan to diagnose the decline. You should clarify what “retention” means in this context, determine whether the drop is real or measurement-related, identify which seller cohorts or behaviors are affected, and build hypotheses across product, marketplace, content creation, distribution, buyer response, policy, seasonality, and competitive dynamics.

Assume you are working with cross-functional partners across Reels, Marketplace, ranking, notifications, integrity, ads, and data science. The interviewer is looking for a structured RCA approach that moves from anomaly validation to segmentation, hypothesis testing, evidence gathering, mitigation, and prevention.

The analysis should consider:

- How to define the affected seller population and the exact retention metric, including numerator, denominator, time window, and seller activity threshold.

- Whether the decline is broad-based or isolated to specific cohorts, such as new vs. existing sellers, category, geography, seller size, content format, acquisition channel, or device/app version.

- Instrumentation and data-quality checks, including tracking changes, logging gaps, identity mapping between Marketplace and Reels, bot/spam filtering, and metric pipeline changes.

- Product-surface changes that could affect seller behavior, such as Reels creation tools, Marketplace sharing flows, ranking distribution, notifications, messaging entry points, or seller dashboards.

- Marketplace-specific drivers, including buyer demand, inquiry quality, listing conversion, trust/safety enforcement, policy changes, and content moderation impact.

- External and temporal factors such as seasonality, macro commerce trends, competitor creator tools, ad pricing shifts, or changes in seller marketing behavior.

- How to prioritize hypotheses using impact size, timing correlation, affected segments, experiment history, and supporting qualitative or quantitative evidence.

- Mitigation and prevention planning, including short-term monitoring, rollback or containment options, owner alignment, and long-term alerting.

Your goal is to present a clear, rigorous RCA plan that would help Meta determine why marketplace sellers are retaining less in Reels, how serious the issue is, which teams need to act, and what evidence is needed before taking corrective action.