Conversion in Marketplace fell after a redesign. How would you investigate

Problem Statement Description

Product context: Meta is a social technology company; its products include Facebook, Instagram, WhatsApp, Messenger, Threads, Quest, creator tools, and ads.

Meta Marketplace has recently launched a redesign, and shortly after the launch, conversion declined. In this context, conversion may relate to the critical Marketplace funnel for families looking to buy, sell, or inquire about local goods, such as discovering listings, opening item details, messaging sellers, saving items, or completing a transaction off-platform. Your task is to investigate the drop as a product RCA, not to jump directly into redesign recommendations.

Assume Marketplace operates at large scale across surfaces, devices, geographies, listing categories, and user segments, with strong dependencies on ranking, search, messaging, trust and safety, notifications, ads, and seller supply. The redesign may have changed visual layout, navigation, listing density, calls to action, filters, messaging entry points, or perceived trust signals. The investigation should distinguish whether the decline reflects a true user behavior change, a measurement issue, rollout artifact, segment-specific regression, or an external factor.

You should frame the anomaly clearly, define the affected conversion metric and funnel stage, validate the data, segment the impact, form hypotheses, identify evidence needed, and outline how you would mitigate user and business impact while continuing the investigation.

The experience should consider:

- The exact conversion definition, numerator, denominator, attribution window, and whether it differs for buyers, sellers, or family-oriented use cases.

- Timing of the decline relative to redesign rollout, experiment exposure, app versions, platform changes, seasonality, and Marketplace traffic mix.

- Funnel segmentation across impressions, listing clicks, detail-page views, saves, messages, seller replies, and completed purchase proxies.

- User and supply cohorts such as new vs. returning users, families, high-intent buyers, casual browsers, sellers, categories, geographies, device types, and acquisition sources.

- Instrumentation checks, including event logging changes, client/server discrepancies, delayed events, bot/spam filtering, and experiment assignment integrity.

- Product hypotheses around discoverability, listing quality, ranking, trust signals, messaging friction, page load performance, notification behavior, and safety interventions.

- Evidence sources such as dashboards, experiment readouts, session replays or qualitative feedback, support tickets, marketplace integrity data, and competitive or external market signals.

- Mitigation and prevention plans, including severity assessment, rollback or holdout decisions, monitoring, communication with cross-functional teams, and follow-up guardrails.

The goal is to demonstrate a structured RCA approach appropriate for a large-scale Meta Marketplace product: isolate where and for whom conversion fell, determine whether the redesign caused the issue, identify the most likely drivers using evidence, and define what actions are needed to protect users, sellers, and marketplace health while preventing similar regressions in future launches.