Design an experimentation dashboard for Grocery

Problem Statement Description

Product context: DoorDash is a local commerce and delivery platform; its products include restaurant delivery, DashPass, grocery and retail delivery, merchant tools, and dasher tools.

DoorDash Grocery runs experiments across a three-sided marketplace: consumers ordering groceries, enterprise merchants fulfilling inventory and substitutions, and Dashers completing time-sensitive deliveries. In this interview, you are being asked to define what an experimentation dashboard should provide for Grocery teams so they can evaluate product, pricing, merchandising, fulfillment, and logistics tests with confidence.

The dashboard should be scoped to experiments that may affect enterprise merchant performance, consumer conversion and retention, Dasher experience, and DoorDash marketplace economics. Grocery has unique measurement challenges such as basket size, item availability, substitutions, pickup readiness, delivery reliability, refund rates, and merchant operational variability across stores and regions.

Your task is not to design the visual UI in detail, but to describe the metrics framework, instrumentation, segmentation, and decision support needed for a reliable experimentation dashboard. The interviewer will expect clarity on how teams would know whether an experiment is successful, harmful, inconclusive, or ready for broader rollout.

The experience should consider:

- Primary success metrics, including clear numerators, denominators, and time windows for Grocery experiments.

- Funnel metrics across discovery, cart building, checkout, fulfillment, delivery, and post-order outcomes.

- Merchant-specific metrics such as item availability, substitution handling, order acceptance, prep readiness, cancellations, refunds, and merchant ROI.

- Marketplace guardrails covering delivery reliability, Dasher earnings or utilization, consumer defects, unit economics, and support contacts.

- Cohort and segmentation needs by merchant, store, geography, customer type, order size, delivery speed, and experiment variant.

- Instrumentation requirements to ensure events are trustworthy across app surfaces, merchant systems, dispatch, delivery tracking, and post-order workflows.

- Statistical and operational decision usefulness, including sample size, confidence, experiment duration, novelty effects, and interpreting mixed results.

- Alerting or monitoring for adverse impacts during live experiments, especially for high-volume merchants or operationally sensitive tests.

The goal is to define a dashboard that helps DoorDash Grocery teams make better experiment decisions while balancing consumer experience, enterprise merchant outcomes, Dasher reliability, and marketplace economics.