Design an experimentation dashboard for Grocery at global scale

Problem Statement Description

Product context: DoorDash is a local commerce and delivery platform; its products include restaurant delivery, DashPass, grocery and retail delivery, merchant tools, and dasher tools.

DoorDash Grocery teams run experiments across a complex three-sided marketplace: consumers ordering groceries, enterprise merchants fulfilling inventory from stores, and Dashers completing time-sensitive deliveries. At global scale, experiments may vary by country, market, merchant chain, store format, catalog quality, fulfillment model, delivery distance, and demand patterns, making it difficult to interpret whether a product change is truly improving the business or simply shifting friction to another participant.

Design an experimentation dashboard for Grocery that helps product, analytics, operations, and merchant-facing teams monitor experiments, understand impact, and make launch decisions. The dashboard should support tests involving consumer shopping experience, substitution flows, pricing and promotions, merchant operations, delivery reliability, and marketplace efficiency, while accounting for the realities of enterprise grocery merchants and local logistics.

This is a metrics-focused question. Your task is not to design the experiment itself or recommend a specific product feature, but to define what the dashboard should measure, how metrics should be structured, how results should be segmented, and how teams should use the dashboard to make reliable decisions.

The experience should consider:

- Clear primary, secondary, and guardrail metrics for Grocery experiments, including consumer conversion, order quality, delivery reliability, merchant outcomes, Dasher impact, and unit economics.

- Precise metric definitions, denominators, attribution windows, and handling of edge cases such as substitutions, cancellations, refunds, stockouts, late deliveries, and partial fulfillment.

- Instrumentation requirements across consumer app events, merchant systems, store operations, Dasher logistics, payments, promotions, support contacts, and post-order quality signals.

- Cohort and segmentation needs by geography, merchant, store, basket size, order type, fulfillment model, new versus repeat customers, delivery distance, and time of day.

- Experiment health checks such as sample ratio mismatch, exposure logging, data freshness, statistical confidence, seasonality, spillover, and marketplace interference.

- Guardrails that prevent optimizing one side of the marketplace at the expense of another, such as improving conversion while worsening merchant workload, Dasher earnings, or refund rates.

- Decision usefulness for global rollout, including how teams compare treatment performance across markets, identify heterogeneous effects, and determine whether results are actionable.

- Dashboard usability for different stakeholders, from PMs and data scientists to operations leaders and enterprise merchant account teams.

The goal is to describe a dashboard that enables DoorDash Grocery teams to evaluate experiments consistently, catch misleading signals early, and make confident rollout, iteration, or rollback decisions in a global marketplace environment.