Design an experimentation dashboard for Wallet

Problem Statement Description

Product context: Uber is a mobility and delivery platform; its products include rides, Uber Eats, grocery and retail delivery, freight, driver and courier tools, and marketplace pricing.

Uber’s Wallet experience for merchants supports money-related workflows such as viewing balances, receiving payouts, understanding fees, handling adjustments or refunds, and using wallet-linked features across a global mobility, delivery, and logistics marketplace. Product teams may run experiments on this Wallet experience to improve merchant trust, payment clarity, adoption of financial features, and operational efficiency.

Your task is to design an experimentation dashboard for Wallet that helps Uber teams evaluate whether product changes are working as intended. The dashboard should support PMs, data scientists, engineers, finance, and local operations teams in monitoring experiments, comparing variants, diagnosing movement in key metrics, and deciding whether to ship, iterate, or stop an experiment.

Because Wallet touches money movement and merchant trust, the dashboard must go beyond a single success metric. It should make metric definitions, denominators, experiment cohorts, data freshness, instrumentation quality, and guardrail metrics clear enough for confident decision-making across different markets, merchant types, and payment workflows.

The experience should consider:

- The primary users of the dashboard and the decisions each user needs to make during and after an experiment

- Core experiment context, including hypothesis, variants, audience, geography, start/end dates, allocation, and eligibility rules

- Metric definitions with clear denominators, such as eligible merchants, active merchants, payout events, wallet sessions, or transactions

- Wallet-specific success metrics, behavioral metrics, financial metrics, and merchant experience signals

- Guardrails for payment reliability, payout delays, failed transactions, support contacts, refunds, fraud/risk signals, and marketplace health

- Cohort and segmentation views by merchant size, market, tenure, payout method, order volume, and platform usage

- Instrumentation checks, sample size visibility, data latency, event logging gaps, and confidence in experiment readouts

- How the dashboard helps teams interpret results without overreacting to noise, seasonality, or local market anomalies

The goal is to describe what an effective experimentation dashboard for Uber Wallet should include, how it should structure metrics and cohorts, and how it should help teams make reliable product decisions while protecting merchant trust, payment reliability, and Uber’s marketplace operations.