Design an experimentation dashboard for Robotaxi

Problem Statement Description

Product context: Tesla is an electric vehicle, energy, and software company; its products include EVs, charging, vehicle software, Autopilot/FSD features, energy storage, and solar products.

Tesla is exploring Robotaxi as an autonomous ride-hailing product that combines vehicles, autonomy software, charging, fleet operations, and rider experiences. In this interview, you are asked to design an experimentation dashboard that would help teams understand whether product, pricing, routing, charging, safety, marketplace, or operator-facing changes are improving the Robotaxi business and customer experience.

Assume multiple stakeholders may use the dashboard: autonomy and safety teams, marketplace/product managers, fleet operators, charging and energy teams, city launch teams, and executives. Experiments may affect different parts of the workflow, such as rider request conversion, pickup reliability, trip completion, autonomous intervention rates, charging availability, vehicle utilization, operator profitability, and rider trust.

Your task is to define what the dashboard should measure, how experiments should be structured and read, and how the dashboard should support decision-making without creating misleading conclusions. Focus on metric clarity, denominators, cohorts, instrumentation, experiment health, guardrails, and how teams would use the dashboard to decide whether to ship, iterate, or stop an experiment.

The experience should consider:

- The primary users of the dashboard and the decisions each user needs to make from experiment results.

- Core success metrics for Robotaxi experiments, including clear numerators, denominators, and time windows.

- Safety, reliability, and customer-experience guardrails that must not regress during experimentation.

- Cohort and segmentation needs, such as city, vehicle model, autonomy software version, time of day, trip type, charging state, rider type, and operator fleet.

- Instrumentation requirements across app events, vehicle telemetry, autonomous driving events, charging sessions, cancellations, support contacts, and completed trips.

- Experiment validity indicators, including sample size, exposure definition, randomization quality, data freshness, and statistical confidence.

- How the dashboard should distinguish marketplace effects from vehicle, autonomy, charging, or operational effects.

- How results should be presented so teams can compare experiments, diagnose issues, and make launch or rollback decisions.

The goal is to describe a practical experimentation dashboard for Tesla Robotaxi that helps teams evaluate product changes rigorously, protect safety and trust, and make faster, evidence-based decisions across a complex autonomous ride-hailing ecosystem.