Design an experimentation dashboard for Wishlists at global scale

Problem Statement Description

Product context: Airbnb is a travel marketplace connecting guests and hosts; its products include stays, experiences, host tools, guest booking, trust and safety, payments, and reviews.

Airbnb’s Wishlists help guests and experience seekers save, organize, compare, and share places or activities they may book later. Product teams may run many experiments on Wishlist creation, saving behavior, collaboration, sharing, recommendations, notifications, and downstream booking conversion across global markets. You are asked to design an experimentation dashboard that helps teams evaluate these tests consistently and confidently at scale.

The dashboard must support decision-making in a global travel marketplace where user intent can be long-cycle, seasonal, multi-device, and influenced by destination, trip type, supply availability, trust, price, and local market dynamics. It should help teams understand whether Wishlist-related changes improve inspiration and planning without harming booking quality, host outcomes, marketplace trust, or user experience.

Focus on defining what the dashboard should measure, how metrics should be structured, what cuts and cohorts are necessary, how experimentation results should be interpreted, and what instrumentation or guardrails are required. Do not assume a single success metric is enough; the dashboard should make trade-offs visible for product, data science, engineering, design, and marketplace stakeholders.

The experience should consider:

- Clear definitions for primary, secondary, diagnostic, and guardrail metrics for Wishlist experiments.

- Appropriate denominators, attribution windows, and event definitions for saves, Wishlist creation, engagement, sharing, collaboration, and downstream bookings.

- Cohorts and segmentation by geography, traveler type, platform, trip intent, new vs. returning users, stay vs. experience interest, and supply characteristics.

- Instrumentation requirements to ensure experiment exposure, user actions, ranking surfaces, notifications, and booking outcomes are captured reliably.

- Statistical interpretation needs, including confidence, sample size, seasonality, novelty effects, and long consideration cycles.

- Marketplace and trust guardrails, such as host supply health, booking quality, cancellations, disputes, pricing sensitivity, and user dissatisfaction.

- Dashboard usability for experiment owners, including result summaries, drill-downs, anomaly flags, and decision-ready views.

- Global scale constraints, including localization, privacy, data latency, experiment overlap, and consistency across web and mobile surfaces.

The goal is to describe a metrics dashboard that enables Airbnb teams to judge whether Wishlist experiments create meaningful user and marketplace value, identify where effects differ across segments, and make informed launch, iterate, or rollback decisions.