Design an experimentation dashboard for Wishlists

Problem Statement Description

Product context: Airbnb is a travel marketplace connecting guests and hosts; its products include stays, experiences, host tools, guest booking, trust and safety, payments, and reviews.

Airbnb’s Wishlists help guests save, organize, compare, and revisit travel ideas across stays and experiences. For experience seekers, Wishlists may sit early in the inspiration journey, before dates, destination, budget, group size, or trip intent are fully formed. Product teams may run experiments on Wishlist creation, save flows, collaboration, recommendations, notifications, sharing, or conversion paths from saved items to bookings.

Design an experimentation dashboard that helps Airbnb teams understand whether Wishlist-related experiments are improving the guest journey without creating marketplace or host-side harm. The dashboard should support product managers, data scientists, designers, and engineering teams who need to evaluate experiment health, interpret impact, and make launch, iterate, or stop decisions.

The scope should focus on defining what the dashboard needs to measure, how metrics should be structured, what segments and cohorts matter, and how teams should detect both positive movement and unintended consequences. Consider that Wishlists may influence long-cycle travel planning, cross-device behavior, collaborative planning, and eventual bookings across stays and experiences.

The experience should consider:

- Clear definitions for primary, secondary, diagnostic, and guardrail metrics related to Wishlist usage and downstream booking behavior

- Appropriate denominators, such as visitors exposed to the experiment, searchers, listing viewers, Wishlist creators, savers, collaborators, or bookers

- Instrumentation needs for key events, including impressions, saves, removals, Wishlist creation, sharing, collaboration, revisits, clicks from Wishlist to listing, and booking progression

- Cohorts and segmentation by user type, geography, platform, trip intent, destination type, new versus returning users, and experience-seeker behavior

- Experiment-readiness checks, including sample size, exposure quality, allocation balance, event logging quality, and statistical confidence

- Marketplace and trust guardrails, such as host inquiry quality, cancellation behavior, guest confusion, booking displacement, and supply concentration

- Decision usefulness for stakeholders, including how the dashboard helps compare variants, detect anomalies, understand funnel movement, and support launch decisions

Your goal is to describe a metrics dashboard framework that would allow Airbnb to evaluate Wishlist experiments rigorously and practically, while accounting for the unique role of Wishlists in travel inspiration, planning, collaboration, and conversion.