Design an experimentation dashboard for Social Listening

Problem Statement Description

Product context: Spotify is an audio streaming company; its products include music, podcasts, audiobooks, playlists, recommendations, creator tools, subscriptions, and ads.

Spotify is exploring Social Listening experiences for audiobook listeners, such as discovering, sharing, discussing, or listening around audiobooks with friends, book clubs, or communities. Product, data science, design, and engineering teams need an experimentation dashboard that helps them evaluate whether these experiences create meaningful value without harming the core listening experience.

In this metrics interview, you are not being asked to design the Social Listening feature itself. Your task is to define what the dashboard should measure and expose so teams can understand experiment performance, diagnose user journey friction, and make launch, iterate, or rollback decisions across markets, platforms, and listener segments.

The dashboard should support experiments across surfaces such as audiobook detail pages, playback flows, recommendations, sharing questions, notifications, and social entry points. It should be useful for comparing variants, understanding cohort behavior, and separating true product impact from tracking or experiment-quality issues.

The experience should consider:

- The primary experiment objective and how success should be defined for Social Listening among audiobook listeners.

- Metric definitions with clear numerators, denominators, attribution windows, and eligibility criteria.

- Funnel instrumentation from exposure to entry, share or invite, session join, audiobook playback, completion, and repeat use.

- Cohort breakdowns such as new vs. existing audiobook listeners, subscription state, geography, platform, listening frequency, and social graph density.

- Guardrail metrics for retention, listening satisfaction, playback disruption, notification fatigue, privacy controls, spam or abuse, and cannibalization of other Spotify listening.

- Experiment health checks such as randomization quality, sample ratio mismatch, event completeness, latency, and logging consistency.

- Decision usefulness, including how dashboard views should support launch decisions, deeper diagnosis, and comparison across experiment variants.

The goal is to describe a dashboard that gives Spotify teams confidence in whether Social Listening improves audiobook discovery, engagement, and retention while preserving user trust and the quality of the broader audio experience.