Build a metric tree for Ads after a major redesign

Problem Statement Description

Product context: Spotify is an audio streaming company; its products include music, podcasts, audiobooks, playlists, recommendations, creator tools, subscriptions, and ads.

Spotify has recently completed a major redesign of its Ads experience for podcast listeners. The redesign may affect how ads are selected, inserted, heard, skipped, remembered, and monetized across podcast sessions, while also impacting advertisers, podcast creators, and Spotify’s listener experience.

You are asked to build a metric tree for Ads that helps Spotify understand whether the redesign is working. The focus is on podcast fans, but the tree should acknowledge the broader ads marketplace: listeners who experience ads, advertisers who pay for outcomes, and creators whose content is monetized through those ads.

Your metric tree should be useful for product decision-making after launch, not just a list of metrics. It should clarify what success means, how metrics connect to business and user outcomes, where denominators matter, and which guardrails would prevent Spotify from over-optimizing revenue at the expense of listener retention or creator trust.

The metric tree should consider:

- The primary success metric for the redesigned podcast ads experience and why it represents meaningful value.

- Clear metric definitions, including numerators, denominators, and the unit of analysis such as ad impression, session, listener, show, advertiser, or campaign.

- Listener-side cohorts such as free users, premium users exposed to podcast ads, new podcast listeners, heavy podcast fans, geography, device type, and content category.

- Advertiser-side outcomes such as delivery, reach, engagement, conversion quality, budget utilization, and repeat demand.

- Creator and marketplace health, including monetization coverage, fill rates, revenue distribution, and impact on podcast consumption.

- Instrumentation needs across ad request, ad decisioning, impression delivery, playback completion, skip or drop-off behavior, and downstream advertiser events.

- Guardrail metrics for listener satisfaction, retention, podcast listening time, ad load, latency, reporting accuracy, privacy compliance, and ad relevance.

- How the metric tree would support decisions such as rollout expansion, format iteration, targeting changes, pricing adjustments, or rollback.

The goal is to demonstrate how you would structure a practical measurement system for Spotify Ads after a redesign, showing the relationship between user experience, advertiser value, creator monetization, and Spotify’s long-term audio ads business.