Design an experimentation dashboard for Kids Profiles at global scale

Problem Statement Description

Product context: Netflix is a streaming entertainment company; its products include subscription video, original films and series, recommendations, profiles, games, and ad-supported plans.

Netflix wants to evaluate experiments that affect Kids Profiles across global markets, including experiences that may influence family engagement, content discovery, retention, and reactivation of households with lapsed or at-risk usage. These experiments could involve recommendations, UI changes, parental controls, localization, games, ads-supported plan experiences, or new content surfaces designed specifically for children and family co-viewing contexts.

Your task is to define what an experimentation dashboard should show so product, data science, engineering, and content teams can understand whether changes to Kids Profiles are helping users without creating unintended harm. The dashboard must support global scale, multiple device types, varied market maturity, different household compositions, and the special measurement challenges of kids’ usage, where the account owner, parent, and child may not be the same user.

The focus is not to design the visual UI in detail, but to specify the metrics, cuts, instrumentation, decision logic, and guardrails needed for reliable experiment readouts. The dashboard should help teams compare experiment variants, detect issues early, interpret results by cohort, and decide whether to ship, iterate, or stop an experiment.

The experience should consider:

- Clear primary, secondary, and guardrail metrics for Kids Profiles, including how each metric is defined and what denominator is used.

- Experiment unit and attribution choices, such as household, profile, device, session, or account-level assignment, and how those choices affect interpretation.

- Cohorts and segmentation across geography, language, age range, plan type, device, tenure, household structure, content maturity settings, and lapsed or returning subscriber behavior.

- Instrumentation requirements for key events such as profile entry, search, browse, play start, completion, exits, parental control interactions, errors, and cross-profile behavior.

- Global experimentation concerns, including localization, content availability differences, seasonality, school holidays, network quality, and market-specific competitive behavior.

- Guardrails for child safety, parent trust, streaming quality, excessive friction, inappropriate recommendations, ads suitability where relevant, and negative impact on adult account engagement.

- Statistical and operational readout needs, including sample size, confidence, novelty effects, metric latency, multiple concurrent experiments, anomaly detection, and experiment health checks.

- Decision usefulness for different stakeholders, including product managers, data scientists, executives, content teams, and regional teams.

The goal is to describe a rigorous experimentation dashboard that enables Netflix to make trustworthy product decisions for Kids Profiles at global scale, balancing engagement and retention outcomes with safety, quality, localization, and family trust.