Define success metrics for marketplace quality score serving retail staff

Problem Statement Description

You are evaluating a marketplace quality score experience used by retail staff. The score is intended to help frontline teams understand marketplace standards, identify areas where quality is below expectation, and complete the right learning content so they can improve day-to-day marketplace execution.

The interview asks you to define success metrics for this score, with learning completion as the primary business goal. Treat this as a metrics design problem: clarify what the product is trying to change, who is included in measurement, how completion should be counted, and how the score’s usefulness should be assessed without relying on vanity metrics.

Your answer should cover both the direct learning outcome and the surrounding product/system health needed to trust the quality score. Consider that retail staff may work across stores, shifts, roles, regions, and varying levels of marketplace responsibility, so metric definitions and cohorts matter.

The experience should consider:

- The primary success metric for learning completion, including the exact numerator, denominator, and completion window.

- How to distinguish assigned, started, in-progress, completed, overdue, and repeated learning activity.

- Cohorts such as role type, store, region, tenure, shift pattern, marketplace category, and baseline quality score.

- Instrumentation needed to connect quality score exposure, recommended learning, staff engagement, and completion events.

- Guardrail metrics for learning quality, staff burden, content relevance, score accuracy, and operational disruption.

- How to evaluate whether the marketplace quality score is actionable rather than merely visible.

- How metric trends would inform product decisions, experiments, prioritization, or rollout.

The goal is to present a clear metrics framework that lets a product team determine whether the marketplace quality score is helping retail staff complete the right learning in a measurable, trustworthy, and operationally useful way.