Define the North Star metric for Copilot serving developers

Problem Statement Description

Product context: Microsoft is a productivity, software, AI, gaming, and cloud company; its products include Windows, Microsoft 365, Teams, LinkedIn, Xbox, Azure, Dynamics, and Copilot.

Microsoft wants to evaluate whether Copilot is creating sustained value for developers across their software-development workflow, from understanding requirements and writing code to debugging, reviewing, testing, and shipping. Your task is to define a North Star metric for Copilot serving developers that captures meaningful productivity improvement without reducing the product to superficial activity or usage volume.

Assume the product is used by individual developers and teams inside professional engineering environments, including enterprise contexts where trust, security, code quality, compliance, and integration with existing tools matter. The metric should be useful to product, engineering, and business leaders when deciding whether Copilot is improving developer outcomes and where to invest next.

You should be explicit about what the metric measures, who is included, how it is counted, and why it reflects durable user value. Avoid relying only on vanity signals such as questions sent, suggestions shown, or raw active users unless you connect them to a clear developer outcome.

The experience should consider:

- The core developer workflow Copilot supports, such as code generation, code explanation, debugging, test creation, documentation, and review assistance.

- The exact metric definition, including numerator, denominator, time window, and qualifying user or team population.

- How the metric would be instrumented across IDEs, repositories, pull requests, CI/CD systems, and developer feedback surfaces.

- Relevant cohorts, such as individual developers vs. teams, new vs. retained users, enterprise vs. smaller teams, language/framework segments, and novice vs. experienced developers.

- Guardrail metrics for code quality, security, reliability, review burden, developer trust, and misuse or over-automation.

- How to distinguish real productivity gains from increased activity, copy-paste behavior, or low-quality accepted suggestions.

- How the metric would support decision-making for product roadmap, adoption health, monetization, and enterprise trust.

- Any limitations, trade-offs, or risks in the chosen metric and how you would monitor them.

The goal is to define a clear, practical North Star metric that reflects Copilot’s value to developers and Microsoft while remaining measurable, actionable, and safe to optimize.