Design an experiment to measure whether Terminal improved outcomes for developers

Problem Statement Description

Product context: Stripe is financial infrastructure for internet businesses; its products include payments, Checkout, Billing, Connect, Radar, Issuing, Terminal, and tax tools.

Stripe Terminal helps businesses build in-person payment experiences using Stripe’s hardware, SDKs, APIs, and developer tooling. In this interview, you are asked to design an experiment to determine whether Terminal has improved outcomes for developers who integrate, test, and launch in-person payments.

Focus on the developer workflow: discovering Terminal, reading docs, ordering or simulating hardware, integrating SDKs/APIs, handling test payments, debugging errors, completing certification or compliance steps, and moving merchants into production. The experiment should distinguish between developer activity that looks good on the surface and outcomes that meaningfully improve developer success and merchant launch readiness.

This is a metrics question, so your task is to define what “improved outcomes” means, how you would measure it, how you would structure the experiment, and how Stripe should interpret the results. You should be explicit about denominators, cohorts, instrumentation, guardrails, and how the findings would support a product decision.

The experience should consider:

- The primary developer outcome metric, including who is counted in the denominator and what qualifies as success.

- Funnel stages from first Terminal intent through successful live payment processing.

- Instrumentation across docs, dashboard actions, SDK/API usage, test transactions, hardware setup, errors, and production activation.

- Cohorts such as new vs. existing Stripe developers, platform developers vs. direct merchants, geography, integration type, and hardware or reader model.

- Experiment design choices such as randomization unit, control/treatment exposure, duration, sample size needs, and contamination risks.

- Guardrail metrics for payment reliability, error rates, support contacts, compliance issues, merchant conversion, and developer satisfaction.

- Decision usefulness: how results would indicate whether Terminal improved developer outcomes enough to justify rollout, iteration, or further diagnosis.

The goal is to design a clear and credible measurement approach that helps Stripe understand whether Terminal meaningfully improves developer success, not just whether usage increased.