Design an experiment to measure whether Terminal improved outcomes for developers

Problem Statement Description

Product context: Stripe is financial infrastructure for internet businesses; its products include payments, Checkout, Billing, Connect, Radar, Issuing, Terminal, and tax tools.

Stripe Terminal lets businesses accept in-person payments using Stripe’s hardware, SDKs, APIs, and integration tooling. In this interview, you are asked to design an experiment to determine whether Terminal has improved outcomes for developers who build and maintain these integrations for merchants.

The focus is not simply whether Terminal usage increased, but whether the developer experience and developer-facing outcomes meaningfully improved. You should think through what “better outcomes” means for developers working with Terminal: getting started, integrating successfully, debugging issues, launching payment flows, maintaining reliability, and supporting merchant growth across physical payment environments.

Your experiment should be framed as a metrics and measurement problem in a Stripe context, where payments reliability, compliance, conversion, and trust matter. The interviewer is looking for how you define success, structure the experiment, instrument the journey, interpret results, and avoid misleading conclusions.

The experience should consider:

- Which developer population is in scope, such as new Terminal integrators, existing Stripe developers adopting Terminal, platform developers, or merchants’ internal engineering teams

- How to define the primary outcome metric, including the numerator, denominator, time window, and what counts as a successful developer outcome

- How to instrument key developer journey steps, such as documentation visits, SDK setup, API calls, reader registration, test transactions, first live transaction, and production stability

- How to choose cohorts or experiment groups while accounting for merchant size, region, platform type, integration complexity, hardware availability, and prior Stripe experience

- What guardrail metrics are needed for payment reliability, failed transactions, support burden, fraud/compliance risk, merchant conversion, and developer frustration

- How to distinguish developer experience improvements from external factors such as sales assistance, documentation changes, hardware shipping delays, seasonality, or merchant demand

- How long the experiment should run and what decision thresholds would make the result useful for product, engineering, and go-to-market teams

The goal is to describe a rigorous experiment that would help Stripe decide whether Terminal is genuinely improving developer outcomes, how confidently that impact can be measured, and what product decisions the results should inform.