Define success metrics for mobile calendar assistant serving new users

Problem Statement Description

You are evaluating a mobile calendar assistant experience for new users. The assistant may help users connect calendars, grant permissions, create or modify events, find available times, add invitees, handle reminders, or complete other first-session scheduling workflows. The main business goal is workflow completion: whether new users can successfully finish the intended calendar task without confusion, abandonment, or unnecessary manual effort.

This is a metrics interview question. Your task is to define a practical measurement framework that shows whether the product is helping new users complete their first meaningful workflow and whether that success is healthy, reliable, and scalable. The scope should include how you would define the core success metric, what the denominator should be, how you would instrument the user journey, and how you would interpret results across different new-user cohorts.

The experience should consider:

- What counts as a “new user” and what time window should be used to evaluate their first calendar-assistant workflow.

- How to define workflow start, workflow completion, partial completion, abandonment, and retry behavior.

- The correct denominator for completion rate, including users exposed to the assistant versus users who intentionally begin a workflow.

- Instrumentation needed across permissions, account linking, assistant questions, calendar actions, confirmations, errors, and user edits.

- Cohorts such as mobile OS, acquisition source, calendar provider, locale, accessibility needs, and first-use intent.

- Guardrail metrics for incorrect event creation, privacy concerns, notification fatigue, latency, crashes, user trust, and manual undo behavior.

- How the metrics would support product decisions, such as onboarding changes, assistant quality improvements, or rollout expansion.

The goal is to propose a metrics approach that would help a product team confidently understand whether the mobile calendar assistant is delivering successful first-use outcomes for new users while protecting trust, reliability, and long-term adoption.