Choosing the Right Metrics
Select primary and guardrail metrics that ensure your experiments generate actionable insights without causing unintended harm.
Primary Metrics: What You're Optimizing
Your primary metric is the single most important outcome you're trying to improve. Every experiment should have exactly one.
Characteristics of Strong Primary Metrics
- Directly tied to business value (revenue, retention, etc.)
- Sensitive enough to detect changes within your timeframe
- Measurable within 2–4 weeks
- Difficult to game
Primary Metrics by Funnel Stage
| Stage | Key Metrics |
|---|---|
| Acquisition | Signup conversion rate, CPA, Landing page conversion |
| Activation | Onboarding completion, Time to first value, Feature adoption |
| Retention | Day-7/Day-30 retention, Churn rate, DAU/WAU |
| Revenue | Free to paid conversion, ARPU, LTV |
| Referral | Invite send rate, Invite acceptance rate, Viral coefficient |
Guardrail Metrics: Protecting Against Harm
Every experiment should define 2–4 guardrail metrics to ensure you're not causing collateral damage.
Performance: Page load time, API response times Quality: Error rates, support ticket volume Satisfaction: NPS, CSAT, unsubscribe rates Business Value: Revenue per user, LTV
Using the AI Metric Validator
The AI Metric Validator automatically checks your chosen metrics for common pitfalls:
- Vanity metrics that look impressive but don't drive business value
- Lagging indicators that take too long to measure
- Confounding variables that could distort results
- Validity issues where the metric doesn't measure what you think
It also suggests alternative primary metrics, secondary metrics, and guardrail metrics you may not have considered, along with sample size and duration recommendations.
⚠️ Warning: Composite metrics (combining multiple factors) make interpretation unclear. Prefer single, direct metrics for your primary measurement.
