Validation demo · simulated data
Experiment Copilot
Turn a supported finding into a testable plan, protect key guardrails and interpret the result with a deterministic Ship / Iterate / Stop rule.
Supported hypothesis
Shorten mobile onboarding Step 3.
If new mobile players complete a shorter Step 3, D1 retention should improve without materially harming monetization or session quality. The evidence supports testing this hypothesis; it does not prove causality.
Control
Current Step 3
5 required actions
Variant A
Short Step 3
3 required actions
Audience
New mobile players
Primary KPI
D1 retention
+10% relative lift
Minimum sample
3,000 players
Runtime
7–14 days
Guardrails
✓ ARPDAUMax −5%
✓ Payer conversionMax −5%
✓ Crash rateMax +2%
Decision rule
Stop when a protected guardrail materially deteriorates. Ship only after the sample threshold is reached and the primary KPI clears the target lift. Otherwise iterate.
Example observed result
Variant A after 3,420 players
D1 lift
+12.4%
ARPDAU
+1.2%
Payer conversion
+0.4%
Crash rate
+0.1%
Copilot interpretation
SHIP
The experiment cleared the +10% D1 target with the minimum sample reached and no guardrail breach. The result supports shipping the shorter Step 3 to the tested mobile audience, followed by continued monitoring after rollout.