Measurement uncertainty · Daily supplements · Case study · 3 min read
The dashboard said scale. The bank said stop.
Platform reporting ran hot. The real question was not which campaign to scale. It was whether any of them deserved more money yet. Identifying details are changed.Measurement uncertainty
A checkout change left purchase events firing twice. Reported revenue ran far ahead of settled cash while small orders lost money under free shipping.
Profit per paid order turned positive while the reported return fell. Both facts are part of the story.Situation.
The account
Daily supplements: one hero product carrying most of revenue, and a free-shipping promise set years earlier, when shipping was cheaper. Revenue grew every quarter. Cash tightened every growth month.
What the numbers first suggested.
The first read
The brief was to scale an account whose dashboard showed a healthy return. The bank disagreed. A checkout upgrade had left two purchase events firing on the same orders, so platform reporting ran far ahead of settled revenue.
The account was not underperforming. It was misreporting.
What was actually happening.
The problem
Under the corrected numbers, the hero product lost money on small orders once shipping and returns were counted. The business was confidently buying unprofitable orders at scale. More spend would only have made that bigger.
What pointed there
An event audit traced the inflation to the checkout change. Reconciliation showed reported revenue far above bank deposits, and most hero-product orders sat just under the free-shipping line.
The decision.
The moves
01Hold spend flat until platform and bank agree within a set range. Fix the numbers before spending against them.
02Raise the free-shipping threshold, with a bundle designed to clear it at full price.
The work
- Measurement
Deduplicated events, server-side tracking, and a monthly reconciliation against the bank.
- Offer
Threshold raised and a bundle built to clear it. Average order value did the rest.
- Media
Spend held down on purpose while the numbers were rebuilt, then restored once they passed.
- Retention
Welcome, reorder, and winback flows rebuilt on the corrected tracking.
What we decided against
- Scaling on the broken number. It would have multiplied the reporting error and the loss underneath it.
- Cutting spend immediately. A big move based on a number we did not trust yet.
- Relaunching everything at once. With every variable changing, nobody could say afterward what worked.
What changed.
The outcome
The gap between platform and bank closed to a fraction of what it was. Order value rose enough to make the hero product profitable per order. Revenue dipped during the hold, on purpose and on schedule, then recovered at better quality. The reported return ended lower than when the account arrived, and the business was healthier.
How sure we are
Profit per order comes from a model, and every model contains choices. The rebuilt retention flows were too young to judge, so this study claims nothing for them.
What we would remember next time