Tokopedia ran a weight-based free-shipping program in Surabaya and Makassar. Its reported performance was the volume of orders that used it — which is how most subsidies justify themselves, and why most subsidies survive longer than they should.
The question nobody could answer was how many of those orders needed the subsidy. A buyer who would have ordered at full shipping cost and instead ordered free is not incremental volume. They are a discount.
Ran the analysis end to end and carried the recommendation — terminate — through to the decision. Recommending the end of a live program is a different task from measuring one, and most of the work was in the second half.
Synthetic control. There is no parallel Surabaya running without the program, so you build one: take cities that never received it, and weight them into a composite that tracks the real city closely in the period before launch.
If the composite tracks well beforehand, its path afterwards is a defensible estimate of what would have happened anyway. The gap between the real city and its synthetic twin after launch is the subsidy’s actual contribution.
Turning off a subsidy always looks like a loss in the first weekly report. The analysis had to be strong enough to survive that, and stated clearly enough that people remembered what it predicted.
The gap between each city and its synthetic twin did not justify the spend.
A good pre-period fit does not guarantee the composite stays valid afterwards. Something specific to Surabaya and Makassar — a competitor’s push, a local shock — would show up as a gap and be misread as subsidy effect. The estimate is only as good as the assumption that nothing else changed in those two cities and not in the donor pool.
The program was terminated in both cities, freeing $XX million in annual subsidy.