Loyalty Program Cost Analysis

The claim, and what survives a control group

Every row below is computed by the same function. The only thing that changes is which members are treated as comparable to which.

What the program costs, against what it is billed at

The licence arrives as an invoice, so it is the number everybody knows. The discount arrives as revenue that never showed up, so it is the number nobody sees. Both are money. Only one of them is on a renewal form.

PLATFORM LICENCE — INVOICED

$7,800 14.4% of the true cost

DISCOUNT HANDED BACK — OFF MARGIN

$46,335 3,089 redemptions × $15

Annual run rate $54,135 — the program as configured.

“Only a few percent redeem”

A participation rate is a division, and the argument is always about the bottom half of it. 14,390 of these members hold zero points. They cannot redeem — not rarely, not reluctantly; there is no mechanism by which they could. Counting them answers a question nobody asked.

5.2%

1,382 ÷ 26,500 ENROLLED MEMBERS

Neither number is wrong. They answer different questions, and only one of them is the question “is anybody using this”. A program whose problem is that half its members never bought anything has an acquisition problem, and no change to the rewards will touch it.

The four terms, sent before any of this was run

The vendor offered to run a before-and-after study. These four terms went back first, in writing, while nobody had seen a result. This panel has no controls and that is deliberate: a study design a visitor can adjust after seeing the outcome is the exact failure the terms exist to prevent.

1. TERM 1

   Match the control group on pre-period order count and spend — not redeemer against non-redeemer.

   Redeeming requires points, and points require prior spending. Comparing redeemers with everyone else compares heavy buyers with light ones and calls the difference a program effect.

2. TERM 2

   Equal window lengths either side of each member's first redemption.

   Unequal windows let the measured period be chosen after the fact. Anchoring on each member's own first redemption also stops a calendar quarter's seasonality from being read as an effect.

3. TERM 3

   Report revenue per customer, not order counts.

   A reward is a discount, and a discount reliably splits baskets. Order counts can rise while revenue per customer falls, and only one of those two numbers is the business question.

4. TERM 4

   Publish the cohort definitions and the group sizes.

   Every claim above depends on who ended up in which group. A result whose denominators are not stated cannot be checked by anybody, including the person who produced it.

The daily sync that produced the data

Five stages, no third-party dependencies, one cron entry. Open a stage for what it does and whether it could honestly be shown here.

1. ›1 · PullThree resources, cursor-paginated at 250 records a page.

   Customers, orders and points activity, each walked page by page with retry and backoff on 429, 500, 502, 503 and 504, and a deliberate pause between pages so a background job never becomes the reason a storefront's API budget runs out. Standard library only — no HTTP client, no dataframe, nothing to keep patched.

   Not runnable here, and not mocked here. It is an authenticated call to a third party's production API.

2. ›2 · CommitWritten to a .partial file, renamed only when the pull completes.

3. ›3 · ExportOne pass over the customer file, accumulating into a member export.

4. ›4 · PruneSnapshots older than fourteen days are deleted — except month-starts, kept forever.

5. ›5 · DashboardA static HTML page, generated, with member emails masked.

What is established, what runs here, and what nobody knows

The three are not interchangeable, and a page that ran a convincing simulation without saying so would be inviting you to read the third as the first.

• ESTABLISHED

  The vendor's own headline figures were reproducible from raw data.

  In the real project, independently pulled records reproduced the vendor's redemption counts, discount total and redeeming-customer count closely enough that the data was never in dispute. Only the interpretation was. Those figures are not shown here.

• ESTABLISHED

  A redeemer-versus-non-redeemer comparison is confounded by prior spend.

  This is a property of the comparison, not of any one program: eligibility to redeem is caused by prior spending. It holds wherever points are earned by buying.

• RUNS HERE

  Matching on pre-period behaviour collapses the advantage on THIS page's data.

  Computed live, in your browser, on a cohort generated with no treatment effect at all. It demonstrates that the method works. It is not evidence about the real program.

• RUNS HERE

  The licence is a small fraction of what a loyalty program costs.

  The arithmetic on this page is live and the structure is the real one — an invoiced licence plus discount handed back off margin. The dollar amounts are invented.

• ESTABLISHED

  Participation rates quoted over all members are computed over a base that cannot participate.

  Members holding zero points cannot redeem under any circumstances. Whether they belong in the denominator is a question with an answer, and it is no.

• NOT KNOWN

  The real program does or does not cause incremental revenue.

  Nothing observational settles this, including everything above. That is why the recommendation was a 90-day holdout — withhold the program from a random tenth of members and measure. Until that runs, the honest answer is that nobody knows.

• DESCRIBED ONLY

  The daily sync, the platform API and the generated dashboard.

  Described, not executed. Every stage of it is an authenticated call to a third party or a write to a disk, and neither exists on this page.

Why the real program is not simulated here

A page that replayed the actual program’s members, redemptions and revenue would be the most persuasive version of this and the least honest. Those figures, together, describe one retailer’s unit economics and one vendor’s commercial terms; publishing them re-synthesised would be a guess dressed as a record, and publishing them unchanged would be someone else’s confidential data on a portfolio.

There is a stronger reason. The finding was that an observational comparison cannot answer this question — so a demo that showed the real numbers being compared observationally would be reproducing the error the project exists to correct. A cohort built with a known answer can be checked against that answer. Real data cannot, which is the entire argument for running the holdout.