docs(sales-summaries): validate the balance fixes against a restored backup

Measures R1 (reversed tips) and R2 (Square service charges) over 3,342
client-days of real production data, restored from backup point 209608347
(newest transaction 2026-08-14 22:52).

R2 clears 307 client-days with zero regressions, which is the gate the plan set
for it. R1 clears 78 with 2 regressions, both days that balanced only because
two errors cancelled, and both on clients that share a Square location.

After both fixes, 98.6% of the remaining dollar error sits in the 10 contended
client pairs — Phase 0's to fix, not a calculation defect. Outside those, the
entire residual is $73.98, of which all but six client-days is sub-10c register
rounding. The three predicted material days reproduce to the cent.

Also records an amendment to Phase 1: 2,122,161 charges (12.8%) carry neither
:charge/client nor :charge/location, so the re-key cannot scope them from the
entity. They are payout-path stubs and order-referenced tenders; none are
orphaned, so scoping must come from the referencing order or expected-deposit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-15 00:23:38 -07:00
parent 9e4cb851ff
commit b4b68b36cf

View File

@@ -0,0 +1,170 @@
---
title: Sales Summary Balance — validation against a restored production backup
type: validation
date: 2026-08-15
status: measured
---
# Sales Summary Balance — validation against a restored production backup
Measures the two calculation fixes (R1 reversed tips, R2 Square service charges) against real
production data, and checks them against the predictions in
[`docs/plans/2026-08-13-fix-sales-summary-balance-plan.md`](plans/2026-08-13-fix-sales-summary-balance-plan.md).
**Headline: the fixes behave as predicted.** R2 clears 307 client-days with zero regressions.
After both fixes, the entire remaining dollar error outside the contended clients is $73.98,
and all but six client-days of it is sub-10¢ register rounding.
---
## 1. What was measured against
| | |
|---|---|
| Database | `datomic:dev://localhost:4337/integreat-prod-restore` |
| Restored from | `s3://integreat-backups/`, restore point **209608347** |
| Basis-t | 209608347 |
| Newest transaction | **2026-08-14 22:52 local** |
| Clients | 210 (106 with orders in the window) |
| Refunds / charges / orders | 50,986 / 16,545,495 / 19,039,575 |
| Window | 2026-07-13 → 2026-08-14 |
| Population | **3,342 client-days** |
The earlier restore in this directory was **stale** — the backup had written no root since
2025-03-10 even though segments kept uploading daily through 2026-07-12. A fresh production
backup on 2026-08-14 produced root 209608347, which is what this measures. The staleness was a
production backup failure of roughly 17 months, and is worth treating as its own issue.
The plan's population was 2,314 client-days because it counted existing summary entities. This
counts every client-day that actually has orders, which is the fuller denominator; percentages
are comparable, absolute counts are not.
## 2. Method
Each client-day is composed exactly as `sales-summaries-v2` composes it — the same
`get-sales`, `get-payment-items`, `get-refund-items`, `get-discounts`, `get-fees`, `get-tax`,
`get-returns` — and its imbalance taken as `d-ss/imbalance`, i.e. debits minus credits. Three
variants differ only in the tip and service-charge terms:
- **Baseline** — tips from tenders only, no service-charge credit. What production does today.
- **R1** — tips additive: tendered tips plus tips on orders carrying no tender.
- **R2** — R1 plus Square service charges, both signs.
Nothing was transacted. A day counts as off when `|imbalance| >= $0.005`, i.e. it does not
round to zero.
## 3. Results
| Stage | Days off | Clean | Total \|Δ\| |
|---|---|---|---|
| Baseline (production today) | 569 | 82.97% | $32,921.40 |
| + R1 additive tip | 493 | 85.25% | $32,437.67 |
| **+ R2 Square service charges** | **186** | **94.43%** | **$5,166.49** |
The plan predicted 82.7% clean at baseline including duplicates. Measured: **82.97%**.
### Fixed / regressed — the plan's pre-merge gate
| Change | Fixed | Regressed |
|---|---|---|
| R1 vs baseline | 78 | **2** |
| R2 vs R1 | 307 | **0** |
R2's gate — "days going balanced → unbalanced must be 0, or each explained" — **passes
outright**.
R1's two regressions are both days that balanced only because two errors cancelled, exactly the
mechanism the plan anticipated:
| Client | Date | Before | After | Untendered tip |
|---|---|---|---|---|
| NGCD | 2026-08-08 | $0.00 | +$11.80 | $11.80 |
| NGDG | 2026-08-05 | $0.00 | +$10.00 | $10.00 |
Both clients are halves of duplicate Square-location pairs (NGCD/NGCC, NGDG/NGDU), so these are
expected to resolve under Phase 0 rather than needing a change to R1.
## 4. Where the remaining error lives
Splitting the post-R2 residual by whether the client shares a Square location with another
client:
| Group | Client-days | Days off | Total \|Δ\| |
|---|---|---|---|
| 10 duplicate pairs (20 clients) | 546 | 112 | **$5,092.51** |
| Everyone else (96 clients) | 2,796 | 74 | **$73.98** |
**98.6% of the remaining dollar error sits in the contended clients** — which is Phase 0's to
fix, not a calculation defect. This is the plan's central claim, and it holds.
### The $73.98 residual, in full
| Client | Days | Total \|Δ\| | |
|---|---|---|---|
| PNSP | 30 | $0.91 | register rounding |
| PNLP | 26 | $0.77 | register rounding |
| GLAD | 12 | $0.20 | register rounding |
| NGEB | 2 | $45.00 | ezCater fee semantics (plan §15.4) |
| NGPS | 1 | $9.62 | unexplained |
| NGGB | 1 | $7.59 | partial final day |
| NGSC | 1 | $5.02 | partial final day |
| NGTY | 1 | $4.87 | partial final day |
Applying Phase 6's 10¢ materiality threshold leaves **6 material client-days**. The largest
sub-threshold day is **9.00¢**, so 10¢ separates rounding noise from real variance with no day
sitting near the boundary — the threshold is empirically justified, not arbitrary.
Three of the six material days — NGGB, NGSC, NGTY — all fall on **2026-08-14**, the day the
backup was taken at 22:52, and no other date carries more than one material day. They are
partial-day artifacts of the backup cut, not defects.
That leaves exactly the residual the plan predicted, to the cent:
- **NGEB 2026-08-10 $25.00** and **NGEB 2026-07-29 $20.00** — predicted
- **NGPS 2026-08-12 +$9.62** — predicted, still unexplained
## 5. Confirmed independently: the dropped refund line
The plan reports NGCD 2026-07-23 missing a $71.94 refund line because the refund belonged to
NGCC at recompute time. That client-day measures **+$71.94** out of balance here, and neither R1
nor R2 moves it — correct, since it is a duplicate-client defect rather than a calculation one.
## 6. Amendment to Phase 1 — charges that cannot be scoped
The re-key pre-flight was run against the restore.
**Refunds — clean.** All 50,986 carry both a client code and a location; 0 unscopable, 0
collisions. Collisions are in fact structurally impossible: the old key is `:db.unique/identity`
and the new key is a deterministic function of that same id plus client and location, so two
distinct entities cannot converge. The run confirms the reasoning.
**Charges — not clean.** Of 16,545,495 charges, **2,122,161 (12.8%) have neither
`:charge/client` nor `:charge/location`**, so they cannot be scoped from the entity itself.
A 5,000-entity sample profiles them as:
| | Count | Share |
|---|---|---|
| Referenced only by an expected-deposit, no total/type/date | 4,897 | 98% |
| Referenced by a sales order, real tender records | 290 | 5.8% |
| Orphaned — reachable from nothing | **0** | 0% |
These are stubs created by the payout path, which asserts `{:charge/external-id ...}` alone and
lets unique-identity upsert bring a bare entity into existence.
**Consequence:** Phase 1 cannot read scoping off the charge. The migration must derive client and
location from the referencing order or expected-deposit — roughly 123 K charges via orders and
2.0 M via deposits, extrapolated. Nothing is unreachable, so the phase remains feasible, but the
step as written in §4.2 would strand 12.8% of charges on the legacy scheme.
The payout call site in `square.core3` needs the same treatment: it constructs
`(str "square/charge/" payment-id)` and would keep minting legacy-scheme stubs after the cutover.
It has `client` and `location` in scope already, so the fix is local.
## 7. Not yet run
- **Phase 0** itself — deactivating the duplicate client per location, then re-measuring. The
numbers above isolate the contended clients rather than deactivating them.
- **Charges referenced by more than one order** (§3.3), the gate on any historical cleanup.
- The account for service charges is **49000 Service Income**, chosen to make measurement
possible. It does not affect balance at all — only whether a day can reach `accepted?` — so
every number here is independent of that choice.