docs(sales-summaries): validate the balance fixes against a restored backup
Measures R1 (reversed tips) and R2 (Square service charges) over 3,342 client-days of real production data, restored from backup point 209608347 (newest transaction 2026-08-14 22:52). R2 clears 307 client-days with zero regressions, which is the gate the plan set for it. R1 clears 78 with 2 regressions, both days that balanced only because two errors cancelled, and both on clients that share a Square location. After both fixes, 98.6% of the remaining dollar error sits in the 10 contended client pairs — Phase 0's to fix, not a calculation defect. Outside those, the entire residual is $73.98, of which all but six client-days is sub-10c register rounding. The three predicted material days reproduce to the cent. Also records an amendment to Phase 1: 2,122,161 charges (12.8%) carry neither :charge/client nor :charge/location, so the re-key cannot scope them from the entity. They are payout-path stubs and order-referenced tenders; none are orphaned, so scoping must come from the referencing order or expected-deposit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
170
docs/2026-08-15-sales-summary-balance-validation.md
Normal file
170
docs/2026-08-15-sales-summary-balance-validation.md
Normal file
@@ -0,0 +1,170 @@
|
||||
---
|
||||
title: Sales Summary Balance — validation against a restored production backup
|
||||
type: validation
|
||||
date: 2026-08-15
|
||||
status: measured
|
||||
---
|
||||
|
||||
# Sales Summary Balance — validation against a restored production backup
|
||||
|
||||
Measures the two calculation fixes (R1 reversed tips, R2 Square service charges) against real
|
||||
production data, and checks them against the predictions in
|
||||
[`docs/plans/2026-08-13-fix-sales-summary-balance-plan.md`](plans/2026-08-13-fix-sales-summary-balance-plan.md).
|
||||
|
||||
**Headline: the fixes behave as predicted.** R2 clears 307 client-days with zero regressions.
|
||||
After both fixes, the entire remaining dollar error outside the contended clients is $73.98,
|
||||
and all but six client-days of it is sub-10¢ register rounding.
|
||||
|
||||
---
|
||||
|
||||
## 1. What was measured against
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Database | `datomic:dev://localhost:4337/integreat-prod-restore` |
|
||||
| Restored from | `s3://integreat-backups/`, restore point **209608347** |
|
||||
| Basis-t | 209608347 |
|
||||
| Newest transaction | **2026-08-14 22:52 local** |
|
||||
| Clients | 210 (106 with orders in the window) |
|
||||
| Refunds / charges / orders | 50,986 / 16,545,495 / 19,039,575 |
|
||||
| Window | 2026-07-13 → 2026-08-14 |
|
||||
| Population | **3,342 client-days** |
|
||||
|
||||
The earlier restore in this directory was **stale** — the backup had written no root since
|
||||
2025-03-10 even though segments kept uploading daily through 2026-07-12. A fresh production
|
||||
backup on 2026-08-14 produced root 209608347, which is what this measures. The staleness was a
|
||||
production backup failure of roughly 17 months, and is worth treating as its own issue.
|
||||
|
||||
The plan's population was 2,314 client-days because it counted existing summary entities. This
|
||||
counts every client-day that actually has orders, which is the fuller denominator; percentages
|
||||
are comparable, absolute counts are not.
|
||||
|
||||
## 2. Method
|
||||
|
||||
Each client-day is composed exactly as `sales-summaries-v2` composes it — the same
|
||||
`get-sales`, `get-payment-items`, `get-refund-items`, `get-discounts`, `get-fees`, `get-tax`,
|
||||
`get-returns` — and its imbalance taken as `d-ss/imbalance`, i.e. debits minus credits. Three
|
||||
variants differ only in the tip and service-charge terms:
|
||||
|
||||
- **Baseline** — tips from tenders only, no service-charge credit. What production does today.
|
||||
- **R1** — tips additive: tendered tips plus tips on orders carrying no tender.
|
||||
- **R2** — R1 plus Square service charges, both signs.
|
||||
|
||||
Nothing was transacted. A day counts as off when `|imbalance| >= $0.005`, i.e. it does not
|
||||
round to zero.
|
||||
|
||||
## 3. Results
|
||||
|
||||
| Stage | Days off | Clean | Total \|Δ\| |
|
||||
|---|---|---|---|
|
||||
| Baseline (production today) | 569 | 82.97% | $32,921.40 |
|
||||
| + R1 additive tip | 493 | 85.25% | $32,437.67 |
|
||||
| **+ R2 Square service charges** | **186** | **94.43%** | **$5,166.49** |
|
||||
|
||||
The plan predicted 82.7% clean at baseline including duplicates. Measured: **82.97%**.
|
||||
|
||||
### Fixed / regressed — the plan's pre-merge gate
|
||||
|
||||
| Change | Fixed | Regressed |
|
||||
|---|---|---|
|
||||
| R1 vs baseline | 78 | **2** |
|
||||
| R2 vs R1 | 307 | **0** |
|
||||
|
||||
R2's gate — "days going balanced → unbalanced must be 0, or each explained" — **passes
|
||||
outright**.
|
||||
|
||||
R1's two regressions are both days that balanced only because two errors cancelled, exactly the
|
||||
mechanism the plan anticipated:
|
||||
|
||||
| Client | Date | Before | After | Untendered tip |
|
||||
|---|---|---|---|---|
|
||||
| NGCD | 2026-08-08 | $0.00 | +$11.80 | −$11.80 |
|
||||
| NGDG | 2026-08-05 | $0.00 | +$10.00 | −$10.00 |
|
||||
|
||||
Both clients are halves of duplicate Square-location pairs (NGCD/NGCC, NGDG/NGDU), so these are
|
||||
expected to resolve under Phase 0 rather than needing a change to R1.
|
||||
|
||||
## 4. Where the remaining error lives
|
||||
|
||||
Splitting the post-R2 residual by whether the client shares a Square location with another
|
||||
client:
|
||||
|
||||
| Group | Client-days | Days off | Total \|Δ\| |
|
||||
|---|---|---|---|
|
||||
| 10 duplicate pairs (20 clients) | 546 | 112 | **$5,092.51** |
|
||||
| Everyone else (96 clients) | 2,796 | 74 | **$73.98** |
|
||||
|
||||
**98.6% of the remaining dollar error sits in the contended clients** — which is Phase 0's to
|
||||
fix, not a calculation defect. This is the plan's central claim, and it holds.
|
||||
|
||||
### The $73.98 residual, in full
|
||||
|
||||
| Client | Days | Total \|Δ\| | |
|
||||
|---|---|---|---|
|
||||
| PNSP | 30 | $0.91 | register rounding |
|
||||
| PNLP | 26 | $0.77 | register rounding |
|
||||
| GLAD | 12 | $0.20 | register rounding |
|
||||
| NGEB | 2 | $45.00 | ezCater fee semantics (plan §15.4) |
|
||||
| NGPS | 1 | $9.62 | unexplained |
|
||||
| NGGB | 1 | $7.59 | partial final day |
|
||||
| NGSC | 1 | $5.02 | partial final day |
|
||||
| NGTY | 1 | $4.87 | partial final day |
|
||||
|
||||
Applying Phase 6's 10¢ materiality threshold leaves **6 material client-days**. The largest
|
||||
sub-threshold day is **9.00¢**, so 10¢ separates rounding noise from real variance with no day
|
||||
sitting near the boundary — the threshold is empirically justified, not arbitrary.
|
||||
|
||||
Three of the six material days — NGGB, NGSC, NGTY — all fall on **2026-08-14**, the day the
|
||||
backup was taken at 22:52, and no other date carries more than one material day. They are
|
||||
partial-day artifacts of the backup cut, not defects.
|
||||
|
||||
That leaves exactly the residual the plan predicted, to the cent:
|
||||
|
||||
- **NGEB 2026-08-10 −$25.00** and **NGEB 2026-07-29 −$20.00** — predicted
|
||||
- **NGPS 2026-08-12 +$9.62** — predicted, still unexplained
|
||||
|
||||
## 5. Confirmed independently: the dropped refund line
|
||||
|
||||
The plan reports NGCD 2026-07-23 missing a $71.94 refund line because the refund belonged to
|
||||
NGCC at recompute time. That client-day measures **+$71.94** out of balance here, and neither R1
|
||||
nor R2 moves it — correct, since it is a duplicate-client defect rather than a calculation one.
|
||||
|
||||
## 6. Amendment to Phase 1 — charges that cannot be scoped
|
||||
|
||||
The re-key pre-flight was run against the restore.
|
||||
|
||||
**Refunds — clean.** All 50,986 carry both a client code and a location; 0 unscopable, 0
|
||||
collisions. Collisions are in fact structurally impossible: the old key is `:db.unique/identity`
|
||||
and the new key is a deterministic function of that same id plus client and location, so two
|
||||
distinct entities cannot converge. The run confirms the reasoning.
|
||||
|
||||
**Charges — not clean.** Of 16,545,495 charges, **2,122,161 (12.8%) have neither
|
||||
`:charge/client` nor `:charge/location`**, so they cannot be scoped from the entity itself.
|
||||
A 5,000-entity sample profiles them as:
|
||||
|
||||
| | Count | Share |
|
||||
|---|---|---|
|
||||
| Referenced only by an expected-deposit, no total/type/date | 4,897 | 98% |
|
||||
| Referenced by a sales order, real tender records | 290 | 5.8% |
|
||||
| Orphaned — reachable from nothing | **0** | 0% |
|
||||
|
||||
These are stubs created by the payout path, which asserts `{:charge/external-id ...}` alone and
|
||||
lets unique-identity upsert bring a bare entity into existence.
|
||||
|
||||
**Consequence:** Phase 1 cannot read scoping off the charge. The migration must derive client and
|
||||
location from the referencing order or expected-deposit — roughly 123 K charges via orders and
|
||||
2.0 M via deposits, extrapolated. Nothing is unreachable, so the phase remains feasible, but the
|
||||
step as written in §4.2 would strand 12.8% of charges on the legacy scheme.
|
||||
|
||||
The payout call site in `square.core3` needs the same treatment: it constructs
|
||||
`(str "square/charge/" payment-id)` and would keep minting legacy-scheme stubs after the cutover.
|
||||
It has `client` and `location` in scope already, so the fix is local.
|
||||
|
||||
## 7. Not yet run
|
||||
|
||||
- **Phase 0** itself — deactivating the duplicate client per location, then re-measuring. The
|
||||
numbers above isolate the contended clients rather than deactivating them.
|
||||
- **Charges referenced by more than one order** (§3.3), the gate on any historical cleanup.
|
||||
- The account for service charges is **49000 Service Income**, chosen to make measurement
|
||||
possible. It does not affect balance at all — only whether a day can reach `accepted?` — so
|
||||
every number here is independent of that choice.
|
||||
Reference in New Issue
Block a user