diff --git a/docs/2026-08-15-sales-summary-balance-validation.md b/docs/2026-08-15-sales-summary-balance-validation.md index 9cdf6161..bc87b993 100644 --- a/docs/2026-08-15-sales-summary-balance-validation.md +++ b/docs/2026-08-15-sales-summary-balance-validation.md @@ -168,3 +168,122 @@ It has `client` and `location` in scope already, so the fix is local. - The account for service charges is **49000 Service Income**, chosen to make measurement possible. It does not affect balance at all — only whether a day can reach `accepted?` — so every number here is independent of that choice. + + +--- + +# Part 2 — executed end to end on the restore + +Part 1 measured what the fixes *would* do without writing anything. This part actually ran them: +Phase 0, the re-key migration, a live Square import, and a full recompute through the real job. +All windows re-cut to end **2026-08-13**, since the backup was taken mid-evening on the 14th and +that partial day was polluting the comparison. + +## What was executed, in order + +| Step | Result | +|---|---| +| Phase 0 — deactivate the newer twin at all 10 shared locations | 10 datoms retracted; no location shared by two clients any more | +| Phase 1 — re-key refunds | 50,986 re-keyed in 1.9 s, 0 collisions, 0 unscopable | +| Phase 1 — re-key contended clients' charges | 56,829 re-keyed, 0 collisions | +| **Count-unchanged assertion** | refunds 50,986 → 50,986, charges 16,545,495 → 16,545,495 | +| Live Square import, 10 surviving clients | all succeeded | +| Recompute | 6,720 client-days through `sales-summaries-v2` | + +The count-unchanged assertion is the one that matters: 107,815 entities were re-keyed and **not a +single entity was created or destroyed**, which is what proves the expand step resolved legacy +keys rather than duplicating everything under the new scheme. + +## Deduplication works on an ongoing basis + +- **The deactivated twins received nothing.** Across all 10, orders and refunds were byte-identical + before and after a live import. Phase 0 holds. +- **Refunds stopped alternating and became two stable copies.** 98 of NGCC's 125 refunds are now + also held by NGCD as separate entities. Each client owns its own; neither can take the other's. + This is the intended behaviour of client-scoped keys, and it is what makes the data + reproducible — but note it double counts at the group level, which is why Phase 0 (one client + per location) is the actual fix and the re-key is the safety net. + +### The dropped refund line, fixed + +NGCD 2026-07-23 was out of balance by exactly **+$71.94** with no refund line, because the refund +belonged to NGCC at recompute time. After the full pipeline the summary carries a **"Card Refunds" +line of $71.94** and the day's imbalance is **9.1e-13 — zero**. + +## Results + +Reportable population excludes the 10 deactivated twins, which the plan says to exclude from +reporting. + +| | Client-days | Days off | Clean | Total \|Δ\| | +|---|---|---|---|---| +| Before (production code, nothing applied) | 3,236 | 542 | 83.25% | $30,982.11 | +| **After (everything applied)** | 3,030 | **71** | **97.66%** | **$405.79** | + +A **98.7% reduction in dollar error**. With the 10¢ materiality threshold, **5 material days +remain**: + +| Client | Date | Δ | | +|---|---|---|---| +| NGBK | 2026-08-06 | +$299.42 | new — see below | +| NGDA | 2026-08-01 | −$50.00 | the known auto-gratuity-as-service-charge order | +| NGEB | 2026-08-10 | −$25.00 | ezCater fee semantics, predicted | +| NGEB | 2026-07-29 | −$20.00 | ezCater fee semantics, predicted | +| NGPS | 2026-08-12 | +$9.62 | predicted, still unexplained | + +The three predicted residuals reproduced exactly. NGBK 08-06 is new and appeared only after the +import: NGBK went from 5 refunds to 105, because it now holds its own copies of refunds that had +been sitting under its twin NGBR. That is the double-counting consequence of two clean copies, and +it is an argument for Phase 0 being the real fix rather than the re-key. + +## Two defects found by executing rather than simulating + +### 1. Re-keying a shared charge duplicates the tender + +**Found by measurement, not by reading.** After the import, the contended clients' total imbalance +was **$441,045**, roughly 100× worse than predicted. The cause: + +1. 35,870 of 56,829 charges had **two parent orders**, one per twin. +2. The migration's `scope-of` resolved such a charge through whichever referencing order it found + first — sometimes the twin's — so it was re-keyed to, say, `square/charge/NGDU-DU-`. +3. On import, NGDG computed `square/charge/NGDG-DB-`, did not find it, fell back to the legacy + key — which the migration had already removed — and so **created a second charge**. +4. `:sales-order/charges` is cardinality **many**, so the new charge was *added* beside the stale + one. One order, two tenders for one payment. Tender roughly doubled. + +Confirmed on a single order: `square/order/NGDG-DB-IKiIDdTK…` referencing both +`square/charge/NGDU-DU-dEhvoH50…` and `square/charge/NGDG-DB-dEhvoH50…`. + +This is exactly what plan §4.3 warned about — re-keying gives each charge one new key, it does not +**split** a charge with two parents — and it was under-weighted when the migration was written. + +**Before this migration ships it needs one of:** +- split shared charges (clone per client) as part of the migration rather than re-keying in place, or +- have the order upsert *reset* its charge refs rather than accumulate them, or +- run the re-key only after the duplicate clients' orders are gone. + +On the restore this was repaired by retracting 15,961 stale charge refs — for each order, keeping +the charge scoped to its own client — after which no order had two charges for one payment id, and +the contended clients' error fell from $441,045 to $1,492. + +### 2. `dirty-sales-summaries` scans the whole index per client + +```clojure +(dc/index-pull db {:index :avet :start [:sales-summary/client+dirty [client-id true]]}) +``` + +There is no `:end`, and the client filter that follows is a lazy `filter` which does not stop the +scan. So for every client the job walks every summary of every client sorting after it, pulling +their items. It is O(n²) in the number of summaries. + +Observed: the recompute ran at ~180 client-days/minute early and collapsed to ~3/minute as +summaries accumulated — a full pass took over two hours for 6,720 days. Adding an `:end` bound at +the client boundary should fix it. + +## Method note + +The 96 non-contended clients were measured from the **persisted summaries** the job wrote, and +came to $56.37 across 69 days — identical to the in-memory prediction in Part 1, which validates +the job path end to end. The 20 contended clients were re-measured in memory after the charge +repair, because a second full pass through the job would have taken hours for the reason in defect +2. The two methods agree exactly where both were run.