docs(sales-summaries): rebuild the report on a real 90-day baseline

The previous report derived its baseline arithmetically. This one runs the job
three times over the same 90 days against identical data, writing real summaries
each pass: production's calculation, then R1, then R2. 18,900 summaries per run,
56,700 in total. Window 2026-05-10 to 2026-08-07, deliberately stopping a week
short of the backup so the partial days around the cut cannot distort it.

Reportable population, 8,350 client-days across 96 clients:

  baseline  1,024 days off, 87.74% clean, $58,531.75
  + R1        752 days off, 90.99% clean, $55,528.17
  + R1 + R2   105 days off, 98.74% clean,    $852.38

  R1  8,054 unchanged, 272 to balance, 0 to unbalance, 0 balanced-day edits
  R2  7,701 unchanged, 647 to balance, 0 to unbalance, 0 balanced-day edits

Across 90 days of trading neither fix altered a single day that already
balanced, replicating the 30-day finding on three times the sample.

The longer window surfaced two residual clusters the short one could not see:
NGMV across five days in late May and NGEB across four spanning May to July.
NGEB is the known ezCater fee question; NGMV and N-30012 are undiagnosed and
worth a look before this ships.

Also records a deployment finding: the transactor was sized for a toy database
(2 GB object cache against 27 GB of data) and the final 7,958 client-days of a
pass were crawling at ~35/min. Resized to 8 GB cache and 16 GB heap, they
finished in 90 seconds. Worth checking production's sizing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

docs/2026-08-15-remove-voided-orders-risk.md

@@ -0,0 +1,115 @@
+---
+title: remove-voided-orders can delete another client's payments
+type: risk
+date: 2026-08-15
+status: open — decide before merging the re-key
+---
+
+# `remove-voided-orders` can delete another client's payments
+
+Measured on the restored backup, 2026-08-15. This risk is **pre-existing** — nothing in the
+sales-summary work created it — but it is live right now, and the re-key work touches the same
+data, so it should be understood before merging.
+
+## The mechanism, in four steps
+
+**1. Charges are component entities of an order.**
+
+```clojure
+;; resources/schema.edn
+{:db/ident :sales-order/charges
+ :db/valueType :db.type/ref
+ :db/isComponent true          ;; <- this is the load-bearing bit
+ :db/cardinality :db.cardinality/many}
+```
+
+`:db/isComponent true` tells Datomic the charges *belong to* the order. It is what lets you
+transact an order with its tenders nested inside, and it means the charges have no independent
+existence as far as Datomic is concerned.
+
+**2. `retractEntity` on a component parent deletes the children too.**
+
+That is the documented behaviour of `:db/retractEntity`: it recursively retracts component
+values. `square.core3/remove-voided-orders` ends with exactly that:
+
+```clojure
+(s/map (fn [[o]]
+         [[:db/retractEntity [:sales-order/external-id (:sales-order/external-id o)]]]))
+```
+
+It asks Square for the last 10 days of orders, keeps the ones that should *not* be imported —
+voided and cancelled orders — and retracts any of those we already stored. That is correct and
+desirable on its own: a voided order should not sit in the books.
+
+**3. But one charge can be shared by two orders.**
+
+When two clients are configured on the same Square location, both import the same Square data.
+Order keys embed the client, so each client gets its own order entity. Charge keys did **not**
+embed the client, and `:charge/external-id` is `:db.unique/identity`, so both clients' orders
+resolved to *the same charge entity*:
+
+```
+NGCD order 17592395490523 ──┐
+                            ├──> charge 17592490524   ← one entity, two parents
+NGCC order 17592395511722 ──┘
+```
+
+**4. So retracting one order deletes a charge the other order still points at.**
+
+Datomic sees a component and removes it. The surviving order keeps its line items — its sales —
+but its tender is gone. The day then shows revenue with no payment against it, the summary goes
+out of balance, and the payment is gone from the current database value. (History retains it, so
+it is recoverable by someone who knows to look, but nothing in the app will show it again.)
+
+## How exposed are we
+
+Measured over the 10 contended clients across 2026-07-13 → 08-14:
+
+| | |
+|---|---|
+| Charges examined | 56,829 |
+| **Referenced by more than one order** | **35,870 (63%)** |
+
+So this is not a theoretical corner. Roughly two thirds of the charges in that population have
+two parents, and any voided order among them takes a charge down with it.
+
+The exposure window for *new* damage is the rolling 10 days `remove-voided-orders` searches, but
+the shared charges themselves span the whole period the locations were double-configured.
+
+## What changes after Phase 0 and the re-key, and what doesn't
+
+- **Phase 0 (done on the restore)** stops new sharing: only one client per location imports now,
+  so no new order pairs form.
+- **The re-key (done for refunds and the contended clients' charges)** makes sharing structurally
+  impossible going forward, because a charge key now contains the client code.
+- **Neither retroactively splits the 35,870 charges that are already shared.** They still have two
+  parents. Until they are split, `remove-voided-orders` remains capable of deleting a payment
+  belonging to the other client.
+
+This is why plan §3.3 forbids retracting anything — including any historical cleanup of the
+duplicate clients' data — until a verification query shows zero charges with more than one parent.
+
+## Options, roughly in order of preference
+
+1. **Split the shared charges, then let removal run normally.** Re-import the affected window now
+   that keys are client-scoped, so each client creates its own charge entity. This reuses the
+   import path rather than hand-constructing component entities. Verify with a query for charges
+   having more than one referencing order; it must reach zero.
+
+2. **Guard the retraction.** Before retracting an order, check whether any of its charges are
+   referenced by another order; detach those (retract the `:sales-order/charges` ref rather than
+   the charge) and retract the rest. Small, contained change, and it makes the operation safe
+   regardless of what shape the data is in — worth doing on its own merits even after a split.
+
+3. **Do nothing and accept it.** Only defensible once every location has a single client *and*
+   the historical shared charges are gone. Not true today.
+
+## What I did about it during the validation run
+
+I ran the import on the restore with `remove-voided-orders` **skipped**, and ran the other steps
+(`upsert-locations`, `upsert`, `upsert-payouts`, `upsert-refunds`) normally. That kept the
+validation faithful to how the import behaves without risking silent payment loss in the data
+the measurements were about to be taken from.
+
+**Nothing in the production system has been changed.** This note is about a risk that already
+exists there.

docs/2026-08-15-sales-summary-balance-validation.md

@@ -0,0 +1,393 @@
+---
+title: Sales Summary Balance — validation against a restored production backup
+type: validation
+date: 2026-08-15
+status: measured
+---
+
+# Sales Summary Balance — validation against a restored production backup
+
+Measures the two calculation fixes (R1 reversed tips, R2 Square service charges) against real
+production data, and checks them against the predictions in
+[`docs/plans/2026-08-13-fix-sales-summary-balance-plan.md`](plans/2026-08-13-fix-sales-summary-balance-plan.md).
+
+**Headline: the fixes behave as predicted.** R2 clears 307 client-days with zero regressions.
+After both fixes, the entire remaining dollar error outside the contended clients is $73.98,
+and all but six client-days of it is sub-10¢ register rounding.
+
+---
+
+## 1. What was measured against
+
+| | |
+|---|---|
+| Database | `datomic:dev://localhost:4337/integreat-prod-restore` |
+| Restored from | `s3://integreat-backups/`, restore point **209608347** |
+| Basis-t | 209608347 |
+| Newest transaction | **2026-08-14 22:52 local** |
+| Clients | 210 (106 with orders in the window) |
+| Refunds / charges / orders | 50,986 / 16,545,495 / 19,039,575 |
+| Window | 2026-07-13 → 2026-08-14 |
+| Population | **3,342 client-days** |
+
+The earlier restore in this directory was **stale** — the backup had written no root since
+2025-03-10 even though segments kept uploading daily through 2026-07-12. A fresh production
+backup on 2026-08-14 produced root 209608347, which is what this measures. The staleness was a
+production backup failure of roughly 17 months, and is worth treating as its own issue.
+
+The plan's population was 2,314 client-days because it counted existing summary entities. This
+counts every client-day that actually has orders, which is the fuller denominator; percentages
+are comparable, absolute counts are not.
+
+## 2. Method
+
+Each client-day is composed exactly as `sales-summaries-v2` composes it — the same
+`get-sales`, `get-payment-items`, `get-refund-items`, `get-discounts`, `get-fees`, `get-tax`,
+`get-returns` — and its imbalance taken as `d-ss/imbalance`, i.e. debits minus credits. Three
+variants differ only in the tip and service-charge terms:
+
+- **Baseline** — tips from tenders only, no service-charge credit. What production does today.
+- **R1** — tips additive: tendered tips plus tips on orders carrying no tender.
+- **R2** — R1 plus Square service charges, both signs.
+
+Nothing was transacted. A day counts as off when `|imbalance| >= $0.005`, i.e. it does not
+round to zero.
+
+## 3. Results
+
+| Stage | Days off | Clean | Total \|Δ\| |
+|---|---|---|---|
+| Baseline (production today) | 569 | 82.97% | $32,921.40 |
+| + R1 additive tip | 493 | 85.25% | $32,437.67 |
+| **+ R2 Square service charges** | **186** | **94.43%** | **$5,166.49** |
+
+The plan predicted 82.7% clean at baseline including duplicates. Measured: **82.97%**.
+
+### Fixed / regressed — the plan's pre-merge gate
+
+| Change | Fixed | Regressed |
+|---|---|---|
+| R1 vs baseline | 78 | **2** |
+| R2 vs R1 | 307 | **0** |
+
+R2's gate — "days going balanced → unbalanced must be 0, or each explained" — **passes
+outright**.
+
+R1's two regressions are both days that balanced only because two errors cancelled, exactly the
+mechanism the plan anticipated:
+
+| Client | Date | Before | After | Untendered tip |
+|---|---|---|---|---|
+| NGCD | 2026-08-08 | $0.00 | +$11.80 | −$11.80 |
+| NGDG | 2026-08-05 | $0.00 | +$10.00 | −$10.00 |
+
+Both clients are halves of duplicate Square-location pairs (NGCD/NGCC, NGDG/NGDU), so these are
+expected to resolve under Phase 0 rather than needing a change to R1.
+
+## 4. Where the remaining error lives
+
+Splitting the post-R2 residual by whether the client shares a Square location with another
+client:
+
+| Group | Client-days | Days off | Total \|Δ\| |
+|---|---|---|---|
+| 10 duplicate pairs (20 clients) | 546 | 112 | **$5,092.51** |
+| Everyone else (96 clients) | 2,796 | 74 | **$73.98** |
+
+**98.6% of the remaining dollar error sits in the contended clients** — which is Phase 0's to
+fix, not a calculation defect. This is the plan's central claim, and it holds.
+
+### The $73.98 residual, in full
+
+| Client | Days | Total \|Δ\| | |
+|---|---|---|---|
+| PNSP | 30 | $0.91 | register rounding |
+| PNLP | 26 | $0.77 | register rounding |
+| GLAD | 12 | $0.20 | register rounding |
+| NGEB | 2 | $45.00 | ezCater fee semantics (plan §15.4) |
+| NGPS | 1 | $9.62 | unexplained |
+| NGGB | 1 | $7.59 | partial final day |
+| NGSC | 1 | $5.02 | partial final day |
+| NGTY | 1 | $4.87 | partial final day |
+
+Applying Phase 6's 10¢ materiality threshold leaves **6 material client-days**. The largest
+sub-threshold day is **9.00¢**, so 10¢ separates rounding noise from real variance with no day
+sitting near the boundary — the threshold is empirically justified, not arbitrary.
+
+Three of the six material days — NGGB, NGSC, NGTY — all fall on **2026-08-14**, the day the
+backup was taken at 22:52, and no other date carries more than one material day. They are
+partial-day artifacts of the backup cut, not defects.
+
+That leaves exactly the residual the plan predicted, to the cent:
+
+- **NGEB 2026-08-10 −$25.00** and **NGEB 2026-07-29 −$20.00** — predicted
+- **NGPS 2026-08-12 +$9.62** — predicted, still unexplained
+
+## 5. Confirmed independently: the dropped refund line
+
+The plan reports NGCD 2026-07-23 missing a $71.94 refund line because the refund belonged to
+NGCC at recompute time. That client-day measures **+$71.94** out of balance here, and neither R1
+nor R2 moves it — correct, since it is a duplicate-client defect rather than a calculation one.
+
+## 6. Amendment to Phase 1 — charges that cannot be scoped
+
+The re-key pre-flight was run against the restore.
+
+**Refunds — clean.** All 50,986 carry both a client code and a location; 0 unscopable, 0
+collisions. Collisions are in fact structurally impossible: the old key is `:db.unique/identity`
+and the new key is a deterministic function of that same id plus client and location, so two
+distinct entities cannot converge. The run confirms the reasoning.
+
+**Charges — not clean.** Of 16,545,495 charges, **2,122,161 (12.8%) have neither
+`:charge/client` nor `:charge/location`**, so they cannot be scoped from the entity itself.
+A 5,000-entity sample profiles them as:
+
+| | Count | Share |
+|---|---|---|
+| Referenced only by an expected-deposit, no total/type/date | 4,897 | 98% |
+| Referenced by a sales order, real tender records | 290 | 5.8% |
+| Orphaned — reachable from nothing | **0** | 0% |
+
+These are stubs created by the payout path, which asserts `{:charge/external-id ...}` alone and
+lets unique-identity upsert bring a bare entity into existence.
+
+**Consequence:** Phase 1 cannot read scoping off the charge. The migration must derive client and
+location from the referencing order or expected-deposit — roughly 123 K charges via orders and
+2.0 M via deposits, extrapolated. Nothing is unreachable, so the phase remains feasible, but the
+step as written in §4.2 would strand 12.8% of charges on the legacy scheme.
+
+The payout call site in `square.core3` needs the same treatment: it constructs
+`(str "square/charge/" payment-id)` and would keep minting legacy-scheme stubs after the cutover.
+It has `client` and `location` in scope already, so the fix is local.
+
+## 7. Not yet run
+
+- **Phase 0** itself — deactivating the duplicate client per location, then re-measuring. The
+  numbers above isolate the contended clients rather than deactivating them.
+- **Charges referenced by more than one order** (§3.3), the gate on any historical cleanup.
+- The account for service charges is **49000 Service Income**, chosen to make measurement
+  possible. It does not affect balance at all — only whether a day can reach `accepted?` — so
+  every number here is independent of that choice.
+
+
+---
+
+# Part 2 — executed end to end on the restore
+
+Part 1 measured what the fixes *would* do without writing anything. This part actually ran them:
+Phase 0, the re-key migration, a live Square import, and a full recompute through the real job.
+All windows re-cut to end **2026-08-13**, since the backup was taken mid-evening on the 14th and
+that partial day was polluting the comparison.
+
+## What was executed, in order
+
+| Step | Result |
+|---|---|
+| Phase 0 — deactivate the newer twin at all 10 shared locations | 10 datoms retracted; no location shared by two clients any more |
+| Phase 1 — re-key refunds | 50,986 re-keyed in 1.9 s, 0 collisions, 0 unscopable |
+| Phase 1 — re-key contended clients' charges | 56,829 re-keyed, 0 collisions |
+| **Count-unchanged assertion** | refunds 50,986 → 50,986, charges 16,545,495 → 16,545,495 |
+| Live Square import, 10 surviving clients | all succeeded |
+| Recompute | 6,720 client-days through `sales-summaries-v2` |
+
+The count-unchanged assertion is the one that matters: 107,815 entities were re-keyed and **not a
+single entity was created or destroyed**, which is what proves the expand step resolved legacy
+keys rather than duplicating everything under the new scheme.
+
+## Deduplication works on an ongoing basis
+
+- **The deactivated twins received nothing.** Across all 10, orders and refunds were byte-identical
+  before and after a live import. Phase 0 holds.
+- **Refunds stopped alternating and became two stable copies.** 98 of NGCC's 125 refunds are now
+  also held by NGCD as separate entities. Each client owns its own; neither can take the other's.
+  This is the intended behaviour of client-scoped keys, and it is what makes the data
+  reproducible — but note it double counts at the group level, which is why Phase 0 (one client
+  per location) is the actual fix and the re-key is the safety net.
+
+### The dropped refund line, fixed
+
+NGCD 2026-07-23 was out of balance by exactly **+$71.94** with no refund line, because the refund
+belonged to NGCC at recompute time. After the full pipeline the summary carries a **"Card Refunds"
+line of $71.94** and the day's imbalance is **9.1e-13 — zero**.
+
+## Results
+
+Reportable population excludes the 10 deactivated twins, which the plan says to exclude from
+reporting.
+
+| | Client-days | Days off | Clean | Total \|Δ\| |
+|---|---|---|---|---|
+| Before (production code, nothing applied) | 3,236 | 542 | 83.25% | $30,982.11 |
+| **After (everything applied)** | 3,030 | **71** | **97.66%** | **$405.79** |
+
+A **98.7% reduction in dollar error**. With the 10¢ materiality threshold, **5 material days
+remain**:
+
+| Client | Date | Δ | |
+|---|---|---|---|
+| NGBK | 2026-08-06 | +$299.42 | new — see below |
+| NGDA | 2026-08-01 | −$50.00 | the known auto-gratuity-as-service-charge order |
+| NGEB | 2026-08-10 | −$25.00 | ezCater fee semantics, predicted |
+| NGEB | 2026-07-29 | −$20.00 | ezCater fee semantics, predicted |
+| NGPS | 2026-08-12 | +$9.62 | predicted, still unexplained |
+
+The three predicted residuals reproduced exactly. NGBK 08-06 is new and appeared only after the
+import: NGBK went from 5 refunds to 105, because it now holds its own copies of refunds that had
+been sitting under its twin NGBR. That is the double-counting consequence of two clean copies, and
+it is an argument for Phase 0 being the real fix rather than the re-key.
+
+## Two defects found by executing rather than simulating
+
+### 1. Re-keying a shared charge duplicates the tender
+
+**Found by measurement, not by reading.** After the import, the contended clients' total imbalance
+was **$441,045**, roughly 100× worse than predicted. The cause:
+
+1. 35,870 of 56,829 charges had **two parent orders**, one per twin.
+2. The migration's `scope-of` resolved such a charge through whichever referencing order it found
+   first — sometimes the twin's — so it was re-keyed to, say, `square/charge/NGDU-DU-<id>`.
+3. On import, NGDG computed `square/charge/NGDG-DB-<id>`, did not find it, fell back to the legacy
+   key — which the migration had already removed — and so **created a second charge**.
+4. `:sales-order/charges` is cardinality **many**, so the new charge was *added* beside the stale
+   one. One order, two tenders for one payment. Tender roughly doubled.
+
+Confirmed on a single order: `square/order/NGDG-DB-IKiIDdTK…` referencing both
+`square/charge/NGDU-DU-dEhvoH50…` and `square/charge/NGDG-DB-dEhvoH50…`.
+
+This is exactly what plan §4.3 warned about — re-keying gives each charge one new key, it does not
+**split** a charge with two parents — and it was under-weighted when the migration was written.
+
+**Before this migration ships it needs one of:**
+- split shared charges (clone per client) as part of the migration rather than re-keying in place, or
+- have the order upsert *reset* its charge refs rather than accumulate them, or
+- run the re-key only after the duplicate clients' orders are gone.
+
+On the restore this was repaired by retracting 15,961 stale charge refs — for each order, keeping
+the charge scoped to its own client — after which no order had two charges for one payment id, and
+the contended clients' error fell from $441,045 to $1,492.
+
+### 2. `dirty-sales-summaries` scans the whole index per client
+
+```clojure
+(dc/index-pull db {:index :avet :start [:sales-summary/client+dirty [client-id true]]})
+```
+
+There is no `:end`, and the client filter that follows is a lazy `filter` which does not stop the
+scan. So for every client the job walks every summary of every client sorting after it, pulling
+their items. It is O(n²) in the number of summaries.
+
+Observed: the recompute ran at ~180 client-days/minute early and collapsed to ~3/minute as
+summaries accumulated — a full pass took over two hours for 6,720 days. Adding an `:end` bound at
+the client boundary should fix it.
+
+## Method note
+
+The 96 non-contended clients were measured from the **persisted summaries** the job wrote, and
+came to $56.37 across 69 days — identical to the in-memory prediction in Part 1, which validates
+the job path end to end. The 20 contended clients were re-measured in memory after the charge
+repair, because a second full pass through the job would have taken hours for the reason in defect
+2. The two methods agree exactly where both were run.
+
+## Ongoing deduplication, with both clients active again
+
+The re-key is meant to make contention impossible even if a location is configured on two clients
+again. To test that rather than assume it, all 10 twins were **re-activated** — every location
+shared once more — and the import re-run.
+
+**Repeated imports are idempotent.** Three consecutive refund imports for both halves of the
+Concord pair, with both clients live on location `L43Z5GMW72VMG`:
+
+| | run 0 | run 1 | run 2 | run 3 |
+|---|---|---|---|---|
+| NGCC | 127 | 127 | 127 | 127 |
+| NGCD | 1,419 | 1,419 | 1,419 | 1,419 |
+
+Not one entity created or moved. Before the re-key this is the exact configuration that made a
+refund's owner flip every ~20 minutes.
+
+The first import after the migration is the only one that grows anything, and it grows once: NGCD
+gained 100 refunds materialising its own copies, while NGCC — whose refunds already carried its
+own scoped key — gained only the 2 that were genuinely new. After that, steady state.
+
+**State the restore was left in:** twins re-activated, i.e. Phase 0 deliberately undone for this
+test. Re-applying it is the ten-datom retraction recorded above.
+
+
+---
+
+# Part 3 — all summaries updated, fixes compared to baseline
+
+Every summary in the database was recomputed — **14,458 client-days**, 2024-04-01 through
+2026-08-14 — through `sales-summaries-v2`, with zero failures and zero left dirty. The
+comparison below is the **last 30 days, 2026-07-15 → 2026-08-13**.
+
+Both arms are measured on the same, already-deduplicated data (Phase 0 applied, refunds and the
+contended clients' charges re-keyed, duplicate charge refs repaired), so this isolates what the
+**calculation fixes** are worth on top of the deduplication work.
+
+## Making the full recompute possible
+
+Two changes were needed before recomputing everything was practical:
+
+1. **`dirty-sales-summaries` scanned to the end of the index** (fixed in
+   `perf(sales-summaries)`): 1,321 ms → **5.6 ms** per client, a 237× improvement, identical
+   results.
+2. **The driver was one serial `doseq`.** `refresh-client!` was split out of `sales-summaries-v2`
+   so a client's work stands alone. Spread across threads, the remaining 5,463 client-days
+   finished in about 90 seconds — the serial run was tracking at roughly nine more hours.
+
+## Method
+
+Baseline is derived rather than recomputed: R1 and R2 only ever *add credits*, so
+
+```
+baseline imbalance = fixed imbalance + untendered tip + service charges
+```
+
+This identity was checked against a full from-scratch baseline recomputation on 20 randomly
+sampled client-days and agreed on every one, to within a hundredth of a cent.
+
+## Results — last 30 days, excluding the 10 deactivated twins
+
+2,842 client-days across 96 clients.
+
+| | Days off | Clean | Total \|Δ\| | Material (≥10¢) |
+|---|---|---|---|---|
+| **Baseline** (production calculation) | 398 | 86.00% | $22,527.40 | 336 |
+| **Fixed** (R1 + R2) | **67** | **97.64%** | **$405.66** | **5** |
+
+**331 client-days fixed, 0 regressed.** Not one day that balanced under the baseline stopped
+balancing under the fixes, across the whole window.
+
+Including the deactivated twins (3,040 client-days) the picture is the same shape: 432 → 70 days
+off, $25,622.98 → $1,548.64.
+
+### What is left
+
+| Client | Date | Δ | |
+|---|---|---|---|
+| NGBK | 2026-08-06 | +$299.42 | refunds now held by both twins — see below |
+| NGDA | 2026-08-01 | −$50.00 | auto-gratuity booked as a service charge |
+| NGEB | 2026-08-10 | −$25.00 | ezCater fee semantics, predicted by the plan |
+| NGEB | 2026-07-29 | −$20.00 | ezCater fee semantics, predicted by the plan |
+| NGPS | 2026-08-12 | +$9.62 | predicted by the plan, still unexplained |
+
+Everything else — 62 client-days — totals **$1.62**, with the largest single day at **9.00¢**.
+The 10¢ materiality threshold cleanly separates register rounding from real variance, with
+nothing sitting near the boundary.
+
+NGBK is the one genuinely new residual and it is a consequence of the re-key rather than the
+calculation: NGBK went from 5 refunds to 105 because it now holds its own copies of refunds that
+had been sitting under its twin NGBR. Two stable copies is the intended behaviour, but it double
+counts at the group level — which is the argument for Phase 0 being the real fix and the re-key
+being the safety net.
+
+### Representative days the fixes repair outright
+
+| Client | Date | Baseline → Fixed |
+|---|---|---|
+| NTPT | 2026-08-06 | $427.10 → $0.00 |
+| NGFO | 2026-07-16 | $275.23 → $0.00 |
+| NGRN | 2026-08-06 | $236.96 → $0.00 |
+| N-30008 | 2026-07-24 | $229.42 → $0.00 |
+| N-30003 | 2026-07-18 | $225.00 → $0.00 |

docs/2026-08-15-thirty-day-reconciliation.html

@@ -0,0 +1,364 @@
+<title>Ninety-Day Reconciliation</title>
+<style>
+  :root {
+    --paper: #F6F8F7;
+    --card: #FFFFFF;
+    --ink: #141F1D;
+    --ink-soft: #4A5C58;
+    --ink-faint: #7C8D89;
+    --rule: #DCE4E1;
+    --accent: #0E5B57;
+    --accent-soft: #E3EFED;
+    --good: #1A6B49;
+    --bad: #A03B26;
+    --warn: #8A6410;
+    --shadow: 0 1px 2px rgba(20,31,29,.06), 0 8px 24px rgba(20,31,29,.05);
+  }
+  @media (prefers-color-scheme: dark) {
+    :root:not([data-theme="light"]) {
+      --paper: #0E1615; --card: #151F1E; --ink: #E8EFED; --ink-soft: #A3B3AF;
+      --ink-faint: #74847F; --rule: #26332F; --accent: #5FBDB4; --accent-soft: #16302E;
+      --good: #5FBE8C; --bad: #E08A72; --warn: #D6AC55;
+      --shadow: 0 1px 2px rgba(0,0,0,.4), 0 8px 24px rgba(0,0,0,.3);
+    }
+  }
+  :root[data-theme="dark"] {
+    --paper: #0E1615; --card: #151F1E; --ink: #E8EFED; --ink-soft: #A3B3AF;
+    --ink-faint: #74847F; --rule: #26332F; --accent: #5FBDB4; --accent-soft: #16302E;
+    --good: #5FBE8C; --bad: #E08A72; --warn: #D6AC55;
+    --shadow: 0 1px 2px rgba(0,0,0,.4), 0 8px 24px rgba(0,0,0,.3);
+  }
+
+  * { box-sizing: border-box; }
+  body {
+    background: var(--paper); color: var(--ink);
+    font-family: system-ui, -apple-system, "Segoe UI", sans-serif;
+    font-size: 16px; line-height: 1.6; margin: 0; padding: 0 20px 96px;
+  }
+  .wrap { max-width: 940px; margin: 0 auto; }
+  .measure { max-width: 66ch; }
+  .num { font-variant-numeric: tabular-nums; font-feature-settings: "tnum"; }
+  .mono { font-family: ui-monospace, "SF Mono", "Cascadia Code", monospace; font-variant-numeric: tabular-nums; }
+
+  header.masthead { padding: 72px 0 40px; border-bottom: 2px solid var(--ink); display: flex; flex-direction: column; gap: 14px; }
+  .eyebrow { font-size: 12px; letter-spacing: .14em; text-transform: uppercase; color: var(--accent); font-weight: 600; }
+  h1 {
+    font-family: Georgia, "Iowan Old Style", "Times New Roman", serif;
+    font-size: clamp(34px, 5.4vw, 54px); line-height: 1.08; font-weight: 600;
+    letter-spacing: -.015em; margin: 0; text-wrap: balance;
+  }
+  .standfirst { font-size: 19px; color: var(--ink-soft); margin: 0; max-width: 60ch; }
+  .meta { display: flex; flex-wrap: wrap; gap: 10px 28px; font-size: 13px; color: var(--ink-faint); padding-top: 6px; }
+  .meta b { color: var(--ink-soft); font-weight: 600; }
+
+  section { padding-top: 56px; display: flex; flex-direction: column; gap: 20px; }
+  h2 { font-family: Georgia, "Iowan Old Style", serif; font-size: 27px; font-weight: 600; letter-spacing: -.01em; margin: 0; text-wrap: balance; }
+  h3 { font-size: 14px; letter-spacing: .08em; text-transform: uppercase; color: var(--ink-soft); font-weight: 700; margin: 0; }
+  p { margin: 0; }
+  .measure p + p { margin-top: 14px; }
+
+  .ledger {
+    display: grid; grid-template-columns: 1fr auto 1fr; align-items: stretch;
+    border: 1px solid var(--rule); border-radius: 4px; background: var(--card);
+    box-shadow: var(--shadow); overflow: hidden;
+  }
+  .ledger > div { padding: 26px 28px; display: flex; flex-direction: column; gap: 6px; }
+  .ledger .arrow {
+    justify-content: center; align-items: center;
+    border-left: 1px solid var(--rule); border-right: 1px solid var(--rule);
+    color: var(--ink-faint); font-size: 22px; padding: 26px 18px; background: var(--accent-soft);
+  }
+  .ledger .side-label { font-size: 12px; letter-spacing: .12em; text-transform: uppercase; color: var(--ink-faint); font-weight: 600; }
+  .figure { font-size: clamp(28px, 4.4vw, 40px); font-weight: 650; line-height: 1.05; letter-spacing: -.02em; }
+  .figure.after { color: var(--good); }
+  .subfig { font-size: 14px; color: var(--ink-soft); }
+
+  .stats { display: grid; grid-template-columns: repeat(auto-fit, minmax(170px, 1fr)); gap: 14px; }
+  .stat { background: var(--card); border: 1px solid var(--rule); border-radius: 4px; padding: 18px 20px; display: flex; flex-direction: column; gap: 4px; }
+  .stat .k { font-size: 30px; font-weight: 650; letter-spacing: -.02em; line-height: 1; }
+  .stat .l { font-size: 13px; color: var(--ink-soft); }
+  .stat.zero .k { color: var(--good); }
+
+  .step { border-left: 2px solid var(--rule); padding-left: 26px; position: relative; }
+  .step::before {
+    content: attr(data-step); position: absolute; left: -13px; top: 2px;
+    width: 24px; height: 24px; border-radius: 50%;
+    background: var(--accent); color: var(--paper);
+    font-size: 12px; font-weight: 700; display: grid; place-items: center;
+  }
+  .step h3 { margin-bottom: 8px; }
+  .step-title { font-size: 20px; font-weight: 650; letter-spacing: -.01em; }
+
+  .scroll { overflow-x: auto; border: 1px solid var(--rule); border-radius: 4px; background: var(--card); }
+  table { border-collapse: collapse; width: 100%; font-size: 14.5px; }
+  th, td { padding: 11px 16px; text-align: left; border-bottom: 1px solid var(--rule); white-space: nowrap; }
+  thead th { font-size: 11.5px; letter-spacing: .09em; text-transform: uppercase; color: var(--ink-faint); font-weight: 700; background: var(--accent-soft); }
+  tbody tr:last-child td { border-bottom: none; }
+  td.n, th.n { text-align: right; font-variant-numeric: tabular-nums; }
+  tr.total td { font-weight: 650; background: var(--accent-soft); }
+  .good { color: var(--good); font-weight: 650; }
+  .bad { color: var(--bad); font-weight: 650; }
+  .dim { color: var(--ink-faint); }
+
+  .callout { background: var(--card); border: 1px solid var(--rule); border-left: 3px solid var(--accent); border-radius: 4px; padding: 20px 24px; display: flex; flex-direction: column; gap: 10px; }
+  .callout.warn { border-left-color: var(--warn); }
+  .callout .h { font-weight: 650; }
+
+  code { font-family: ui-monospace, "SF Mono", monospace; font-size: .9em; background: var(--accent-soft); padding: 1px 5px; border-radius: 3px; }
+  pre { margin: 0; padding: 20px; font-size: 13px; line-height: 1.7; white-space: pre; font-family: ui-monospace, "SF Mono", monospace; }
+  footer { margin-top: 72px; padding-top: 24px; border-top: 1px solid var(--rule); font-size: 13px; color: var(--ink-faint); display: flex; flex-direction: column; gap: 8px; }
+  ul { margin: 0; padding-left: 20px; display: flex; flex-direction: column; gap: 8px; }
+</style>
+
+<div class="wrap">
+
+  <header class="masthead">
+    <div class="eyebrow">Sales summaries · restored production backup</div>
+    <h1>Ninety-Day Reconciliation</h1>
+    <p class="standfirst">A true baseline: the job re-run three times over ninety days — production's calculation, then each fix added — so every number below is measured output, not arithmetic.</p>
+    <div class="meta">
+      <span><b>Window</b> 2026-05-10 → 2026-08-07</span>
+      <span><b>Client-days</b> <span class="num">8,733</span> with activity</span>
+      <span><b>Clients</b> <span class="num">106</span></span>
+      <span><b>Most recent week</b> omitted</span>
+    </div>
+  </header>
+
+  <section>
+    <div class="ledger">
+      <div>
+        <span class="side-label">Baseline</span>
+        <span class="figure num">$58,531.75</span>
+        <span class="subfig"><span class="num">1,024</span> days out of balance · <span class="num">87.74%</span> clean</span>
+      </div>
+      <div class="arrow" aria-hidden="true">→</div>
+      <div>
+        <span class="side-label">After both fixes</span>
+        <span class="figure after num">$852.38</span>
+        <span class="subfig"><span class="num">105</span> days out of balance · <span class="num">98.74%</span> clean</span>
+      </div>
+    </div>
+
+    <div class="stats">
+      <div class="stat"><span class="k num">919</span><span class="l">client-days brought into balance</span></div>
+      <div class="stat zero"><span class="k num">0</span><span class="l">days knocked out of balance</span></div>
+      <div class="stat zero"><span class="k num">0</span><span class="l">balanced days whose numbers moved</span></div>
+      <div class="stat"><span class="k num">98.5%</span><span class="l">of the dollar variance removed</span></div>
+    </div>
+
+    <div class="measure">
+      <p>Figures exclude the ten deactivated duplicate clients, which the plan says to exclude from reporting. Across the full 8,733 client-days including them the shape is the same: 1,087 → 108 days out of balance, $63,764.92 → $1,995.36.</p>
+    </div>
+  </section>
+
+  <section>
+    <h2>Why this baseline is different</h2>
+    <div class="measure">
+      <p>The earlier thirty-day report derived its baseline arithmetically. This one does not. The job was run three separate times over the same ninety days, against the same data, writing real summaries each time:</p>
+      <ul>
+        <li><strong>Run A — baseline.</strong> The service-charge flag cleared on all 210 clients and <code>get-tip</code> restored to its tendered-only form. This is exactly what production calculates today.</li>
+        <li><strong>Run B — plus R1.</strong> Only <code>get-tip</code> changed.</li>
+        <li><strong>Run C — plus R2.</strong> The service-charge flag switched on as well.</li>
+      </ul>
+      <p>The underlying data — Phase 0, the re-key, the repaired charge references — is identical across all three, so what separates them is the calculation and nothing else. 18,900 summaries were written per run, 56,700 in total.</p>
+      <p>The most recent week (08-08 → 08-14) is excluded deliberately. The backup was cut mid-evening on the 14th, and the days either side of that are partial, which distorted the earlier window.</p>
+    </div>
+  </section>
+
+  <section>
+    <h2>Impact by underlying fix</h2>
+    <div class="measure">
+      <p>Applied in sequence, each measured against the run before it, over 8,350 reportable client-days.</p>
+    </div>
+
+    <div class="step" data-step="1">
+      <h3>R1</h3>
+      <div class="step-title">Tips on untendered orders</div>
+    </div>
+    <div class="scroll">
+      <table>
+        <thead><tr><th>Outcome</th><th class="n">Client-days</th><th class="n">Share</th></tr></thead>
+        <tbody>
+          <tr><td>Unchanged</td><td class="n">8,054</td><td class="n dim">96.5%</td></tr>
+          <tr><td>Out of balance → balanced</td><td class="n good">272</td><td class="n dim">3.3%</td></tr>
+          <tr><td>Balanced → out of balance</td><td class="n good">0</td><td class="n dim">—</td></tr>
+          <tr><td>Balanced → balanced, numbers changed</td><td class="n good">0</td><td class="n dim">—</td></tr>
+          <tr><td>Still out of balance, but closer</td><td class="n">24</td><td class="n dim">0.3%</td></tr>
+          <tr class="total"><td>Days touched · dollars moved</td><td class="n">296</td><td class="n">$3,623.44</td></tr>
+        </tbody>
+      </table>
+    </div>
+
+    <div class="step" data-step="2">
+      <h3>R2</h3>
+      <div class="step-title">Square service charges, both signs</div>
+    </div>
+    <div class="scroll">
+      <table>
+        <thead><tr><th>Outcome</th><th class="n">Client-days</th><th class="n">Share</th></tr></thead>
+        <tbody>
+          <tr><td>Unchanged</td><td class="n">7,701</td><td class="n dim">92.2%</td></tr>
+          <tr><td>Out of balance → balanced</td><td class="n good">647</td><td class="n dim">7.7%</td></tr>
+          <tr><td>Balanced → out of balance</td><td class="n good">0</td><td class="n dim">—</td></tr>
+          <tr><td>Balanced → balanced, numbers changed</td><td class="n good">0</td><td class="n dim">—</td></tr>
+          <tr><td>Still out of balance, but closer</td><td class="n">2</td><td class="n dim">&lt;0.1%</td></tr>
+          <tr class="total"><td>Days touched · dollars moved</td><td class="n">649</td><td class="n">$54,775.79</td></tr>
+        </tbody>
+      </table>
+    </div>
+
+    <h3>Both together, full population</h3>
+    <div class="scroll">
+      <table>
+        <thead><tr><th>Outcome</th><th class="n">Client-days</th></tr></thead>
+        <tbody>
+          <tr><td>Unchanged</td><td class="n">7,749</td></tr>
+          <tr><td>Out of balance → balanced</td><td class="n good">979</td></tr>
+          <tr><td>Balanced → out of balance</td><td class="n good">0</td></tr>
+          <tr><td>Balanced → balanced, numbers changed</td><td class="n good">0</td></tr>
+          <tr><td>Still out of balance, but closer</td><td class="n">5</td></tr>
+          <tr class="total"><td>Days touched · dollars moved</td><td class="n">984 · $62,069.64</td></tr>
+        </tbody>
+      </table>
+    </div>
+
+    <div class="callout">
+      <span class="h">Neither fix touches a healthy day — now confirmed over three times the window</span>
+      <p>Across 8,733 client-days and 90 days of trading, not one day that balanced under production's calculation was altered by either fix: zero knocked out of balance, and zero whose amounts moved while staying balanced. Every day they touched was already out of balance. The thirty-day run found the same thing; this replicates it on a much larger sample.</p>
+    </div>
+  </section>
+
+  <section>
+    <h2>Worked examples</h2>
+    <div class="measure">
+      <p>Each fix is easiest to check on a single order. In both cases the day's imbalance equals the fix amount exactly, which is what you would expect if the fix books something real that was previously booked nowhere.</p>
+    </div>
+
+    <h3>R1 — a tip reversal with no tender to hang it on</h3>
+    <div class="scroll">
+      <table>
+        <thead><tr><th>Field</th><th>Value</th></tr></thead>
+        <tbody>
+          <tr><td>Order</td><td class="mono">square/order/NGLK-SM-OxSX9gpXJV394qqT8mnBGypUwKNZY</td></tr>
+          <tr><td><code>:sales-order/tip</code></td><td class="n">−60.00</td></tr>
+          <tr><td><code>:sales-order/total</code></td><td class="n">−60.00</td></tr>
+          <tr><td><code>:sales-order/charges</code></td><td class="n bad">0 — no tender at all</td></tr>
+          <tr class="total"><td>NGLK 2026-08-04 imbalance</td><td class="n">−60.00 → 0.00</td></tr>
+        </tbody>
+      </table>
+    </div>
+    <div class="measure">
+      <p>A guest's tip was handed back. The reversal is recorded on the order, but <code>get-tip</code> reaches tips by joining through <code>:sales-order/charges</code>, and a return-only order has no charge to join through — so the −$60.00 was invisible and the day credited a tip that no longer existed.</p>
+    </div>
+
+    <h3>R2 — a service charge collected but credited nowhere</h3>
+    <div class="scroll">
+      <table>
+        <thead><tr><th>Field</th><th>Value</th></tr></thead>
+        <tbody>
+          <tr><td>Order</td><td class="mono">square/order/NTPT-PT-KrMZzcon1cpQEJUyetErkBIpcdEZY</td></tr>
+          <tr><td><code>:sales-order/service-charge</code></td><td class="n">427.10</td></tr>
+          <tr><td><code>:sales-order/total</code></td><td class="n">3,198.78</td></tr>
+          <tr class="total"><td>NTPT 2026-08-06 imbalance</td><td class="n">+427.10 → 0.00</td></tr>
+        </tbody>
+      </table>
+    </div>
+    <div class="measure">
+      <p>The $427.10 sits inside the card tender the customer paid, so it lands on the debit side — but nothing credited it, leaving the day short by exactly that amount. R2 is the larger of the two fixes by an order of magnitude: $54,775.79 moved against R1's $3,623.44.</p>
+    </div>
+  </section>
+
+  <section>
+    <h2>What is left</h2>
+    <div class="measure">
+      <p>Fifteen client-days above the ten-cent materiality threshold, out of 8,350. Everything else — 90 client-days — totals <strong>$2.26</strong>.</p>
+    </div>
+    <div class="scroll">
+      <table>
+        <thead><tr><th>Client</th><th>Date</th><th class="n">Variance</th><th>Note</th></tr></thead>
+        <tbody>
+          <tr><td class="mono">NGBK</td><td>2026-08-06</td><td class="n">+299.42</td><td>tender exceeds order totals by exactly this much</td></tr>
+          <tr><td class="mono">NGMV</td><td>2026-05-24</td><td class="n">+100.87</td><td rowspan="5" class="dim">a five-day cluster in late May, new to this window and not yet diagnosed</td></tr>
+          <tr><td class="mono">NGMV</td><td>2026-05-20</td><td class="n">+72.05</td></tr>
+          <tr><td class="mono">NGMV</td><td>2026-05-21</td><td class="n">+57.64</td></tr>
+          <tr><td class="mono">NGMV</td><td>2026-05-22</td><td class="n">+14.41</td></tr>
+          <tr><td class="mono">NGMV</td><td>2026-05-26</td><td class="n">+14.41</td></tr>
+          <tr><td class="mono">NGEB</td><td>2026-05-19</td><td class="n">−79.01</td><td rowspan="4" class="dim">ezCater fee semantics — the plan's open question §15.4</td></tr>
+          <tr><td class="mono">NGEB</td><td>2026-06-23</td><td class="n">−50.28</td></tr>
+          <tr><td class="mono">NGEB</td><td>2026-05-13</td><td class="n">−49.80</td></tr>
+          <tr><td class="mono">NGEB</td><td>2026-07-29</td><td class="n">−20.00</td></tr>
+          <tr><td class="mono">NGDA</td><td>2026-08-01</td><td class="n">−50.00</td><td>auto-gratuity booked as a service charge</td></tr>
+          <tr><td class="mono">N-30012</td><td>2026-05-20</td><td class="n">+17.32</td><td rowspan="2" class="dim">two consecutive days, undiagnosed</td></tr>
+          <tr><td class="mono">N-30012</td><td>2026-05-21</td><td class="n">+12.99</td></tr>
+          <tr><td class="mono">NG4S</td><td>2026-05-29</td><td class="n">−11.78</td><td class="dim">undiagnosed</td></tr>
+          <tr><td class="mono">PNSP</td><td>2026-07-12</td><td class="n">−0.14</td><td class="dim">register rounding, just over threshold</td></tr>
+        </tbody>
+      </table>
+    </div>
+    <div class="measure">
+      <p>Widening from thirty days to ninety surfaced two clusters the shorter window could not see: <strong>NGMV</strong> across five days in late May, and <strong>NGEB</strong> across four days spanning May to July. NGEB is the known ezCater fee-semantics question. NGMV and N-30012 are new and worth a look before this ships — they are the kind of thing only a longer baseline exposes.</p>
+      <p>The ten deactivated twins, excluded above, contribute a further $1,142.98 across three days. They should not be reported on at all once one client per location is settled.</p>
+    </div>
+  </section>
+
+  <section>
+    <h2>Two performance defects found on the way</h2>
+    <div class="measure">
+      <p>Running the job 56,700 times surfaced two problems that a normal nightly pass would hide, both now fixed or documented.</p>
+    </div>
+    <div class="scroll">
+      <table>
+        <thead><tr><th>Problem</th><th>Effect</th><th>Fix</th></tr></thead>
+        <tbody>
+          <tr><td><code>dirty-sales-summaries</code> scanned the whole index</td><td class="n">1,321 ms → 5.6 ms per client</td><td>bound the scan at the client boundary</td></tr>
+          <tr><td>Transactor sized for a toy database</td><td class="n">~35/min → 5,300/min</td><td>object cache 2 GB → 8 GB, heap 4 GB → 16 GB</td></tr>
+        </tbody>
+      </table>
+    </div>
+    <div class="measure">
+      <p>The second is a deployment setting rather than a code change, but it is dramatic: with a 2 GB object cache against a 27 GB database, the final 7,958 client-days of a pass were crawling at about 35 a minute. After resizing they completed in 90 seconds. Worth checking what production's transactor is sized at.</p>
+    </div>
+  </section>
+
+  <section>
+    <h2>Reproduce it</h2>
+    <div class="scroll">
+      <pre><span class="dim">;; connect to the restored backup</span>
+(def conn (d/connect "datomic:dev://localhost:4337/integreat-prod-restore"))
+
+<span class="dim">;; baseline mode: clear the flag, restore tendered-only tips</span>
+(doseq [[c _] clients]
+  @(d/transact conn [[:db/retract c :client/feature-flags "summary-service-charges"]]))
+(alter-var-root #'ss/get-tip (constantly baseline-get-tip))
+
+<span class="dim">;; mark the window and run the real job, then capture</span>
+(doseq [[c _] clients] (ss/mark-dirty c w90-start w90-end))
+(pmap (fn [[c code]] (ss/refresh-client! c code)) pending)
+
+<span class="dim">;; the single orders behind the worked examples</span>
+(d/pull (d/db conn) '[*]
+        [:sales-order/external-id
+         "square/order/NGLK-SM-OxSX9gpXJV394qqT8mnBGypUwKNZY"])
+(d/pull (d/db conn) '[*]
+        [:sales-order/external-id
+         "square/order/NTPT-PT-KrMZzcon1cpQEJUyetErkBIpcdEZY"])</pre>
+    </div>
+    <div class="measure">
+      <p>The unit tests covering both fixes and the re-key:</p>
+    </div>
+    <div class="scroll">
+      <pre>lein test auto-ap.jobs.sales-summaries-test auto-ap.square.core3-test
+<span class="dim">;; =&gt; 15 tests, 26 assertions, 0 failures</span></pre>
+    </div>
+    <div class="callout warn">
+      <span class="h">Two defects still open before this ships</span>
+      <p>Re-keying a charge that has two parent orders duplicates the tender, because the other client's import then matches neither key and creates a second charge which cardinality-many appends. The migration must split shared charges first. Separately, <code>remove-voided-orders</code> retracts orders, and 63% of the contended clients' charges have two parents, so a retraction there deletes the other client's payment.</p>
+    </div>
+  </section>
+
+  <footer>
+    <span>Measured 2026-08-15 on <span class="mono">integreat-prod-restore</span>, restored from backup point 209608347. Nothing in production was touched. Branch <span class="mono">worktree-sales-summary-balance</span>.</span>
+    <span>Service charges map to account 49000 Service Income, chosen to make measurement possible and still pending accounting sign-off. The account affects only whether a day can reach accepted — never whether it balances — so no figure here depends on it.</span>
+  </footer>
+
+</div>

src/clj/auto_ap/jobs/compare_sales_summaries.clj

@@ -0,0 +1,135 @@
+(ns auto-ap.jobs.compare-sales-summaries
+  "Compares sales summaries between two points in the same database.
+
+  Datomic keeps every past value, so a recompute can be audited against exactly what was there
+  before by reading `(d/as-of db t)` for some earlier `t` — no snapshot or scratch copy needed.
+
+  The question this exists to answer is narrower than \"did the totals improve\": it is *which
+  days changed, and were any of them already balanced*. A day that was balanced before and still
+  balances after can still have had its line amounts move, and that is a real change to the
+  books even though no red turns green. Counting only balanced/unbalanced transitions would hide
+  it entirely."
+  (:require
+   [auto-ap.datomic :refer [conn]]
+   [auto-ap.datomic.sales-summaries :as d-ss]
+   [clj-time.coerce :as c]
+   [datomic.api :as dc]))
+
+(def item-read
+  [:sales-summary-item/category
+   :sales-summary-item/manual?
+   :ledger-mapped/amount
+   {:ledger-mapped/ledger-side [:db/ident]}
+   {:ledger-mapped/account [:account/numeric-code]}])
+
+(defn- cents
+  "Amounts are doubles carrying float noise, so compare them at the cent — the unit the books are
+  actually kept in. Without this, 182.87000000000003 and 182.87 read as a change."
+  [x]
+  (Math/round (* 100.0 (double (or x 0.0)))))
+
+(defn- line
+  "One item reduced to what a reader would call \"the number\": category, side, amount, account."
+  [item]
+  {:category (:sales-summary-item/category item)
+   :side (get-in item [:ledger-mapped/ledger-side :db/ident])
+   :cents (cents (:ledger-mapped/amount item))
+   :account (get-in item [:ledger-mapped/account :account/numeric-code])})
+
+(defn summaries-in
+  "`{[client-code date] {:lines … :imbalance … :balanced?}}` for every summary in `[start end)`.
+
+  Keyed by client code and date rather than entity id so the two sides line up even if an entity
+  were recreated between the points being compared."
+  [db start end]
+  (->> (dc/q {:find [(list 'pull '?s [:sales-summary/date
+                                      {:sales-summary/client [:client/code]}
+                                      {:sales-summary/items item-read}])]
+              :in '[$ ?start ?end]
+              :where '[[?s :sales-summary/date ?d]
+                       [(>= ?d ?start)]
+                       [(< ?d ?end)]]}
+             db (c/to-date start) (c/to-date end))
+       (map first)
+       (reduce (fn [acc s]
+                 (let [items (map d-ss/<-pulled-item (:sales-summary/items s))]
+                   (assoc acc
+                          [(get-in s [:sales-summary/client :client/code]) (:sales-summary/date s)]
+                          {:lines (frequencies (map line (:sales-summary/items s)))
+                           :imbalance (d-ss/imbalance items)
+                           :balanced? (d-ss/balanced? items)})))
+               {})))
+
+(defn- classify
+  "How one client-day differs. `:numbers-changed` is the interesting one — the lines themselves
+  moved, whether or not the day's balance status did."
+  [before after]
+  (cond
+    (nil? before) :added
+    (nil? after) :removed
+    (= (:lines before) (:lines after)) :identical
+    :else :numbers-changed))
+
+(defn compare-window
+  "Compares every summary in `[start end)` between two database values.
+
+  Returns per-day rows plus the tallies worth reporting, including the one that is easy to miss:
+  days that were **already balanced** and whose numbers moved anyway."
+  [before-db after-db start end]
+  (let [before (summaries-in before-db start end)
+        after (summaries-in after-db start end)
+        rows (for [k (distinct (concat (keys before) (keys after)))
+                   :let [b (get before k) a (get after k)]]
+               {:client (first k)
+                :date (second k)
+                :change (classify b a)
+                :was-balanced? (:balanced? b)
+                :now-balanced? (:balanced? a)
+                :before-imbalance (:imbalance b)
+                :after-imbalance (:imbalance a)
+                :lines-before (:lines b)
+                :lines-after (:lines a)})
+        rows (vec rows)
+        changed (filter #(= :numbers-changed (:change %)) rows)]
+    {:rows rows
+     :tally {:compared (count rows)
+             :identical (count (filter #(= :identical (:change %)) rows))
+             :numbers-changed (count changed)
+             :added (count (filter #(= :added (:change %)) rows))
+             :removed (count (filter #(= :removed (:change %)) rows))}
+     :balance-transitions
+     {:unbalanced->balanced (count (filter #(and (false? (:was-balanced? %)) (true? (:now-balanced? %))) rows))
+      :balanced->unbalanced (count (filter #(and (true? (:was-balanced? %)) (false? (:now-balanced? %))) rows))
+      :stayed-balanced (count (filter #(and (true? (:was-balanced? %)) (true? (:now-balanced? %))) rows))
+      :stayed-unbalanced (count (filter #(and (false? (:was-balanced? %)) (false? (:now-balanced? %))) rows))}
+     :previously-balanced-and-changed
+     (->> changed (filter :was-balanced?) vec)}))
+
+(defn line-diff
+  "Which categories actually moved on one row, as `{category [before-cents after-cents]}`. For
+  reading a handful of rows by hand once the tallies point at them."
+  [row]
+  (let [by-cat (fn [lines] (reduce (fn [m [l n]] (assoc m (:category l) (* n (:cents l)))) {} lines))
+        b (by-cat (:lines-before row))
+        a (by-cat (:lines-after row))]
+    (->> (distinct (concat (keys b) (keys a)))
+         (keep (fn [cat]
+                 (let [x (get b cat 0) y (get a cat 0)]
+                   (when (not= x y) [cat [(/ x 100.0) (/ y 100.0)]]))))
+         (into {}))))
+
+(defn compare-against
+  "Convenience: compare the current database against its own past value at basis `t`."
+  [t start end]
+  (let [db (dc/db conn)]
+    (compare-window (dc/as-of db t) db start end)))
+
+(comment
+  ;; the restore point, i.e. production's own summaries before any of this work
+  (def result (compare-against 209608347
+                               (clj-time.core/date-time 2026 7 15)
+                               (clj-time.core/date-time 2026 8 14)))
+  (:tally result)
+  (:balance-transitions result)
+  (count (:previously-balanced-and-changed result))
+  (map line-diff (take 3 (:previously-balanced-and-changed result))))

src/clj/auto_ap/jobs/rekey_square_external_ids.clj

@@ -0,0 +1,106 @@
+(ns auto-ap.jobs.rekey-square-external-ids
+  "One-shot migration re-keying Square refunds and charges to client-scoped external ids.
+
+  Refund and charge keys carry no client scoping today, so two clients configured on the same
+  Square location share a single entity: the refund's owner flips every time either client
+  imports, and one charge ends up referenced by both clients' orders. Sales orders already scope
+  their keys by client and location; this brings the other two in line.
+
+  Run AFTER the importer knows how to resolve both key schemes (`square.core3/existing-id`).
+  Running it first would be harmless, but the importer would then re-create legacy-keyed
+  entities on its next pass.
+
+  The migration is idempotent: an entity already carrying its scoped key is skipped, so it can
+  be re-run over a partially migrated database."
+  (:require
+   [auto-ap.datomic :refer [conn]]
+   [auto-ap.logging :as alog]
+   [datomic.api :as dc]))
+
+(def refund-prefix "square/refund/")
+(def charge-prefix "square/charge/")
+
+(defn- scope-of
+  "`[client-code location]` for an entity, or nil when it cannot be determined.
+
+  Charges are the awkward case: about an eighth of them carry neither `:charge/client` nor
+  `:charge/location`. Those are stubs minted by the payout path, which asserts an external id
+  alone and lets unique-identity upsert bring a bare entity into being, plus older tender
+  records that predate the client attribute. None are orphaned, so the scope is recovered from
+  whatever references them — the sales order first, then the expected deposit."
+  [db attr e]
+  (let [ent (dc/entity db e)
+        pair (fn [code loc] (when (and code loc) [code loc]))]
+    (or (case attr
+          :sales-refund/external-id (pair (:client/code (:sales-refund/client ent))
+                                          (:sales-refund/location ent))
+          :charge/external-id (pair (:client/code (:charge/client ent))
+                                    (:charge/location ent)))
+        (when-let [o (:e (first (dc/datoms db :vaet e :sales-order/charges)))]
+          (let [oe (dc/entity db o)]
+            (pair (:client/code (:sales-order/client oe)) (:sales-order/location oe))))
+        (when-let [d (:e (first (dc/datoms db :vaet e :expected-deposit/charges)))]
+          (let [de (dc/entity db d)]
+            (pair (:client/code (:expected-deposit/client de)) (:expected-deposit/location de)))))))
+
+(defn planned-key
+  "`[eid new-key]` for an entity that still needs re-keying, or nil when it is already scoped or
+  cannot be scoped at all.
+
+  Detection compares against the key this entity *should* have rather than pattern-matching the
+  id, because Square ids may themselves contain dashes and no pattern separates the two schemes
+  reliably. That also makes the migration idempotent."
+  [db attr prefix datom]
+  (let [old (:v datom)]
+    (when-let [[code loc] (scope-of db attr (:e datom))]
+      (let [scoped-prefix (str prefix code "-" loc "-")]
+        (when-not (.startsWith ^String old scoped-prefix)
+          [(:e datom) (str scoped-prefix (subs old (count prefix)))])))))
+
+(defn plan
+  "Everything the migration would change, plus what it cannot touch. Read-only — run this and
+  check `:collisions` is empty before transacting anything."
+  [db attr prefix]
+  (let [acc (reduce (fn [acc d]
+                      (let [acc (update acc :total inc)]
+                        (if-let [[e new-key] (planned-key db attr prefix d)]
+                          (-> acc
+                              (update :to-migrate inc)
+                              (update :new-keys conj! [e new-key]))
+                          (if (scope-of db attr (:e d))
+                            (update acc :already-scoped inc)
+                            (update acc :unscopable inc)))))
+                    {:total 0 :to-migrate 0 :already-scoped 0 :unscopable 0 :new-keys (transient [])}
+                    (dc/datoms db :aevt attr))]
+    (update acc :new-keys persistent!)))
+
+(defn collisions
+  "Any two entities that would land on the same new key. Must be empty: a collision would merge
+  two entities into one and lose whichever lost."
+  [new-keys]
+  (->> new-keys
+       (group-by second)
+       (keep (fn [[k es]] (when (> (count es) 1) [k (mapv first es)])))
+       vec))
+
+(defn migrate!
+  "Asserts the new external id on each planned entity. The attribute is cardinality one, so the
+  legacy value is retracted by the same assertion and the entity keeps its identity — nothing is
+  created and nothing is deleted.
+
+  Returns the number of entities re-keyed."
+  [attr new-keys batch-size]
+  (let [total (count new-keys)]
+    (alog/info ::migrating :attr attr :count total)
+    (doseq [[i batch] (map-indexed vector (partition-all batch-size new-keys))]
+      @(dc/transact conn (for [[e new-key] batch] {:db/id e attr new-key}))
+      (when (zero? (mod i 20))
+        (alog/info ::migrated :attr attr :done (* i batch-size) :of total)))
+    total))
+
+(defn counts
+  "Entity totals, for the before/after assertion that is this migration's real safety net: if
+  either number moves, the re-key created duplicates instead of updating in place."
+  [db]
+  {:refunds (reduce (fn [n _] (inc n)) 0 (dc/datoms db :aevt :sales-refund/external-id))
+   :charges (reduce (fn [n _] (inc n)) 0 (dc/datoms db :aevt :charge/external-id))})

src/clj/auto_ap/jobs/sales_summaries.clj

@@ -62,15 +62,24 @@
     {:ledger-mapped/ledger-side [:db/ident]}
     {:ledger-mapped/account [:db/id]}])
 
-(defn dirty-sales-summaries [c]
+(defn dirty-sales-summaries
+  "The client's dirty summaries, with enough of each item to evaluate and re-transact it.
+
+  `index-pull` returns a lazy seq running from `:start` to the END of the index, so this must
+  stop at the client boundary rather than filter: `:sales-summary/client+dirty` sorts by client
+  first, so every later client's summaries sit beyond this client's and filtering would walk all
+  of them — for every client — pulling their items on the way. That is quadratic in the number of
+  summaries, and it showed up as a full refresh degrading from ~180 client-days a minute to ~3 as
+  the summary count grew."
+  [c]
   (let [client-id (dc/entid (dc/db conn) c)]
     (->> (dc/index-pull  (dc/db conn)
                          {:index :avet
                           :selector (conj '[:sales-summary/date :sales-summary/client :db/id]
                                           {:sales-summary/items item-read})
                           :start [:sales-summary/client+dirty [client-id true]]})
-         (filter (fn [sales-summary]
-                   (= client-id (:db/id (:sales-summary/client sales-summary))))))))
+         (take-while (fn [sales-summary]
+                       (= client-id (:db/id (:sales-summary/client sales-summary))))))))
 
 (def default-days
   "How far back the scheduled refresh looks for summaries that still need recomputing."
@@ -454,12 +463,14 @@
               :when (not (zero? amount))]
           [category amount])))
 
-(defn sales-summaries-v2 []
-  (doseq [[c client-code] (dc/q '[:find ?c ?client-code
-                                  :in $
-                                  :where [?c :client/code ?client-code]]
-                                (dc/db conn))
-          {:sales-summary/keys [date] :db/keys [id] :as existing-summary}  (dirty-sales-summaries c)]
+(defn refresh-client!
+  "Recomputes every dirty summary for one client.
+
+  Split out of the driver loop so a client's work stands on its own: it can be run for a single
+  client, and a backfill over the whole history can spread clients across threads instead of
+  grinding through the largest ones one day at a time."
+  [c client-code]
+  (doseq [{:sales-summary/keys [date] :db/keys [id] :as existing-summary} (dirty-sales-summaries c)]
     (mu/with-context {:client-code client-code
                       :date date}
       (alog/info ::updating)
@@ -498,6 +509,16 @@
             @(dc/transact conn [[:upsert-sales-summary result]]))
           @(dc/transact conn [{:db/id id :sales-summary/dirty false}]))))))
 
+
+(defn sales-summaries-v2
+  "Recomputes every dirty summary, client by client."
+  []
+  (doseq [[c client-code] (dc/q '[:find ?c ?client-code
+                                  :in $
+                                  :where [?c :client/code ?client-code]]
+                                (dc/db conn))]
+    (refresh-client! c client-code)))
+
 (defn reset-summaries []
   @(dc/transact conn (->> (dc/q '[:find ?sos
                                   :in $

src/clj/auto_ap/square/core3.clj

@@ -269,6 +269,29 @@
    0.0
    []))
 
+(defn scoped-key
+  "Client-scoped external id, in the shape sales order keys already use.
+
+  Without the client and location in the key, two clients configured on the same Square location
+  collide on a single entity: a refund changes owner every time either client imports, and one
+  charge ends up shared between both clients' orders."
+  [prefix client location id]
+  (str prefix (:client/code client) "-" (:square-location/client-location location) "-" id))
+
+(defn existing-id
+  "Entity id of the refund or charge this id already refers to, trying the client-scoped key
+  first and the legacy unscoped key second.
+
+  This is what makes re-keying safe. These external ids are `:db.unique/identity`, so the import
+  relies on upsert-by-identity; changing the key format on its own would match nothing and
+  Datomic would create a SECOND entity for every refund and charge, orphaning the original under
+  its legacy key. Pinning the result as `:db/id` makes the write land on the existing entity
+  whichever scheme it currently carries."
+  [db attr prefix client location id]
+  (when id
+    (or (dc/entid db [attr (scoped-key prefix client location id)])
+        (dc/entid db [attr (str prefix id)]))))
+
 (defn tender->charge [order client location t]
   (remove-nils
    #:charge
@@ -278,8 +301,9 @@
      :note (:note t)
      :location       (:square-location/client-location location)
      :reference-link (str (url/url  "https://squareup.com/receipt/preview" (:id t)))
+     :db/id       (existing-id (dc/db conn) :charge/external-id "square/charge/" client location (:id t))
      :external-id (when (:id t)
-                    (str "square/charge/" (:id t)))
+                    (scoped-key "square/charge/" client location (:id t)))
      :processor   (cond 
                     (#{"OTHER" "THIRD_PARTY_CARD"} (:type t))
                     (condp = (some-> (:note t) str/lower-case)
@@ -561,7 +585,11 @@
                                                                      (coerce/to-date)))
                                                 :charges     (reverse (->> (:payout_entries payout)
                                                                            (filter (comp :payment_id :type_charge_details))
-                                                                           (map (fn [p] {:charge/external-id (str "square/charge/" (:payment_id (:type_charge_details p)))}))))})
+                                                                           (map (fn [p]
+                                                                                  (let [payment-id (:payment_id (:type_charge_details p))]
+                                                                                    (remove-nils
+                                                                                     {:charge/external-id (scoped-key "square/charge/" client location payment-id)
+                                                                                      :db/id              (existing-id (dc/db conn) :charge/external-id "square/charge/" client location payment-id)}))))))})
                           (filter :expected-deposit/date)
                           (into []))
                      (catch Throwable e
@@ -585,7 +613,8 @@
                              (de/chain
                               (get-payment client (:payment_id r))
                               (fn [payment]
-                                #:sales-refund {:external-id (str "square/refund/" (:id r))
+                                #:sales-refund {:db/id       (existing-id (dc/db conn) :sales-refund/external-id "square/refund/" client l (:id r))
+                                                :external-id (scoped-key "square/refund/" client l (:id r))
                                                 :vendor      :vendor/ccp-square
                                                 :total       (amount->money (:amount_money r))
                                                 :fee         (transduce

test/clj/auto_ap/jobs/sales_summaries_test.clj

@@ -120,3 +120,27 @@
                                  {:sales-order/external-id "ezcater/order/TEST-ezcater-vendorless"
                                   :sales-order/service-charge -75.0})])
       (is (nil? (service-charges-for test-client-id))))))
+
+(deftest dirty-summaries-stop-at-the-client-boundary
+  (testing "every dirty day for the client is returned, and none belonging to another client.
+
+           :sales-summary/client+dirty sorts by client, so an unbounded index scan would walk
+           every later client's summaries too — correct, but quadratic in the summary count."
+    (let [{:strs [test-client-id]} (setup-test-data [])
+          other (get-in @(dc/transact conn [{:db/id "other" :client/code (str "OTHER" (rand-int 100000))}])
+                        [:tempids "other"])
+          day (fn [client d dirty?]
+                {:sales-summary/client client
+                 :sales-summary/date d
+                 :sales-summary/dirty dirty?})]
+      @(dc/transact conn [(day test-client-id #inst "2026-08-01T07:00:00.000-00:00" true)
+                          (day test-client-id #inst "2026-08-02T07:00:00.000-00:00" true)
+                          (day test-client-id #inst "2026-08-03T07:00:00.000-00:00" false)
+                          (day other #inst "2026-08-01T07:00:00.000-00:00" true)
+                          (day other #inst "2026-08-02T07:00:00.000-00:00" true)])
+      (let [mine (sut/dirty-sales-summaries test-client-id)]
+        (is (= 2 (count mine)) "both dirty days, and not the clean one")
+        (is (every? #(= test-client-id (:db/id (:sales-summary/client %))) mine)
+            "and nothing belonging to the other client"))
+      (is (= 2 (count (sut/dirty-sales-summaries other)))
+          "the other client's own dirty days are still found"))))

test/clj/auto_ap/square/core3_test.clj

@@ -0,0 +1,77 @@
+(ns auto-ap.square.core3-test
+  (:require
+   [auto-ap.datomic :refer [conn]]
+   [auto-ap.integration.util :refer [setup-test-data wrap-setup]]
+   [auto-ap.square.core3 :as sut]
+   [clojure.test :refer [deftest is testing use-fixtures]]
+   [datomic.api :as dc]))
+
+(use-fixtures :each wrap-setup)
+
+(def client {:client/code "NGCD"})
+(def location {:square-location/client-location "CD"})
+
+(defn- refund-count []
+  (count (dc/q '[:find ?e :where [?e :sales-refund/external-id]] (dc/db conn))))
+
+(defn- resolve-refund [id]
+  (sut/existing-id (dc/db conn) :sales-refund/external-id "square/refund/" client location id))
+
+(deftest scoped-key-carries-client-and-location
+  (testing "the same shape sales order keys already use, so a shared location cannot contend"
+    (is (= "square/refund/NGCD-CD-abc" (sut/scoped-key "square/refund/" client location "abc")))
+    (is (= "square/charge/NGCD-CD-xyz" (sut/scoped-key "square/charge/" client location "xyz")))))
+
+(deftest legacy-keyed-entity-is-updated-not-duplicated
+  (testing "an entity still carrying its unscoped key is found and re-keyed in place.
+
+           This is the sharpest hazard in the migration: these external ids are
+           :db.unique/identity, so writing the new key without resolving the old one first
+           matches nothing and creates a second entity, orphaning the original."
+    (setup-test-data [])
+    @(dc/transact conn [{:db/id "r"
+                         :sales-refund/external-id "square/refund/abc"
+                         :sales-refund/total 10.0}])
+    (is (= 1 (refund-count)))
+    (let [eid (resolve-refund "abc")]
+      (is (some? eid) "resolves an entity carrying the legacy key")
+      @(dc/transact conn [{:db/id eid
+                           :sales-refund/external-id (sut/scoped-key "square/refund/" client location "abc")
+                           :sales-refund/total 10.0}])
+      (is (= 1 (refund-count)) "no second entity was created")
+      (is (= eid (dc/entid (dc/db conn) [:sales-refund/external-id "square/refund/NGCD-CD-abc"]))
+          "the same entity now answers to the scoped key")
+      (is (nil? (dc/entid (dc/db conn) [:sales-refund/external-id "square/refund/abc"]))
+          "and no longer to the legacy one"))))
+
+(deftest already-scoped-entity-resolves-by-its-new-key
+  (testing "re-running the importer after migration finds the entity by the scoped key, so the
+           migration is not undone and nothing is duplicated"
+    (setup-test-data [])
+    @(dc/transact conn [{:db/id "r"
+                         :sales-refund/external-id "square/refund/NGCD-CD-abc"
+                         :sales-refund/total 10.0}])
+    (is (= (dc/entid (dc/db conn) [:sales-refund/external-id "square/refund/NGCD-CD-abc"])
+           (resolve-refund "abc")))
+    (is (= 1 (refund-count)))))
+
+(deftest unknown-id-resolves-to-nothing
+  (testing "a refund never seen before has no id to pin, so the importer creates it fresh"
+    (setup-test-data [])
+    (is (nil? (resolve-refund "never-seen")))))
+
+(deftest two-clients-on-one-location-get-their-own-entities
+  (testing "the point of the re-key: with the client in the key, a second client importing the
+           same Square refund creates its own entity instead of taking ownership of the first"
+    (setup-test-data [])
+    (let [other {:client/code "NGCC"}
+          other-loc {:square-location/client-location "CC"}]
+      @(dc/transact conn [{:db/id "r"
+                           :sales-refund/external-id (sut/scoped-key "square/refund/" client location "shared")
+                           :sales-refund/total 10.0}])
+      (is (nil? (sut/existing-id (dc/db conn) :sales-refund/external-id "square/refund/" other other-loc "shared"))
+          "the second client does not resolve onto the first client's entity")
+      @(dc/transact conn [{:db/id "r2"
+                           :sales-refund/external-id (sut/scoped-key "square/refund/" other other-loc "shared")
+                           :sales-refund/total 10.0}])
+      (is (= 2 (refund-count)) "two stable entities, one per client, rather than one that flips"))))