Rebuilt the whole analysis from nothing: fresh restore of backup point
209608347, deactivate the ten shared locations, re-key and split every
one of 19,040,785 orders, re-import from Square, then two full ninety-day
recomputes — one with the fixes off, one with them on.
The baseline is now a no-fix recompute rather than production's stored
summaries. That is the stricter comparison: production's figures are in
places months stale, and crediting the fixes with repairing ordinary
staleness flattered them. On the fairer footing the two arithmetic fixes
are worth 979 client-days and $61,769.56, taking the window from 1,258
days out of balance ($69,560.10) to 279 ($7,790.54), with zero days
knocked out of balance and zero already-balanced days altered at line
level.
The migration now runs to completion database-wide: 17,047,142 payments
scoped, nothing left to rename, nothing unscopable, and no payment owned
by more than one order across 400,000 orders checked. The earlier
"transactor-bound, two days" diagnosis was wrong — the bottleneck was GC
in the driving process; the full pass takes about thirteen minutes.
Also corrects compare-sales-summaries: :ledger-mapped/amount, ledger-side
and account are :db/noHistory, so as-of cannot recover past amounts and
a rewritten summary reads back as a false balanced day. Every figure in
the report comes from live captures taken straight after each pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Replaces the partial-completeness section with the measured end state.
card payments 17,046,418 scoped · 0 unscoped · 0 unscopable
refunds 51,986 scoped · 0 unscoped
payouts 144,652 scoped · 0 unscoped · 36 with no owner
cash shifts 69,291 scoped · 0 unscoped
Payments shared between two different clients: 0 in a 250,000-order sample
spread across the database. 104 remain shared between two orders of the same
client, left that way on purpose — Square split one tender across two of that
client's orders, both orders compute the same name so there is no second name a
copy could take, and a copy would double the client's takings. One payment
covering both is the truthful record.
That is what makes the remove-voided-orders guard a required companion rather
than belt-and-braces: the cascade still reaches those 104.
Records that a second complete pass over all 19,040,296 orders took 7.8 minutes
and changed nothing, so the migration is stoppable, resumable and repeatable —
and that with the gate now reading zero, the importer's understanding of the old
name form can be removed, which is the one remaining step.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The report implied the shared-payment problem was solved. Measuring the whole
database shows it is solved for refunds, payouts and shifts, and 1.6% solved for
card payments:
refunds 51,986 scoped · 0 unscoped
payouts 144,652 scoped · 0 unscoped · 36 with no owner
cash-drawer shifts 69,291 scoped · 0 unscoped
card payments 259,763 scoped · 14,241,890 unscoped · 2,122,161 no owner
Sampling 2% of the orders belonging to every client that has ever contended
found 12,954 payments still owned by two orders, extrapolating to roughly
650,000 across that population.
So the balance figures are what the calculation fixes are worth on data where
the sharing had been resolved for the clients and period measured — not a claim
that the database is clean. The report now says so, and the headline stat that
claimed zero shared payments has been replaced with the 1.6% actually renamed.
Records what finishing requires: renaming is driven from orders, so it means
walking all 19,040,296 of them rather than the clients that look shared today,
since nine pairs contended in the past without sharing a location now. Measured
at ~15,000 orders/hour once the 29 GB database stopped fitting in memory — the
same code did 267,012 in 100 seconds with a warm cache — so it belongs in a
maintenance window on sized hardware. It is resumable and a second pass is a
no-op.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The report explained the faults and what repairing them was worth, but never
what the repair actually is. Adds a section showing the real code for each of
the five changes, with the reasoning that is not obvious from reading it.
1 put the client in the record's name — scoped-key, five call sites
2 find the existing record under either name before writing — existing-id,
which is what makes the rename safe to deploy and why the totals did not
double across 213,943 renamed records
3 give every order its own payment record — what :keep and :clone do, and
why the Square id must be recovered from the record's owner rather than by
trimming a prefix, since client codes like N-30003 contain dashes
4 add untendered tips rather than replacing the calculation, because where an
order does have a payment the payment is the correct source
5 credit service charges, with both branches of the vendor test explained —
ezCater commission must be excluded, but whole eras of Square orders carry
no vendor at all, so a test on vendor alone credits nothing
Plus the four supporting changes and why the schema install order mattered
enough to block every test in the suite.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Deletes and restores the database to backup point 209608347, then runs the whole
pipeline on untouched production data with the corrected split: deactivate the
duplicate client per location, give every order its own payment record,
client-scope refunds, payouts and cash-drawer shifts, import live from Square,
and recompute ninety days at each stage.
production today 1,280 days off, 85.34% clean, $75,228.78
+ deduplication 1,087 days off, 87.55% clean, $63,764.92
+ refunded tips 800 days off, 90.84% clean, $60,619.21
+ service charges 108 days off, 98.76% clean, $1,995.36
1,172 client-days brought into balance, 0 knocked out. Neither arithmetic fix
altered a single figure on a day that already balanced. Excluding the ten
deactivated duplicates: 1,217 days and $69,995.61 down to 105 days and $852.38.
The split does what it was missing before: 189,167 payments re-keyed, 77,599
copied so each order owns one, and payments with two parent orders went 11,469
to 0 — holding at 0 after a live import across 48,362 orders. That closes the
component-cascade risk in remove-voided-orders.
Report rewritten for both audiences: what each fault meant for the books, then
the mechanism for engineers, with worked line-level examples, the verification
checks, the residual, and the open decisions. Includes the double-scoping bug
this work introduced and how measurement caught it.
23 tests, 55 assertions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The verification recompute finished. Comparing every summary in the 90-day
window against the run taken before the payout and cash-shift re-key:
18,900 compared, 0 differing, largest difference 0.000000
Supersedes the partial 12,044 figure reported while the pass was still running.
Re-keying 213,943 entities moved no money anywhere, to the last decimal place.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The report asserted that the fixes work; it did not show what changed on the
page. Replaces the worked-examples section with actual summary lines pulled from
the restore, before and after, including debit and credit totals.
NGLK 2026-08-04 is the clearest case. The day already carried a $60.00 card
refund — the guest was given their money back, tip included — while the Tip line
still credited $482.94, because the reversal sits on an order with no tender and
get-tip only reached tips through tenders. Corrected to $422.94, matching the
refund to the cent, and the day closes from -60.00 to 0.00.
NTPT 2026-08-06 shows the other shape: $427.10 of service charge arrived inside
a $4,975.89 card tender, so it landed on the debit side with nothing crediting
it. Credits go 7,350.10 to 7,777.20 against unchanged debits of 7,777.20.
Adds the six largest repairs of each kind with their real Tip and Service
Charges figures, and NGNP 2026-06-25 where both fixes land on one day and pull
in opposite directions — 301.40 credited, 1.80 removed, 299.60 closed — which is
a useful check that the two are independent.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds a [client location start end] arity to upsert-cash-shifts, matching
upsert-payouts, so a backfill can cover a chosen window rather than the fixed
75-day default.
Re-keyed and re-imported the two newly scoped types on the restore, then checked
what it cost:
144,652 expected deposits and 69,291 cash drawer shifts re-keyed
entity counts unchanged on all four Square types, 0 collisions, 0 legacy keys
live re-import over 66 of 102 clients wrote NOTHING — no entities created and
no values rewritten, because every record resolved to the importing client's
own entity through the legacy-key fallback
0 ownership changes since the re-key, across deposits, shifts and refunds
12,044 summaries recomputed afterwards and compared: 0 differences
So re-keying 213,943 entities moved no money at all, which is the result you
want from a migration whose only job is to make contention impossible.
The re-key does not retrospectively re-attribute records claimed by the wrong
client while a location was shared; those stay where they were last written.
Correcting them is a separate exercise, gated on the business deciding which
client owns each location.
Report updated with the isolation audit and a plain-language explanation of the
remove-voided-orders component cascade.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The previous report derived its baseline arithmetically. This one runs the job
three times over the same 90 days against identical data, writing real summaries
each pass: production's calculation, then R1, then R2. 18,900 summaries per run,
56,700 in total. Window 2026-05-10 to 2026-08-07, deliberately stopping a week
short of the backup so the partial days around the cut cannot distort it.
Reportable population, 8,350 client-days across 96 clients:
baseline 1,024 days off, 87.74% clean, $58,531.75
+ R1 752 days off, 90.99% clean, $55,528.17
+ R1 + R2 105 days off, 98.74% clean, $852.38
R1 8,054 unchanged, 272 to balance, 0 to unbalance, 0 balanced-day edits
R2 7,701 unchanged, 647 to balance, 0 to unbalance, 0 balanced-day edits
Across 90 days of trading neither fix altered a single day that already
balanced, replicating the 30-day finding on three times the sample.
The longer window surfaced two residual clusters the short one could not see:
NGMV across five days in late May and NGEB across four spanning May to July.
NGEB is the known ezCater fee question; NGMV and N-30012 are undiagnosed and
worth a look before this ships.
Also records a deployment finding: the transactor was sized for a toy database
(2 GB object cache against 27 GB of data) and the final 7,958 client-days of a
pass were crawling at ~35/min. Resized to 8 GB cache and 16 GB heap, they
finished in 90 seconds. Worth checking production's sizing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every claim in the 30-day report now names the record behind it, so a reader can
check it rather than take it on trust.
R1 square/order/NGLK-SM-OxSX9gpXJV394qqT8mnBGypUwKNZY
tip -60.00, total -60.00, zero charges — the reversal get-tip could not
see, and NGLK 2026-08-04 was off by exactly -60.00
R2 square/order/NTPT-PT-KrMZzcon1cpQEJUyetErkBIpcdEZY
service charge 427.10 inside a 3,198.78 order, credited nowhere, and
NTPT 2026-08-06 was off by exactly +427.10
Also lists the 19 days the data work unbalanced with their before and after
figures — all 19 land back at zero once R1 and R2 apply — and three real rows
behind the balanced-but-changed pattern, where Card Payments falls by exactly
what Fees gains.
Corrects the NGBK 2026-08-06 residual. It was attributed to refunds held by both
twins; tracing the day shows tender of $6,358.99 against order totals of
$6,059.57, a difference of exactly the $299.42 imbalance. The day has no refund
lines, no order carries two charges for one payment id, and the orders missing
line items carry no tender. It is a source-data discrepancy, not a calculation
defect.
Adds the REPL calls that re-derive every figure.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
compare-sales-summaries diffs summaries between two points in the same database
using as-of, so a recompute can be audited against exactly what was there before
without a snapshot or scratch copy.
It answers a question that balance-status counting hides: a day can stay
balanced while its line amounts move, which is a real change to the books even
though no red turns green. Amounts are compared at the cent so floating-point
noise does not read as a change.
Run over 2026-07-15 to 08-13 against production's own summaries at basis-t
209608347:
out of balance -> balanced 309
balanced -> out of balance 0
balanced -> balanced, numbers moved 237
no summary in production 4,058
Broken down by fix, neither calculation change touches a healthy day: R1 fixes
86 with 0 unbalanced and 0 balanced-day edits, R2 fixes 276 on the same terms.
Every one of the 237 balanced-but-changed days comes from the data work, which
is also the only step that unbalances anything — 19 days, all of them absorbed
by R1 and R2, which is why the end-to-end count is zero.
The 4,058 missing summaries include eight consecutive days, Jul 30 to Aug 6,
missing for all 210 clients — the coverage hole the plan predicted from the
scheduler's seven-day lookback, confirmed here independently from the data.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Recomputed every summary in the database — 14,458 client-days spanning
2024-04-01 to 2026-08-14 — through sales-summaries-v2, zero failures, none left
dirty. Compared the fixes against baseline over the last 30 days
(2026-07-15 to 2026-08-13), both arms on the same deduplicated data so this
isolates what the calculation fixes are worth on top of the dedup work.
Excluding the deactivated twins, 2,842 client-days:
baseline 398 days off, 86.00% clean, $22,527.40, 336 material
fixed 67 days off, 97.64% clean, $405.66, 5 material
331 client-days fixed, 0 regressed — not one balanced day stopped balancing.
Of the $405.66 left, $399 sits in five material days, three of which the plan
predicted; the remaining 62 days total $1.62 with the largest at 9.00c, so the
10c threshold separates rounding from real variance with nothing near the
boundary.
Baseline was derived as fixed + untendered tip + service charges, since R1 and
R2 only add credits; the identity was verified against a from-scratch baseline
recomputation on 20 random client-days.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Re-activated all 10 twins so every Square location is shared again, then
re-imported. Three consecutive refund imports for both halves of the Concord
pair leave counts unchanged: NGCC 127/127/127/127, NGCD 1419/1419/1419/1419.
That is the configuration that previously made a refund's owner flip every ~20
minutes. With client-scoped keys each client resolves its own entity and updates
it in place, so repeated imports are idempotent and the two copies are stable.
Only the first import after migration grows anything, once: NGCD materialised
100 of its own copies while NGCC, already carrying its scoped key, gained only
the 2 refunds that were genuinely new.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Executes what Part 1 only measured: Phase 0, the re-key migration, a live Square
import, and a full recompute through sales-summaries-v2, with every window
re-cut to end 08-13 so the backup's partial final day stops skewing the numbers.
Reportable error falls from 542 days off / $30,982.11 to 71 days off / $405.79,
a 98.7% reduction, with 5 material days left — three of them the ones the plan
predicted. 107,815 entities were re-keyed with refund and charge counts
unchanged, which is what proves the expand step resolves legacy keys instead of
duplicating under the new scheme. NGCD 2026-07-23, off by exactly $71.94 with no
refund line, now carries the line and balances to 9.1e-13.
Records two defects that only executing could surface:
Re-keying a charge with two parent orders duplicates the tender. The migration
scoped such charges through whichever order it saw first, so the other client's
import found neither the scoped nor the legacy key and created a second charge,
which cardinality-many then added beside the first. Contended clients' error was
$441,045 until 15,961 stale refs were retracted. §4.3 warned about this and the
migration under-weighted it; it must split shared charges before shipping.
dirty-sales-summaries index-pulls from the client's start point with no :end and
filters lazily, so it scans every later client's summaries too — O(n²), and the
recompute degraded from ~180 to ~3 client-days per minute as summaries piled up.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
:sales-order/charges is a component attribute, so [:db/retractEntity order]
cascades into the charges. Where two clients were configured on one Square
location, both clients' orders resolved to the SAME charge entity, because
charge keys carried no client scoping and :charge/external-id is
:db.unique/identity.
Retracting a voided order therefore deletes a charge the other client's order
still references, leaving that order with sales and no tender: the day goes out
of balance and the payment disappears from the current database value.
Measured on the restore: 35,870 of 56,829 charges (63%) in the contended clients'
recent window have more than one parent order.
Phase 0 stops new sharing and the re-key makes it structurally impossible going
forward, but neither splits the charges that are already shared, so the hazard
outlives both. Records the options, including guarding the retraction so it
detaches shared charges rather than deleting them.
Pre-existing risk, not introduced here. The validation run skipped this step.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measures R1 (reversed tips) and R2 (Square service charges) over 3,342
client-days of real production data, restored from backup point 209608347
(newest transaction 2026-08-14 22:52).
R2 clears 307 client-days with zero regressions, which is the gate the plan set
for it. R1 clears 78 with 2 regressions, both days that balanced only because
two errors cancelled, and both on clients that share a Square location.
After both fixes, 98.6% of the remaining dollar error sits in the 10 contended
client pairs — Phase 0's to fix, not a calculation defect. Outside those, the
entire residual is $73.98, of which all but six client-days is sub-10c register
rounding. The three predicted material days reproduce to the cent.
Also records an amendment to Phase 1: 2,122,161 charges (12.8%) carry neither
:charge/client nor :charge/location, so the re-key cannot scope them from the
entity. They are payout-path stubs and order-referenced tenders; none are
orphaned, so scoping must come from the referencing order or expected-deposit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
## Summary
Completes the automatic sales summary pipeline end-to-end: the `sales-summaries-v2` job now calculates aggregate totals, preserves manual adjustments, and automatically posts balanced journal entries to the ledger.
## What Changed
**New Datomic transaction function** (`upsert-sales-summary-ledger`)
- Transforms detailed `sales-summary-item`s into aggregated `journal-entry` lines grouped by account and ledger side
- Handles the full upsert: posts a new journal entry for summaries with mapped accounts, or retracts the orphaned entry if items no longer qualify
**Enhanced `sales-summaries-v2` job**
- Calculates and stores 13 aggregate total attributes (card/cash/food-app/gift-card payments, refunds, fees, discounts, tax, tip, returns, unknown, net)
- Preserves manual items (`manual? true`) during recalculation — only auto-calculated items are replaced
**Ledger reconciliation**
- `reconcile-ledger` now queries for sales summaries missing journal entries and repairs them via `:upsert-sales-summary-ledger`, alongside existing invoice and transaction repairs
**Schema**
- Added 13 `total-*` attributes on `sales-summary` (all `db.type/double`, no history)
- Registered the new transaction function in `tx.clj` and `datomic.clj`
**Admin UI cleanup**
- Resolved "clientize" and HTMX `client-id` TODOs in the sales summaries admin page
- `new-summary-item` now correctly passes `client-id` via `hx-vals`
- Removed stale TODO comments and placeholder code
## Files Changed (8)
| File | Purpose |
|------|---------|
| `iol_ion/.../upsert_sales_summary_ledger.clj` | New Datomic tx function |
| `iol_ion/.../tx.clj` | Register new tx function |
| `resources/schema.edn` | 13 new `total-*` attributes |
| `src/.../datomic.clj` | Load new tx namespace |
| `src/.../jobs/sales_summaries.clj` | Aggregate totals + manual item preservation |
| `src/.../ledger.clj` | Sales summary repair in `reconcile-ledger` |
| `src/.../ssr/admin/sales_summaries.clj` | UI TODO cleanup |
| `docs/plans/...plan.md` | Implementation plan document |
Co-authored-by: Bryce <bryce@integreatconsult.com>
Reviewed-on: #5
Co-authored-by: Bryce <bryce@brycecovertoperations.com>
Co-committed-by: Bryce <bryce@brycecovertoperations.com>
- Add testing patterns section (Grid Page, Wizard, Permission Gates)
- Convert all behaviors to numbered tables with checkboxes
- Specify test strategy per behavior (Unit/Integration/UI)
- Group by feature area: Display, Filter, Sort, Pay Wizard, etc.
- Add cross-cutting Permissions and Lock Date tables
- Reduce from 496 to 403 lines while being more comprehensive
Add behavior documentation covering all SSR and legacy SPA pages:
- Testing strategy and type definitions (unit/integration/UI)
- Dashboard, Invoice, Payment, Transaction, Ledger pages
- Company/Settings, POS, Admin, Search, Auth pages
- Legacy SPA behavior docs (no UI tests until migrated)
- Edge cases, test data requirements, and dependencies per subsystem
Total: 3,600+ lines of behavior documentation to guide test authorship.
- Extract city/state/zip in location field
- Customer address now split across 3 fields:
- customer-identifier: customer name
- account-number: street address
- location: city, state zip
- All components verified in test
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add new PDF template for Bonanza Produce vendor
- Template uses phone number 530-544-4136 as unique identifier
- Extracts invoice number, date, customer identifier, and total
- Includes passing test for invoice 03881260
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
