Rebuilt the whole analysis from nothing: fresh restore of backup point
209608347, deactivate the ten shared locations, re-key and split every
one of 19,040,785 orders, re-import from Square, then two full ninety-day
recomputes — one with the fixes off, one with them on.
The baseline is now a no-fix recompute rather than production's stored
summaries. That is the stricter comparison: production's figures are in
places months stale, and crediting the fixes with repairing ordinary
staleness flattered them. On the fairer footing the two arithmetic fixes
are worth 979 client-days and $61,769.56, taking the window from 1,258
days out of balance ($69,560.10) to 279 ($7,790.54), with zero days
knocked out of balance and zero already-balanced days altered at line
level.
The migration now runs to completion database-wide: 17,047,142 payments
scoped, nothing left to rename, nothing unscopable, and no payment owned
by more than one order across 400,000 orders checked. The earlier
"transactor-bound, two days" diagnosis was wrong — the bottleneck was GC
in the driving process; the full pass takes about thirteen minutes.
Also corrects compare-sales-summaries: :ledger-mapped/amount, ledger-side
and account are :db/noHistory, so as-of cannot recover past amounts and
a rewritten summary reads back as a false balanced day. Every figure in
the report comes from live captures taken straight after each pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Replaces the partial-completeness section with the measured end state.
card payments 17,046,418 scoped · 0 unscoped · 0 unscopable
refunds 51,986 scoped · 0 unscoped
payouts 144,652 scoped · 0 unscoped · 36 with no owner
cash shifts 69,291 scoped · 0 unscoped
Payments shared between two different clients: 0 in a 250,000-order sample
spread across the database. 104 remain shared between two orders of the same
client, left that way on purpose — Square split one tender across two of that
client's orders, both orders compute the same name so there is no second name a
copy could take, and a copy would double the client's takings. One payment
covering both is the truthful record.
That is what makes the remove-voided-orders guard a required companion rather
than belt-and-braces: the cascade still reaches those 104.
Records that a second complete pass over all 19,040,296 orders took 7.8 minutes
and changed nothing, so the migration is stoppable, resumable and repeatable —
and that with the gate now reading zero, the importer's understanding of the old
name form can be removed, which is the one remaining step.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Running the split across all 19,040,296 orders left 104 payments with two parent
orders in a 250,000-order sample. Every one of them is two orders of the SAME
client; cross-client sharing is gone entirely.
Attempting to split those too was wrong twice over. Both orders compute the same
name, so there is no second name to give a copy and the transaction conflicts.
And a copy would double that client's takings for the day — where Square splits
one tender across two of a client's own orders, one payment covering both is the
truthful record.
So the rule is now explicit: split per client, not per order. The component
cascade still reaches these, which is why remove-voided-orders needs its own
guard regardless of how complete this migration is — that was already the
recommendation and this makes it load-bearing rather than belt-and-braces.
Batch bookkeeping now records which name each charge was claimed under, so a
second order in the same batch computing that same name is left alone instead of
attempting a colliding copy.
Migration state on the restore, measured rather than asserted:
charges 17,046,418 scoped · 0 legacy · 0 unscopable
refunds 51,986 scoped · 0 legacy
payouts 144,652 scoped · 0 legacy · 36 with no owner
shifts 69,291 scoped · 0 legacy
A second full pass walked all 19M orders in 7.8 minutes and changed nothing,
which is the idempotency the tests assert, confirmed at full scale.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The report implied the shared-payment problem was solved. Measuring the whole
database shows it is solved for refunds, payouts and shifts, and 1.6% solved for
card payments:
refunds 51,986 scoped · 0 unscoped
payouts 144,652 scoped · 0 unscoped · 36 with no owner
cash-drawer shifts 69,291 scoped · 0 unscoped
card payments 259,763 scoped · 14,241,890 unscoped · 2,122,161 no owner
Sampling 2% of the orders belonging to every client that has ever contended
found 12,954 payments still owned by two orders, extrapolating to roughly
650,000 across that population.
So the balance figures are what the calculation fixes are worth on data where
the sharing had been resolved for the clients and period measured — not a claim
that the database is clean. The report now says so, and the headline stat that
claimed zero shared payments has been replaced with the 1.6% actually renamed.
Records what finishing requires: renaming is driven from orders, so it means
walking all 19,040,296 of them rather than the clients that look shared today,
since nine pairs contended in the past without sharing a location now. Measured
at ~15,000 orders/hour once the 29 GB database stopped fitting in memory — the
same code did 267,012 in 100 seconds with a warm cache — so it belongs in a
maintenance window on sized hardware. It is resumable and a second pass is a
no-op.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The migration had been run over a chosen subset — the ten clients that share a
Square location today, within a date window. Measuring the whole database showed
how partial that was:
charges 259,763 scoped · 14,241,890 legacy · 2,122,161 with no owner
refunds, payouts, shifts: complete
So 1.6% of charges carried a client-scoped key. The importer's tolerance of both
key schemes was not easing a transition, it was holding the system together.
unscoped-report counts, per entity type, how many keys are scoped, how many are
still legacy, and how many have no owner to scope by. That is the gate: while
legacy is above zero the database is in a mixed state and a stray unscoped
record can still be adopted by whichever client imports it first. At zero, the
fallback lookup in square.core3/existing-id can be removed and the guarantee
becomes structural rather than conventional.
migrate-all! runs it over every order rather than a subset. Splitting has to be
driven from orders because a payment's rightful owner is whichever order refers
to it — and nine client pairs contended in the past without sharing a location
today, so a migration scoped to the current configuration misses all of them.
Two changes were needed to run at that scale. The split no longer remembers
every charge it has seen; whether a charge is claimed is read from the charge,
which needs a map of the whole table otherwise. And claiming a charge now
records the claiming client on it, which is how a later order recovers the
Square id from an already-scoped key — the regression test caught the omission
immediately.
Re-running is now a true no-op rather than a repeated rename.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The report explained the faults and what repairing them was worth, but never
what the repair actually is. Adds a section showing the real code for each of
the five changes, with the reasoning that is not obvious from reading it.
1 put the client in the record's name — scoped-key, five call sites
2 find the existing record under either name before writing — existing-id,
which is what makes the rename safe to deploy and why the totals did not
double across 213,943 renamed records
3 give every order its own payment record — what :keep and :clone do, and
why the Square id must be recovered from the record's owner rather than by
trimming a prefix, since client codes like N-30003 contain dashes
4 add untendered tips rather than replacing the calculation, because where an
order does have a payment the payment is the correct source
5 credit service charges, with both branches of the vendor test explained —
ezCater commission must be excluded, but whole eras of Square orders carry
no vendor at all, so a test on vendor alone credits nothing
Plus the four supporting changes and why the schema install order mattered
enough to block every test in the suite.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Deletes and restores the database to backup point 209608347, then runs the whole
pipeline on untouched production data with the corrected split: deactivate the
duplicate client per location, give every order its own payment record,
client-scope refunds, payouts and cash-drawer shifts, import live from Square,
and recompute ninety days at each stage.
production today 1,280 days off, 85.34% clean, $75,228.78
+ deduplication 1,087 days off, 87.55% clean, $63,764.92
+ refunded tips 800 days off, 90.84% clean, $60,619.21
+ service charges 108 days off, 98.76% clean, $1,995.36
1,172 client-days brought into balance, 0 knocked out. Neither arithmetic fix
altered a single figure on a day that already balanced. Excluding the ten
deactivated duplicates: 1,217 days and $69,995.61 down to 105 days and $852.38.
The split does what it was missing before: 189,167 payments re-keyed, 77,599
copied so each order owns one, and payments with two parent orders went 11,469
to 0 — holding at 0 after a live import across 48,362 orders. That closes the
component-cascade risk in remove-voided-orders.
Report rewritten for both audiences: what each fault meant for the books, then
the mechanism for engineers, with worked line-level examples, the verification
checks, the residual, and the open decisions. Includes the double-scoping bug
this work introduced and how measurement caught it.
23 tests, 55 assertions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The split derived the underlying Square id by stripping only the
"square/charge/" prefix. That is correct the first time a charge is seen, but
once it has been re-keyed to one client, a second order processing the same
charge in a later batch reads NGCC-CC-<id> as the id and scopes it again:
square/charge/NGCD-CD-NGCC-CC-hW59pSj5hAsFBUIQi6cMyY36kN7YY
The importer then computes the correct single-scoped key, does not find it, and
creates a second charge — doubling the tender on exactly the days the import
touched. Found on a restored backup where five contended clients showed
imbalances of $3,000 to $7,000 on Aug 5-7.
Client codes may contain dashes, so the scope cannot be recognised by pattern.
raw-square-id recovers it from the entity instead: whoever the charge currently
belongs to is whose scope its key carries. The run also remembers each charge's
raw id when it first claims it, so later clones reuse it rather than re-deriving
from a mutated key.
Covered by a test that runs the split with a batch size of one, which is the
arrangement that made the second order observe an already-scoped key.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Re-keying alone could not undo an existing shared charge. It handed the single
entity to whichever client was looked at first and left the other order pointing
at a charge it does not own. Because :sales-order/charges is a component
attribute, that is not untidy but dangerous: retracting either order deletes a
charge the other still needs. It also produced double tender when the second
client re-imported and created its own.
split-and-rekey-charges! now gives every order its own charge. The first order
to claim a shared charge keeps it, re-keyed to that order's client and location;
every other order gets a copy carrying the same amounts, scoped to itself, with
its reference repointed. Afterwards no charge has more than one parent order and
the component relationship means what it says.
charges-with-multiple-parents is the §3.3 gate, which must read zero before any
historical cleanup or voided-order retraction is safe.
Four tests cover it: that the shared condition exists to begin with, that the
split produces two distinct entities with amounts copied and one parent each,
that an unshared charge is only re-keyed, and that re-running changes nothing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The verification recompute finished. Comparing every summary in the 90-day
window against the run taken before the payout and cash-shift re-key:
18,900 compared, 0 differing, largest difference 0.000000
Supersedes the partial 12,044 figure reported while the pass was still running.
Re-keying 213,943 entities moved no money anywhere, to the last decimal place.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The report asserted that the fixes work; it did not show what changed on the
page. Replaces the worked-examples section with actual summary lines pulled from
the restore, before and after, including debit and credit totals.
NGLK 2026-08-04 is the clearest case. The day already carried a $60.00 card
refund — the guest was given their money back, tip included — while the Tip line
still credited $482.94, because the reversal sits on an order with no tender and
get-tip only reached tips through tenders. Corrected to $422.94, matching the
refund to the cent, and the day closes from -60.00 to 0.00.
NTPT 2026-08-06 shows the other shape: $427.10 of service charge arrived inside
a $4,975.89 card tender, so it landed on the debit side with nothing crediting
it. Credits go 7,350.10 to 7,777.20 against unchanged debits of 7,777.20.
Adds the six largest repairs of each kind with their real Tip and Service
Charges figures, and NGNP 2026-06-25 where both fixes land on one day and pull
in opposite directions — 301.40 credited, 1.80 removed, 299.60 closed — which is
a useful check that the two are independent.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds a [client location start end] arity to upsert-cash-shifts, matching
upsert-payouts, so a backfill can cover a chosen window rather than the fixed
75-day default.
Re-keyed and re-imported the two newly scoped types on the restore, then checked
what it cost:
144,652 expected deposits and 69,291 cash drawer shifts re-keyed
entity counts unchanged on all four Square types, 0 collisions, 0 legacy keys
live re-import over 66 of 102 clients wrote NOTHING — no entities created and
no values rewritten, because every record resolved to the importing client's
own entity through the legacy-key fallback
0 ownership changes since the re-key, across deposits, shifts and refunds
12,044 summaries recomputed afterwards and compared: 0 differences
So re-keying 213,943 entities moved no money at all, which is the result you
want from a migration whose only job is to make contention impossible.
The re-key does not retrospectively re-attribute records claimed by the wrong
client while a location was shared; those stay where they were last written.
Correcting them is a separate exercise, gated on the business deciding which
client owns each location.
Report updated with the isolation audit and a plain-language explanation of the
remove-voided-orders component cascade.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An audit of every unique-identity entity the importers create found two more
that carry no client scoping:
:expected-deposit/external-id "square/payout/" id
:cash-drawer-shift/external-id "square/cash-drawer-shift/" id
Both endpoints are location-filtered, so two clients configured on one Square
location import the same payout or shift and collide on a single entity, exactly
as refunds and charges did.
This is not hypothetical. Reading ownership out of the history of the restored
backup, the client has actually changed on:
4,069 of 144,688 expected deposits (2.8%)
2,628 of 69,291 cash drawer shifts (3.8%)
3,387 of 51,990 refunds (6.5%, before the earlier fix)
across 19 distinct client pairs — and only 10 of those pairs share a location in
today's configuration. Nine, including NGMJ/NGSC with 1,546 affected entities
and NGAK/NGMH with 952, are invisible to any point-in-time check because the
configuration has since changed. Deactivating what looks shared today is
therefore a snapshot fix; scoping the keys is what makes contention structurally
impossible.
Verified on the restore: 144,652 deposits and 69,291 shifts re-keyed with all
four entity counts unchanged, zero collisions, zero legacy keys remaining. 36
deposits carry neither client nor location and were left alone.
ezCater orders and sales orders were already scoped and need no change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The previous report derived its baseline arithmetically. This one runs the job
three times over the same 90 days against identical data, writing real summaries
each pass: production's calculation, then R1, then R2. 18,900 summaries per run,
56,700 in total. Window 2026-05-10 to 2026-08-07, deliberately stopping a week
short of the backup so the partial days around the cut cannot distort it.
Reportable population, 8,350 client-days across 96 clients:
baseline 1,024 days off, 87.74% clean, $58,531.75
+ R1 752 days off, 90.99% clean, $55,528.17
+ R1 + R2 105 days off, 98.74% clean, $852.38
R1 8,054 unchanged, 272 to balance, 0 to unbalance, 0 balanced-day edits
R2 7,701 unchanged, 647 to balance, 0 to unbalance, 0 balanced-day edits
Across 90 days of trading neither fix altered a single day that already
balanced, replicating the 30-day finding on three times the sample.
The longer window surfaced two residual clusters the short one could not see:
NGMV across five days in late May and NGEB across four spanning May to July.
NGEB is the known ezCater fee question; NGMV and N-30012 are undiagnosed and
worth a look before this ships.
Also records a deployment finding: the transactor was sized for a toy database
(2 GB object cache against 27 GB of data) and the final 7,958 client-days of a
pass were crawling at ~35/min. Resized to 8 GB cache and 16 GB heap, they
finished in 90 seconds. Worth checking production's sizing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every claim in the 30-day report now names the record behind it, so a reader can
check it rather than take it on trust.
R1 square/order/NGLK-SM-OxSX9gpXJV394qqT8mnBGypUwKNZY
tip -60.00, total -60.00, zero charges — the reversal get-tip could not
see, and NGLK 2026-08-04 was off by exactly -60.00
R2 square/order/NTPT-PT-KrMZzcon1cpQEJUyetErkBIpcdEZY
service charge 427.10 inside a 3,198.78 order, credited nowhere, and
NTPT 2026-08-06 was off by exactly +427.10
Also lists the 19 days the data work unbalanced with their before and after
figures — all 19 land back at zero once R1 and R2 apply — and three real rows
behind the balanced-but-changed pattern, where Card Payments falls by exactly
what Fees gains.
Corrects the NGBK 2026-08-06 residual. It was attributed to refunds held by both
twins; tracing the day shows tender of $6,358.99 against order totals of
$6,059.57, a difference of exactly the $299.42 imbalance. The day has no refund
lines, no order carries two charges for one payment id, and the orders missing
line items carry no tender. It is a source-data discrepancy, not a calculation
defect.
Adds the REPL calls that re-derive every figure.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
compare-sales-summaries diffs summaries between two points in the same database
using as-of, so a recompute can be audited against exactly what was there before
without a snapshot or scratch copy.
It answers a question that balance-status counting hides: a day can stay
balanced while its line amounts move, which is a real change to the books even
though no red turns green. Amounts are compared at the cent so floating-point
noise does not read as a change.
Run over 2026-07-15 to 08-13 against production's own summaries at basis-t
209608347:
out of balance -> balanced 309
balanced -> out of balance 0
balanced -> balanced, numbers moved 237
no summary in production 4,058
Broken down by fix, neither calculation change touches a healthy day: R1 fixes
86 with 0 unbalanced and 0 balanced-day edits, R2 fixes 276 on the same terms.
Every one of the 237 balanced-but-changed days comes from the data work, which
is also the only step that unbalances anything — 19 days, all of them absorbed
by R1 and R2, which is why the end-to-end count is zero.
The 4,058 missing summaries include eight consecutive days, Jul 30 to Aug 6,
missing for all 210 clients — the coverage hole the plan predicted from the
scheduler's seven-day lookback, confirmed here independently from the data.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Recomputed every summary in the database — 14,458 client-days spanning
2024-04-01 to 2026-08-14 — through sales-summaries-v2, zero failures, none left
dirty. Compared the fixes against baseline over the last 30 days
(2026-07-15 to 2026-08-13), both arms on the same deduplicated data so this
isolates what the calculation fixes are worth on top of the dedup work.
Excluding the deactivated twins, 2,842 client-days:
baseline 398 days off, 86.00% clean, $22,527.40, 336 material
fixed 67 days off, 97.64% clean, $405.66, 5 material
331 client-days fixed, 0 regressed — not one balanced day stopped balancing.
Of the $405.66 left, $399 sits in five material days, three of which the plan
predicted; the remaining 62 days total $1.62 with the largest at 9.00c, so the
10c threshold separates rounding from real variance with nothing near the
boundary.
Baseline was derived as fixed + untendered tip + service charges, since R1 and
R2 only add credits; the identity was verified against a from-scratch baseline
recomputation on 20 random client-days.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
dirty-sales-summaries index-pulled from [client-id true] and then filtered by
client. index-pull returns a lazy seq running to the END of the index, and a
lazy filter does not stop it, so for every client the job walked every summary
belonging to every client sorting after it — pulling their items along the way.
Quadratic in the number of summaries.
take-while stops at the client boundary instead, which is safe because
:sales-summary/client+dirty sorts by client first, so a client's dirty
summaries are contiguous from that start point.
Measured on a 14,458-summary database: 1,321ms -> 5.6ms per client, a 237x
improvement, with identical results. A full refresh had been degrading from
~180 client-days a minute to ~3 as summaries accumulated.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Re-activated all 10 twins so every Square location is shared again, then
re-imported. Three consecutive refund imports for both halves of the Concord
pair leave counts unchanged: NGCC 127/127/127/127, NGCD 1419/1419/1419/1419.
That is the configuration that previously made a refund's owner flip every ~20
minutes. With client-scoped keys each client resolves its own entity and updates
it in place, so repeated imports are idempotent and the two copies are stable.
Only the first import after migration grows anything, once: NGCD materialised
100 of its own copies while NGCC, already carrying its scoped key, gained only
the 2 refunds that were genuinely new.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Executes what Part 1 only measured: Phase 0, the re-key migration, a live Square
import, and a full recompute through sales-summaries-v2, with every window
re-cut to end 08-13 so the backup's partial final day stops skewing the numbers.
Reportable error falls from 542 days off / $30,982.11 to 71 days off / $405.79,
a 98.7% reduction, with 5 material days left — three of them the ones the plan
predicted. 107,815 entities were re-keyed with refund and charge counts
unchanged, which is what proves the expand step resolves legacy keys instead of
duplicating under the new scheme. NGCD 2026-07-23, off by exactly $71.94 with no
refund line, now carries the line and balances to 9.1e-13.
Records two defects that only executing could surface:
Re-keying a charge with two parent orders duplicates the tender. The migration
scoped such charges through whichever order it saw first, so the other client's
import found neither the scoped nor the legacy key and created a second charge,
which cardinality-many then added beside the first. Contended clients' error was
$441,045 until 15,961 stale refs were retracted. §4.3 warned about this and the
migration under-weighted it; it must split shared charges before shipping.
dirty-sales-summaries index-pulls from the client's start point with no :end and
filters lazily, so it scans every later client's summaries too — O(n²), and the
recompute degraded from ~180 to ~3 client-days per minute as summaries piled up.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
:sales-order/charges is a component attribute, so [:db/retractEntity order]
cascades into the charges. Where two clients were configured on one Square
location, both clients' orders resolved to the SAME charge entity, because
charge keys carried no client scoping and :charge/external-id is
:db.unique/identity.
Retracting a voided order therefore deletes a charge the other client's order
still references, leaving that order with sales and no tender: the day goes out
of balance and the payment disappears from the current database value.
Measured on the restore: 35,870 of 56,829 charges (63%) in the contended clients'
recent window have more than one parent order.
Phase 0 stops new sharing and the re-key makes it structurally impossible going
forward, but neither splits the charges that are already shared, so the hazard
outlives both. Records the options, including guarding the retraction so it
detaches shared charges rather than deleting them.
Pre-existing risk, not introduced here. The validation run skipped this step.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sales order keys already carry client and location; refund and charge keys do
not. That is why two clients configured on the same Square location share one
entity: the refund's owner flips every time either client imports, and a single
charge ends up referenced by both clients' orders. Scoping the keys the same way
makes contention structurally impossible — each client gets its own entity.
The hazard is the cutover. These ids are :db.unique/identity and the import
relies on upsert-by-identity, so changing the key format alone would match
nothing and Datomic would create a SECOND entity for every refund and charge,
orphaning the originals under their legacy keys. square.core3/existing-id
resolves the entity explicitly, scoped key first and legacy key second, and pins
the result as :db/id so the write lands on the existing entity whichever scheme
it currently carries.
All three construction sites are covered: order tenders, refunds, and the payout
path, which mints bare charge stubs from an external id alone.
The migration job re-keys whatever the importer has not yet touched. It recovers
scope from the referencing sales order or expected deposit for the ~12.8% of
charges that carry neither :charge/client nor :charge/location, detects
already-scoped entities by comparing against the key they should have rather
than pattern-matching ids that may themselves contain dashes, and is therefore
idempotent and re-runnable over a partially migrated database.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measures R1 (reversed tips) and R2 (Square service charges) over 3,342
client-days of real production data, restored from backup point 209608347
(newest transaction 2026-08-14 22:52).
R2 clears 307 client-days with zero regressions, which is the gate the plan set
for it. R1 clears 78 with 2 regressions, both days that balanced only because
two errors cancelled, and both on clients that share a Square location.
After both fixes, 98.6% of the remaining dollar error sits in the 10 contended
client pairs — Phase 0's to fix, not a calculation defect. Outside those, the
entire residual is $73.98, of which all but six client-days is sub-10c register
rounding. The three predicted material days reproduce to the cent.
Also records an amendment to Phase 1: 2,122,161 charges (12.8%) carry neither
:charge/client nor :charge/location, so the re-key cannot scope them from the
entity. They are payout-path stubs and order-referenced tenders; none are
orphaned, so scoping must come from the referencing order or expected-deposit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two calculation defects that leave a day out of balance, plus the first tests to
cover sales summaries.
Tips: get-tip joins through :sales-order/charges, so it only sees tips that
settled on a tender. A return-only order has no tender — it carries the reversal
on :sales-order/tip, which nothing read — so the day credited a tip that had
been handed back. Now additive: tendered tips plus the tips on orders that have
no tender at all. Deliberately not a swap, because where an order does have a
tender the tender is the correct source; real orders exist whose tender carries
a tip their :sales-order/tip does not (auto-gratuity booked as a service charge,
wallet tips missing from the net amounts), and reading the order would drop them.
Service charges: nothing reads :sales-order/service-charge. The charge is
collected inside the card tender but no line credits it, so every order carrying
one leaves the day short by exactly that amount. Both signs count — a returned
catering fee arrives as a negative service charge and is subtracted back out of
:sales-order/returns, so dropping negatives would lose the reversal.
The vendor gate is load-bearing: ezCater service charges are commission deducted
from the restaurant rather than collected from the diner, and crediting those
makes the day worse. It matches on :sales-order/vendor where that is set and
falls back to the external id prefix where it is not — whole eras of Square
orders carry no vendor attribute at all, and a gate on vendor alone would have
silently credited nothing.
Service charges stay behind the per-client "summary-service-charges" flag, and
the account they map to (49000 Service Income) needs accounting sign-off before
that flag is enabled anywhere.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds d-ss/imbalance alongside balanced?, and logs it from sales-summaries-v2
together with the categories a balancing investigation keeps returning to.
balanced? answers yes or no; when the answer is no there is currently nothing
recording by how much or in which direction, so an out-of-balance day is only
visible by opening the grid. This turns it into something queryable, and gives
the calculation changes that follow a before-and-after signal rather than an
opinion. No behaviour change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A composite tuple can only be created once its member attributes exist, and the
pieces are split across the two schema files: the tuple
:journal-entry-line/running-balance-tuple is declared in schema.edn while one of
its members, :journal-entry-line/running-balance, is declared in
cloud-migration-schema.edn, which is transacted afterwards.
Against a long-lived database this never surfaced, because those attributes were
installed by separate transactions years apart. Against an empty one it fails
outright with :db.error/invalid-tuple-attrs, which meant transact-schema could
not build a fresh database at all — and since every test fixture starts from an
empty datomic:mem database, the entire test suite died in setup rather than in
any individual test.
Install every plain attribute first, then every tuple.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
sales-summaries-v2 recomputes every dirty summary, but nothing set the dirty
flag on a schedule: mark-all-dirty was only ever called by hand from the
comment block, and the job was registered in neither server.clj's
INTEGREAT_JOB dispatch nor terraform/deploy.tf, so -main was dead code that
could never run in production. Summaries were only recalculated when someone
remembered to do it in the REPL, and POS data keeps arriving after a business
day closes, so a summary computed once on the day was routinely wrong and
stayed wrong.
Add a daily job that marks the trailing 7 days dirty and recomputes them,
leaving finished work alone. "Finished" is the condition the app already calls
Balanced -- debits equal credits and every line is mapped to an account. Since
that is derived rather than stored, a summary that later falls out of balance
is picked up again on the next run.
Extract the Balanced predicate into auto-ap.datomic.sales-summaries so the
grid's pill and the job share one definition, rather than a background job
requiring an SSR namespace. total-debits/total-credits resolve the ledger side
from either a plain keyword or the {:db/ident ...} map a pull returns, and
accepted? requires every item to declare a side: un-normalized pulled items
otherwise sum to 0.0 on both sides, read as balanced, and get skipped
silently and permanently.
Also fix sales-summaries-v2 destroying user-entered line items. It filtered
for :sales-summary-item/manual? to preserve them, but dirty-sales-summaries'
index-pull selector never fetched :sales-summary/items, so manual-items was
always empty. Because items is a component attribute upserted via
[:reset-rels ...], every recompute deleted the hand-entered lines -- often the
very lines that make a summary balance. Harmless while nothing ran on a
schedule; destructive the moment this does.
Register the job in the admin Background Jobs dropdown too, with a days
field: schedules are prod-only, so the admin page is the only way to run it
on staging.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Recategorizes 131 Sysco line-item descriptions per the client-reviewed
"Product Category Verification with codes" sheet: 96 existing rows re-coded
and 35 new rows appended (ids 1796-1830).
Root cause this addresses: get-line-account matches on exact description
string and silently defaults anything unmapped to 50000 Food Costs. Only 147
of the sheet's 321 reviewed rows coded the way the client expected. Note the
sheet's "change" column understates the work -- 20 of its 53 change rows are
no-ops (Paper -> Paper, confirming the gloves/liners/hairnets) and 3 were
already fixed in 38575aa5 / 7a0e256f, while 214,398 lines of movement come
from rows the client ticked as correct against a suggestion that already
differed from production.
Moves, replayed over all 1,022,732 DET lines in sysco-poller:
50000 -> 51500 Dry Goods 90,481 ln $5,215,011.82 105 clients
50000 -> 51450 Dressing & Sauce 58,191 ln $4,445,726.56 98
50000 -> 51400 Bread and Bun 37,705 ln $3,964,462.24 96
50000 -> 52000 Soft Beverage 36,244 ln $1,033,497.52 94
50000 -> 51200 Produce 6,519 ln $460,629.32 98
55000 -> 51500 Dry Goods 6,516 ln $259,750.60 97
50000 -> 74100 Cleaning Supplies 5,630 ln $205,233.19 98
55000 -> 74100 Cleaning Supplies 5,222 ln $125,946.32 99
50000 -> 51120 Chicken/Poultry 265 ln $42,482.50 8
50000 -> 51300 Dairy 36 ln $5,941.69 6
50000 -> 55000 Paperware 54 ln $1,751.22 18
54400 -> 51450 Dressing & Sauce 24 ln $1,413.26 1
total 246,887 ln $15,761,846.24
Only three source accounts are touched: 50000 and 55000 (the two silent
defaults) plus the single intended 54400 -> 51450 vinaigrette row. Nothing
else leaves a deliberately assigned account.
The 7 Misc Charges descriptions are deliberately left alone per Bryce,
including PICKLE CHIP KOSH 1/4 KK, which therefore stays at the 50000
default rather than moving to Produce as the sheet originally suggested.
Also corrects the PAPER & DISP fallback comment in sysco.clj: 440 of its 455
mapped descriptions point at 55000, not all 454. The 15 exceptions (foil
pans -> 51500, scour pads -> 74100) are mapped explicitly, so the
description map still wins ahead of the fallback. No logic changed.
Affects only clients with the code-sysco-items feature flag, and only at
import time -- already-imported invoices keep their existing splits.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Sysco importer codes each line item by exact description match against
resources/sysco_line_item_mapping.csv, silently defaulting to GL 50000 (Food
Costs) when the description is absent. Every new or renamed Sysco SKU
therefore leaks into Food Costs until someone hand-patches the CSV, which is
what 38575aa5 did for 34 descriptions.
Add a category-level fallback consulted after the description map and before
the 50000 default, enabled for PAPER & DISP only. The description mapping
still wins wherever it exists, so nothing already mapped changes.
PAPER & DISP is safe to generalize: all 454 mapped PAPER & DISP rows point at
55000, with no exceptions. Of the 852 distinct descriptions ever invoiced
under that category, only 3 resolved elsewhere, each because a row with a
different category shared the description and won the later-wins (into {}).
One of those, DESSERT CUP, was simply mis-categorized -- it is paper, and its
own lid (id 1782 LID DOME DESSERT CUP) was already 55000 -- so correct id 1772
to PAPER & DISP / 55000. The remaining two stay at 50000 on purpose, since
they are not paper: PAD SCRUB S-S 35 GRAM 1.25 OZ (SUPP & EQUIP) and TEST
STRIP SANITIZER QUAT (CHEMICAL/JANTRL).
Verified by replaying both changes over all 1,022,732 DET lines in the 56,010
CSVs under sysco-poller/: every resulting transition is 50000 -> 55000 (10,994
lines, $728,106.62). No line that already resolved to a non-default account
moved.
Note this only affects clients carrying the code-sysco-items feature flag, and
only on import -- already-imported invoices need a separate recode.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The statement no longer prints "Reel Produce" as text (only
orders@reelproduce.com), switched to MM/DD/YYYY dates, and moved the
invoice number into an "INV #..." transaction description, so no
template matched and the file fell through to the glimpse2 fallback.
Adds a QuickBooks-statement-style template (same shape as Suncrest /
Ocean Queen) keyed on reelproduce.com + Statement, placed after the
existing Reel Produce statement template so the old layout still wins.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The rotation in 111eca41 updated the Plaid secret-key only in prod.edn,
leaving prod-background-worker.edn, prod-cloud-background-worker.edn, and
prod-cloud.edn on the old (now-invalidated) secret. Background worker jobs
loading those files failed with INVALID_API_KEYS.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Sysco importer codes line items by exact-matching the item description
against resources/sysco_line_item_mapping.csv, falling back to GL 50000
(Food Costs) when no entry exists. On master, 8 of the paper-product
descriptions on recent invoices (e.g. BAG PAPER 250 CT, NAPKIN 2PLY INTR
FOLD 6.3X8.26, CONTAINER PAPER 4/110OZ NTG) were missing, so they
defaulted to 50000 instead of 55000 (Paper Costs).
Append the 34 curated mappings (Ids 1762-1795) covering these paper items
(-> 55000) plus the other new items from the same invoices, so they code
correctly on re-import.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Aligns debit/credit amounts to a right column with tabular-nums;
replaces the in-cell delta and balanced text with chip-style status
indicators; shortens the edit dialog and clarifies its totals/unbalanced
footer rows; gives manual line items a subtle accent so they're
distinguishable from auto-generated rows.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
