feat(square): date arity for cash shift import, and verify the re-key is book-neutral
Adds a [client location start end] arity to upsert-cash-shifts, matching
upsert-payouts, so a backfill can cover a chosen window rather than the fixed
75-day default.
Re-keyed and re-imported the two newly scoped types on the restore, then checked
what it cost:
144,652 expected deposits and 69,291 cash drawer shifts re-keyed
entity counts unchanged on all four Square types, 0 collisions, 0 legacy keys
live re-import over 66 of 102 clients wrote NOTHING — no entities created and
no values rewritten, because every record resolved to the importing client's
own entity through the legacy-key fallback
0 ownership changes since the re-key, across deposits, shifts and refunds
12,044 summaries recomputed afterwards and compared: 0 differences
So re-keying 213,943 entities moved no money at all, which is the result you
want from a migration whose only job is to make contention impossible.
The re-key does not retrospectively re-attribute records claimed by the wrong
client while a location was shared; those stay where they were last written.
Correcting them is a separate exercise, gated on the business deciding which
client owns each location.
Report updated with the isolation audit and a plain-language explanation of the
remove-voided-orders component cascade.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -356,6 +356,109 @@
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h2>Isolation: which entities could still swap owners</h2>
|
||||
<div class="measure">
|
||||
<p>Balancing was only half the problem. The other half is whether an entity can change hands
|
||||
between two clients at all. Rather than reason about it, every entity type the importers
|
||||
create with a unique external id was audited, and ownership was read out of the database's
|
||||
own history.</p>
|
||||
</div>
|
||||
|
||||
<h3>Entities whose client has actually changed</h3>
|
||||
<div class="scroll">
|
||||
<table>
|
||||
<thead><tr><th>Entity</th><th class="n">Swapped</th><th class="n">Of total</th><th>Key before</th></tr></thead>
|
||||
<tbody>
|
||||
<tr><td>Expected deposits (Square payouts)</td><td class="n bad">4,069</td><td class="n">144,688</td><td class="mono">square/payout/<id></td></tr>
|
||||
<tr><td>Cash drawer shifts</td><td class="n bad">2,628</td><td class="n">69,291</td><td class="mono">square/cash-drawer-shift/<id></td></tr>
|
||||
<tr><td>Refunds</td><td class="n">3,387</td><td class="n">51,990</td><td class="dim">fixed earlier</td></tr>
|
||||
<tr><td>Sales orders, ezCater orders</td><td class="n good">0</td><td class="n dim">—</td><td class="dim">already scoped</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
<div class="measure">
|
||||
<p>Both payout and cash-shift endpoints are filtered by location, so two clients configured on
|
||||
one Square location import the same records and collide on a single entity — the identical
|
||||
defect refunds and charges had, in two types nobody had looked at.</p>
|
||||
</div>
|
||||
|
||||
<div class="callout warn">
|
||||
<span class="h">Nineteen client pairs have contended, but only ten are visible today</span>
|
||||
<p>Nine pairs no longer share a location in the current configuration, so no point-in-time
|
||||
check would find them — yet their data is still mixed. The largest are NGMJ/NGSC with 1,546
|
||||
affected entities, NGAK/NGMH with 952, NGBW/NGWD with 780 and NGNP/NGVZ with 645. This is the
|
||||
argument for scoping the keys rather than only deactivating what currently looks shared:
|
||||
deactivation is a snapshot fix that rots the next time someone configures a location twice.</p>
|
||||
<p>Worth knowing alongside it: <strong>105 of the clients share a single Square auth token</strong>,
|
||||
so these importers all reach into one Square account. The blast radius of a future
|
||||
misconfiguration is the estate, not a pair.</p>
|
||||
</div>
|
||||
|
||||
<h3>What was done, and what it cost the books</h3>
|
||||
<div class="scroll">
|
||||
<table>
|
||||
<thead><tr><th>Step</th><th>Result</th></tr></thead>
|
||||
<tbody>
|
||||
<tr><td>Re-key expected deposits</td><td class="n">144,652 re-keyed · 0 collisions · 36 unscopable</td></tr>
|
||||
<tr><td>Re-key cash drawer shifts</td><td class="n">69,291 re-keyed · 0 collisions · 0 unscopable</td></tr>
|
||||
<tr><td><strong>Entity counts before vs after</strong></td><td class="n good">unchanged on all four types</td></tr>
|
||||
<tr><td>Legacy-scheme keys remaining</td><td class="n good">0</td></tr>
|
||||
<tr><td>Live re-import, 66 of 102 clients, 90-day window</td><td class="n good">0 entities created · 0 values rewritten</td></tr>
|
||||
<tr><td><strong>Ownership changes since the re-key</strong></td><td class="n good">0 deposits · 0 shifts · 0 refunds</td></tr>
|
||||
<tr><td><strong>Summaries recomputed and compared</strong></td><td class="n good">12,044 · 0 differences</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
<div class="measure">
|
||||
<p>The re-import is the important negative result. Every payout and shift the importer fetched
|
||||
resolved to the client's own entity and wrote nothing — the re-key is transparent to the
|
||||
importer, which is exactly what the legacy-key fallback is for. And recomputing 12,044
|
||||
client-days afterwards reproduced the previous figures to the cent, so re-keying 213,943
|
||||
entities moved no money at all.</p>
|
||||
<p>One caveat worth stating plainly: re-keying stops future contention, but it does not
|
||||
retrospectively re-attribute records that were claimed by the wrong client while the
|
||||
configuration was shared. Those stay where they were last written. Correcting them is a
|
||||
separate exercise, and one to do only once the business decides which client owns each
|
||||
location.</p>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h2>Why <code>remove-voided-orders</code> is dangerous</h2>
|
||||
<div class="measure">
|
||||
<p>The schema says a charge belongs to its order:</p>
|
||||
</div>
|
||||
<div class="scroll">
|
||||
<pre>{:db/ident :sales-order/charges, :db/isComponent true}</pre>
|
||||
</div>
|
||||
<div class="measure">
|
||||
<p><code>:db/isComponent</code> means Datomic treats charges as <em>parts of</em> the order rather
|
||||
than independent records. So <code>[:db/retractEntity <order>]</code> deletes the order
|
||||
<strong>and its charges</strong>. That is the documented behaviour, and normally it is what you want.</p>
|
||||
<p><code>remove-voided-orders</code> asks Square for the last ten days of orders, keeps the ones
|
||||
Square reports as voided, and retracts them. Correct on its own terms.</p>
|
||||
<p>The problem is that where two clients shared a location, both clients' orders resolved to the
|
||||
<strong>same charge entity</strong>, because charge keys carried no client:</p>
|
||||
</div>
|
||||
<div class="scroll">
|
||||
<pre>NGCD order ──┐
|
||||
├──> charge 17592490524 ← one entity, two parents
|
||||
NGCC order ──┘</pre>
|
||||
</div>
|
||||
<div class="measure">
|
||||
<p>Retract either order and Datomic deletes that shared charge. The other client's order survives
|
||||
with its sales intact but its payment gone: the day silently goes out of balance and the tender
|
||||
disappears from the current database value. Measured on the restore,
|
||||
<strong>35,870 of 56,829 charges (63%)</strong> in the contended clients' recent window have two
|
||||
parent orders.</p>
|
||||
<p>The re-key stops new sharing but does not un-share those. Two remedies, either sufficient:
|
||||
split them by re-importing the affected window now that keys are client-scoped, or guard the
|
||||
retraction so it detaches a shared charge instead of deleting it. The guard is small and makes
|
||||
the operation safe whatever shape the data is in.</p>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<footer>
|
||||
<span>Measured 2026-08-15 on <span class="mono">integreat-prod-restore</span>, restored from backup point 209608347. Nothing in production was touched. Branch <span class="mono">worktree-sales-summary-balance</span>.</span>
|
||||
<span>Service charges map to account 49000 Service Income, chosen to make measurement possible and still pending accounting sign-off. The account affects only whether a day can reach accepted — never whether it balances — so no figure here depends on it.</span>
|
||||
|
||||
Reference in New Issue
Block a user