feat(square): date arity for cash shift import, and verify the re-key is book-neutral

Adds a [client location start end] arity to upsert-cash-shifts, matching
upsert-payouts, so a backfill can cover a chosen window rather than the fixed
75-day default.

Re-keyed and re-imported the two newly scoped types on the restore, then checked
what it cost:

  144,652 expected deposits and 69,291 cash drawer shifts re-keyed
  entity counts unchanged on all four Square types, 0 collisions, 0 legacy keys
  live re-import over 66 of 102 clients wrote NOTHING — no entities created and
    no values rewritten, because every record resolved to the importing client's
    own entity through the legacy-key fallback
  0 ownership changes since the re-key, across deposits, shifts and refunds
  12,044 summaries recomputed afterwards and compared: 0 differences

So re-keying 213,943 entities moved no money at all, which is the result you
want from a migration whose only job is to make contention impossible.

The re-key does not retrospectively re-attribute records claimed by the wrong
client while a location was shared; those stay where they were last written.
Correcting them is a separate exercise, gated on the business deciding which
client owns each location.

Report updated with the isolation audit and a plain-language explanation of the
remove-voided-orders component cascade.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-15 13:03:15 -07:00
parent 87f7fd564b
commit 41a1eb04a8
2 changed files with 106 additions and 1 deletions

View File

@@ -356,6 +356,109 @@
</div>
</section>
<section>
<h2>Isolation: which entities could still swap owners</h2>
<div class="measure">
<p>Balancing was only half the problem. The other half is whether an entity can change hands
between two clients at all. Rather than reason about it, every entity type the importers
create with a unique external id was audited, and ownership was read out of the database's
own history.</p>
</div>
<h3>Entities whose client has actually changed</h3>
<div class="scroll">
<table>
<thead><tr><th>Entity</th><th class="n">Swapped</th><th class="n">Of total</th><th>Key before</th></tr></thead>
<tbody>
<tr><td>Expected deposits (Square payouts)</td><td class="n bad">4,069</td><td class="n">144,688</td><td class="mono">square/payout/&lt;id&gt;</td></tr>
<tr><td>Cash drawer shifts</td><td class="n bad">2,628</td><td class="n">69,291</td><td class="mono">square/cash-drawer-shift/&lt;id&gt;</td></tr>
<tr><td>Refunds</td><td class="n">3,387</td><td class="n">51,990</td><td class="dim">fixed earlier</td></tr>
<tr><td>Sales orders, ezCater orders</td><td class="n good">0</td><td class="n dim"></td><td class="dim">already scoped</td></tr>
</tbody>
</table>
</div>
<div class="measure">
<p>Both payout and cash-shift endpoints are filtered by location, so two clients configured on
one Square location import the same records and collide on a single entity — the identical
defect refunds and charges had, in two types nobody had looked at.</p>
</div>
<div class="callout warn">
<span class="h">Nineteen client pairs have contended, but only ten are visible today</span>
<p>Nine pairs no longer share a location in the current configuration, so no point-in-time
check would find them — yet their data is still mixed. The largest are NGMJ/NGSC with 1,546
affected entities, NGAK/NGMH with 952, NGBW/NGWD with 780 and NGNP/NGVZ with 645. This is the
argument for scoping the keys rather than only deactivating what currently looks shared:
deactivation is a snapshot fix that rots the next time someone configures a location twice.</p>
<p>Worth knowing alongside it: <strong>105 of the clients share a single Square auth token</strong>,
so these importers all reach into one Square account. The blast radius of a future
misconfiguration is the estate, not a pair.</p>
</div>
<h3>What was done, and what it cost the books</h3>
<div class="scroll">
<table>
<thead><tr><th>Step</th><th>Result</th></tr></thead>
<tbody>
<tr><td>Re-key expected deposits</td><td class="n">144,652 re-keyed · 0 collisions · 36 unscopable</td></tr>
<tr><td>Re-key cash drawer shifts</td><td class="n">69,291 re-keyed · 0 collisions · 0 unscopable</td></tr>
<tr><td><strong>Entity counts before vs after</strong></td><td class="n good">unchanged on all four types</td></tr>
<tr><td>Legacy-scheme keys remaining</td><td class="n good">0</td></tr>
<tr><td>Live re-import, 66 of 102 clients, 90-day window</td><td class="n good">0 entities created · 0 values rewritten</td></tr>
<tr><td><strong>Ownership changes since the re-key</strong></td><td class="n good">0 deposits · 0 shifts · 0 refunds</td></tr>
<tr><td><strong>Summaries recomputed and compared</strong></td><td class="n good">12,044 · 0 differences</td></tr>
</tbody>
</table>
</div>
<div class="measure">
<p>The re-import is the important negative result. Every payout and shift the importer fetched
resolved to the client's own entity and wrote nothing — the re-key is transparent to the
importer, which is exactly what the legacy-key fallback is for. And recomputing 12,044
client-days afterwards reproduced the previous figures to the cent, so re-keying 213,943
entities moved no money at all.</p>
<p>One caveat worth stating plainly: re-keying stops future contention, but it does not
retrospectively re-attribute records that were claimed by the wrong client while the
configuration was shared. Those stay where they were last written. Correcting them is a
separate exercise, and one to do only once the business decides which client owns each
location.</p>
</div>
</section>
<section>
<h2>Why <code>remove-voided-orders</code> is dangerous</h2>
<div class="measure">
<p>The schema says a charge belongs to its order:</p>
</div>
<div class="scroll">
<pre>{:db/ident :sales-order/charges, :db/isComponent true}</pre>
</div>
<div class="measure">
<p><code>:db/isComponent</code> means Datomic treats charges as <em>parts of</em> the order rather
than independent records. So <code>[:db/retractEntity &lt;order&gt;]</code> deletes the order
<strong>and its charges</strong>. That is the documented behaviour, and normally it is what you want.</p>
<p><code>remove-voided-orders</code> asks Square for the last ten days of orders, keeps the ones
Square reports as voided, and retracts them. Correct on its own terms.</p>
<p>The problem is that where two clients shared a location, both clients' orders resolved to the
<strong>same charge entity</strong>, because charge keys carried no client:</p>
</div>
<div class="scroll">
<pre>NGCD order ──┐
├──&gt; charge 17592490524 ← one entity, two parents
NGCC order ──┘</pre>
</div>
<div class="measure">
<p>Retract either order and Datomic deletes that shared charge. The other client's order survives
with its sales intact but its payment gone: the day silently goes out of balance and the tender
disappears from the current database value. Measured on the restore,
<strong>35,870 of 56,829 charges (63%)</strong> in the contended clients' recent window have two
parent orders.</p>
<p>The re-key stops new sharing but does not un-share those. Two remedies, either sufficient:
split them by re-importing the affected window now that keys are client-scoped, or guard the
retraction so it detaches a shared charge instead of deleting it. The guard is small and makes
the operation safe whatever shape the data is in.</p>
</div>
</section>
<footer>
<span>Measured 2026-08-15 on <span class="mono">integreat-prod-restore</span>, restored from backup point 209608347. Nothing in production was touched. Branch <span class="mono">worktree-sales-summary-balance</span>.</span>
<span>Service charges map to account 49000 Service Income, chosen to make measurement possible and still pending accounting sign-off. The account affects only whether a day can reach accepted — never whether it balances — so no figure here depends on it.</span>