fix(square): do not resolve a record that belongs to another client
Two clients on one Square location double a day's tender in the window
between deploying and finishing the migration. Reproduced end to end:
1. charge X carries the legacy key square/charge/P and belongs to
client B's order
2. client A's payout import resolves X through existing-id's legacy
fallback and renames it into A's scope
3. client B's next order import matches neither scheme, so it mints a
second charge
4. :sales-order/charges is cardinality-many and orders transact as
plain maps, so nothing retracts the first
B's order ends up holding two charges for one payment — $200 of tender
for a $100 payment — and running the migration on that state produces
square/charge/BBB-LB-AAA-LA-P, the same double-scoped shape that already
doubled tender on five clients once during this work.
existing-id's legacy branch now declines any record already owned by a
different client, reading the owner attribute and, for charges that
predate :charge/client, the client of the referencing order. Declining is
also correct on its merits: the write then lands on this client's own
copy, which is what the scoped keys exist to create. The payout path also
writes :charge/client/:charge/location alongside the key, so a charge's
scope and its owner can no longer disagree.
The guard is transitional and gets deleted with the legacy branch it
protects, at rollout step 9.
Rollout resequenced for the decision to leave duplicate client records
active: no deactivation, no "which record survives" call, and the risk
window closed by pausing the importer across deploy + migrate rather than
by removing one of the two writers. Both records converge to independent
stable histories once every key carries its owner.
Also from review:
- migrate-all! now collision-checks the charge pass like the other three
attributes instead of discovering a clash mid-run over 17M rows
- split-and-rekey-charges! logs progress every 200 batches; an
interrupted 19M-order run left no trail
- unscoped-report's docstring no longer promises a zero its :no-owner
column cannot reach; plan is named as the authoritative signal
- the rollout's pre-flight asked for a :collisions key plan never
returns, so it silently passed on every database
- the multi-parent gate sampled (take 400000 (all-order-ids db)), which
streams :aevt — ascending entity id — and so read the OLDEST 2% of
orders: 2019-12-31 to 2021-06-03, before any of the contention it
looks for. Now every order of the last year via the client+date index,
5,159,787 on the restored copy, reading 0
- the report claimed same-client pairs get copied once batches split
them. They do not, at any batch size; verified at batch-size 1 and now
pinned by a test
30 tests, 72 assertions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -230,6 +230,17 @@
|
||||
shifts</strong> — the same figures as at the restore point. Refunds went from 50,986 to 51,986,
|
||||
and all 1,000 of those came from the live Square import run afterwards, not from the renaming.
|
||||
Had the fallback lookup been missing, each of these would have doubled instead.</p>
|
||||
<p><strong>The fallback also has to refuse.</strong> Reading the old name is what stops
|
||||
duplicates; reading <em>anyone's</em> old name is what creates them. Two clients share a Square
|
||||
location, so client A's payout import can resolve a payment that belongs to client B's order,
|
||||
rename it into A's scope, and leave B's next import matching neither name — at which point B
|
||||
mints a second payment and, because an order's payments are a set that is added to rather than
|
||||
replaced, B's order ends up holding both. That is a doubled day's tender, and it was
|
||||
reproduced end to end before being fixed. The lookup now declines any record already owned by
|
||||
a different client, which is also the right answer on its merits: the write then lands on this
|
||||
client's own copy, which is what the scoped names exist to create.</p>
|
||||
<p>This is transitional. Once no legacy names remain, the fallback and the refusal are deleted
|
||||
together and the guarantee stops depending on either.</p>
|
||||
</div>
|
||||
|
||||
<h3>3 · Give every order its own payment record</h3>
|
||||
@@ -250,7 +261,7 @@
|
||||
<div class="measure">
|
||||
<p>Run over the whole database that was <strong>9,100,314 renamed and 200,027 copied</strong>,
|
||||
and payments owned by two orders went from 11,469 in a 20,000-order sample to zero across
|
||||
400,000 orders checked. The record count rose by exactly 200,027 — the number of copies it
|
||||
every order of the last year. The record count rose by about 200,027 — the number of copies it
|
||||
reported making, which is the check that it created what it meant to and nothing else.</p>
|
||||
<p>One subtlety worth recording, because it bit us: the Square id has to be recovered from the
|
||||
record's current owner rather than by trimming a fixed prefix. Client codes contain dashes —
|
||||
@@ -454,10 +465,9 @@
|
||||
<table>
|
||||
<thead><tr><th>Step</th><th>Result</th></tr></thead>
|
||||
<tbody>
|
||||
<tr><td>Deactivate the duplicate client at each shared location</td><td class="n">10 locations · shared locations remaining: <span class="good">0</span></td></tr>
|
||||
<tr><td>Walk every order in the database</td><td class="n">19,040,785 orders</td></tr>
|
||||
<tr><td>Give every order its own payment record</td><td class="n">9,100,314 re-keyed · 200,027 copied</td></tr>
|
||||
<tr><td><strong>Payments owned by two orders</strong></td><td class="n good">0 <span class="dim">across 400,000 orders checked</span></td></tr>
|
||||
<tr><td><strong>Payments owned by two orders</strong></td><td class="n good">0 <span class="dim">across every order of the last year — 5,159,787</span></td></tr>
|
||||
<tr><td>Client-scope refunds, payouts and cash-drawer shifts</td><td class="n good">counts unchanged · 0 collisions</td></tr>
|
||||
<tr><td>Live Square import afterwards</td><td class="n good">0 orders with duplicated payment · 0 shared payments</td></tr>
|
||||
<tr><td>Ownership changes after the change</td><td class="n good">0 refunds · 0 payouts · 0 shifts</td></tr>
|
||||
@@ -465,7 +475,8 @@
|
||||
</table>
|
||||
</div>
|
||||
<div class="measure">
|
||||
<p>The count checks are the ones that matter. If re-keying had gone wrong it would have created a second copy of every record rather than updating the existing one, and the totals would have doubled. They did not move. The payment-copy step is the exception and is meant to add records — it added exactly 200,027, matching the number of copies it reported making.</p>
|
||||
<p>The count checks are the ones that matter. If re-keying had gone wrong it would have created a second copy of every record rather than updating the existing one, and the totals would have doubled. They did not move. The payment-copy step is the exception and is meant to add records — it added exactly 200,027, close to the number of copies it reported making. (Close, not exact: the counter increments while the transaction is being assembled, so two copies that resolve onto one entity are counted twice. It is a good check, not a proof.)</p>
|
||||
<p><strong>The measurement above was taken with the duplicate client records deactivated, and that is not how this will be deployed.</strong> Leaving both records live is the intended configuration — the re-key is what separates them — but it means the deactivation that made this measurement clean will not be there. The gap that opens is narrow and specific: while any record still carries a legacy key, a second client can resolve onto it. That is why the deployment runs the migration with imports paused, and why <code>existing-id</code> now refuses to resolve a record belonging to another client.</p>
|
||||
</div>
|
||||
|
||||
<div class="callout">
|
||||
@@ -568,15 +579,20 @@
|
||||
<p>The problem this work exists to solve is gone: no payment answers to two orders, so the
|
||||
component relationship means what it says and deleting an order can no longer take another
|
||||
order's money with it.</p>
|
||||
<p><strong>One behaviour changed when the pass was run over everything, and it is worth
|
||||
recording.</strong> Where Square splits one tender across two of a single client's own orders,
|
||||
the earlier design left the payment shared on purpose — both orders compute the same name, so
|
||||
there is no second name for a copy to take. That rule only holds for two orders processed in
|
||||
the same batch. Run across nineteen million orders in batches of two thousand, such pairs
|
||||
almost always fall in different batches and the second order now takes a copy. Inside the
|
||||
ninety-day window this changed nothing measurable: the recompute after the database-wide pass
|
||||
matched the one before it to the cent. Outside the window it has not been measured, and it
|
||||
should be before this runs against production.</p>
|
||||
<p><strong>Where Square splits one tender across two of a single client's own orders, the
|
||||
payment stays shared — at any batch size.</strong> Both orders compute the same name, so there
|
||||
is no second name for a copy to take, and a copy would double that client's takings for the
|
||||
day. The mechanism is worth stating precisely, because it is not obvious from reading: once
|
||||
the first order re-keys the payment it also writes the owner attributes in the same
|
||||
transaction, so a later order recovers the bare Square id from those, computes the name the
|
||||
payment already carries, and the guard <code class="mono">(not= old new-key)</code> drops the
|
||||
row before any copy decision is reached. Verified by running the migration at a batch size of
|
||||
one, which forces the two orders into separate batches: no copy is made.</p>
|
||||
<p class="dim">An earlier draft of this report claimed the opposite — that such pairs would be
|
||||
copied once the batches split them — and flagged it as unmeasured risk to check before
|
||||
production. That was wrong, and it is recorded here rather than quietly deleted because it did
|
||||
real damage: an independent reviewer cited this paragraph as evidence and raised a defect that
|
||||
does not exist. A test now pins the behaviour at batch size one.</p>
|
||||
<p>The guard on <code>remove-voided-orders</code> is still worth having regardless. It is
|
||||
cheap, and it makes the safety a property of the deletion rather than of the migration having
|
||||
been run first.</p>
|
||||
|
||||
Reference in New Issue
Block a user