notid/integreat
1
0
Fork 0
Code Issues Pull Requests 4 Actions Packages Projects Releases Wiki Activity

fix(square): do not resolve a record that belongs to another client

Browse Source 
Two clients on one Square location double a day's tender in the window
between deploying and finishing the migration. Reproduced end to end:

  1. charge X carries the legacy key square/charge/P and belongs to
     client B's order
  2. client A's payout import resolves X through existing-id's legacy
     fallback and renames it into A's scope
  3. client B's next order import matches neither scheme, so it mints a
     second charge
  4. :sales-order/charges is cardinality-many and orders transact as
     plain maps, so nothing retracts the first

B's order ends up holding two charges for one payment — $200 of tender
for a $100 payment — and running the migration on that state produces
square/charge/BBB-LB-AAA-LA-P, the same double-scoped shape that already
doubled tender on five clients once during this work.

existing-id's legacy branch now declines any record already owned by a
different client, reading the owner attribute and, for charges that
predate :charge/client, the client of the referencing order. Declining is
also correct on its merits: the write then lands on this client's own
copy, which is what the scoped keys exist to create. The payout path also
writes :charge/client/:charge/location alongside the key, so a charge's
scope and its owner can no longer disagree.

The guard is transitional and gets deleted with the legacy branch it
protects, at rollout step 9.

Rollout resequenced for the decision to leave duplicate client records
active: no deactivation, no "which record survives" call, and the risk
window closed by pausing the importer across deploy + migrate rather than
by removing one of the two writers. Both records converge to independent
stable histories once every key carries its owner.

Also from review:

- migrate-all! now collision-checks the charge pass like the other three
  attributes instead of discovering a clash mid-run over 17M rows
- split-and-rekey-charges! logs progress every 200 batches; an
  interrupted 19M-order run left no trail
- unscoped-report's docstring no longer promises a zero its :no-owner
  column cannot reach; plan is named as the authoritative signal
- the rollout's pre-flight asked for a :collisions key plan never
  returns, so it silently passed on every database
- the multi-parent gate sampled (take 400000 (all-order-ids db)), which
  streams :aevt — ascending entity id — and so read the OLDEST 2% of
  orders: 2019-12-31 to 2021-06-03, before any of the contention it
  looks for. Now every order of the last year via the client+date index,
  5,159,787 on the restored copy, reading 0
- the report claimed same-client pairs get copied once batches split
  them. They do not, at any batch size; verified at batch-size 1 and now
  pinned by a test

30 tests, 72 assertions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Bryce
2026-08-16 08:09:53 -07:00
parent 57a84dae11
commit 10d0d01b82
6 changed files with 375 additions and 180 deletions
Download Patch File Download Diff File Expand all files Collapse all files

27
test/clj/auto_ap/jobs/rekey_square_external_ids_test.clj
View File

@@ -147,3 +147,30 @@
(is (= (str "square/charge/" code "-CD-same1") (:key (first (charges-of o1)))))
(is (= (:eid (first (charges-of o1))) (:eid (first (charges-of o2))))
"both orders still point at the one payment"))))
(deftest two-orders-of-the-same-client-keep-sharing-across-batches
(testing "batch size does not change the same-client rule, which the sibling test cannot show
because both its orders land in one batch.
Once the first order re-keys the charge it also writes :charge/client/:charge/location,
so the second order's raw-square-id takes its owner branch, new-key reconstructs the key
the charge already has, and the (not= old new-key) guard drops the row before :action is
read. A clone here would double that client's takings for the day."
(let [{:strs [test-client-id]} (setup-test-data [])
tx @(dc/transact conn [{:db/id "charge"
:charge/external-id "square/charge/same2"
:charge/total 75.0}
{:db/id "o1" :sales-order/external-id "square/order/y-1"
:sales-order/client test-client-id :sales-order/location "CD"
:sales-order/date sales-date :sales-order/charges ["charge"]}
{:db/id "o2" :sales-order/external-id "square/order/y-2"
:sales-order/client test-client-id :sales-order/location "CD"
:sales-order/date sales-date :sales-order/charges ["charge"]}])
o1 (get-in tx [:tempids "o1"]) o2 (get-in tx [:tempids "o2"])
code (:client/code (dc/entity (dc/db conn) test-client-id))]
(is (= {:rekeyed 1 :cloned 0} (sut/split-and-rekey-charges! [o1 o2] 1))
"batch size 1 puts the two orders in separate batches, and still no copy is made")
(is (= 1 (charge-count)) "one payment, not two")
(is (= (str "square/charge/" code "-CD-same2") (:key (first (charges-of o1)))))
(is (= (:eid (first (charges-of o1))) (:eid (first (charges-of o2))))
"both orders still point at the one payment"))))

83
test/clj/auto_ap/square/core3_test.clj
View File

@@ -76,6 +76,89 @@
:sales-refund/total 10.0}])
(is (= 2 (refund-count)) "two stable entities, one per client, rather than one that flips"))))
(deftest legacy-key-of-another-client-is-not-claimed
(testing "the legacy fallback must not hand one client a record that already belongs to another.
Without this, two clients on one Square location double money in the window between
deploying and finishing the migration."
(setup-test-data [])
(let [tx @(dc/transact conn [{:db/id "mine" :client/code (str "MINE" (rand-int 100000))}
{:db/id "theirs" :client/code (str "THEIRS" (rand-int 100000))}])
mine {:db/id (get-in tx [:tempids "mine"]) :client/code "MINE"}
theirs-id (get-in tx [:tempids "theirs"])]
@(dc/transact conn [{:db/id "r"
:sales-refund/external-id "square/refund/abc"
:sales-refund/client theirs-id
:sales-refund/total 10.0}])
(is (nil? (sut/existing-id (dc/db conn) :sales-refund/external-id "square/refund/"
mine location "abc"))
"a legacy-keyed refund owned by another client is left alone")
(is (some? (sut/existing-id (dc/db conn) :sales-refund/external-id "square/refund/"
{:db/id theirs-id :client/code "THEIRS"} location "abc"))
"its own client still resolves it, so re-keying in place still works"))))
(deftest a-charge-is-owned-by-the-client-of-the-order-that-refers-to-it
(testing "charges predating :charge/client still have orders, and those are exactly the ones
that could be taken by the wrong client"
(let [{:strs [test-client-id]} (setup-test-data [])
other (get-in @(dc/transact conn [{:db/id "o" :client/code (str "OTHER" (rand-int 100000))}])
[:tempids "o"])]
@(dc/transact conn [{:db/id "c" :charge/external-id "square/charge/p1" :charge/total 50.0}
{:db/id "ord" :sales-order/external-id "square/order/x-1"
:sales-order/client test-client-id :sales-order/location "CD"
:sales-order/date #inst "2026-06-03T07:00:00.000-00:00"
:sales-order/charges ["c"]}])
(is (nil? (sut/existing-id (dc/db conn) :charge/external-id "square/charge/"
{:db/id other :client/code "OTHER"} location "p1"))
"ownership is read from the referencing order when :charge/client is absent"))))
(deftest deploy-window-does-not-double-a-second-clients-tender
(testing "the P0 this guard exists for, end to end.
Client A's payout import reaches for a payment whose charge belongs to client B's
order. If A were allowed to re-key it, B's next order import would match neither
scheme, mint a second charge, and — since :sales-order/charges is cardinality-many —
leave B's order holding two charges for one payment."
(let [{:strs [test-client-id]} (setup-test-data [])
b-code (:client/code (dc/entity (dc/db conn) test-client-id))
b {:db/id test-client-id :client/code b-code}
b-loc {:square-location/client-location "LB"}
a-id (get-in @(dc/transact conn [{:db/id "a" :client/code (str "AAA" (rand-int 100000))}])
[:tempids "a"])
a {:db/id a-id :client/code (:client/code (dc/entity (dc/db conn) a-id))}
a-loc {:square-location/client-location "LA"}
tx @(dc/transact conn [{:db/id "x" :charge/external-id "square/charge/P"
:charge/total 100.0 :charge/type-name "CARD"
:charge/client test-client-id :charge/location "LB"}
{:db/id "ob" :sales-order/external-id "square/order/b-1"
:sales-order/client test-client-id :sales-order/location "LB"
:sales-order/date #inst "2026-06-03T07:00:00.000-00:00"
:sales-order/charges ["x"]}])
order-b (get-in tx [:tempids "ob"])
charges-of (fn [o] (map :v (dc/datoms (dc/db conn) :eavt o :sales-order/charges)))]
;; client A's payout import touches the same Square payment
@(dc/transact conn [(into {} (remove (comp nil? val))
{:charge/external-id (sut/scoped-key "square/charge/" a a-loc "P")
:charge/client a-id
:charge/location "LA"
:db/id (sut/existing-id (dc/db conn) :charge/external-id
"square/charge/" a a-loc "P")})])
;; client B's order re-imports
@(dc/transact conn [{:db/id order-b
:sales-order/charges
[(sut/tender->charge {:id "b-1" :created_at "2026-06-03T12:00:00Z"}
b b-loc {:id "P" :type "CARD"
:amount_money {:amount 10000
:currency "USD"}})]}])
(is (= 1 (count (charges-of order-b)))
"B's order still holds exactly one charge for the one payment")
(is (= 100.0 (reduce + 0.0 (map #(:charge/total (dc/entity (dc/db conn) %))
(charges-of order-b))))
"so the day's tender is not doubled")
(is (= (str "square/charge/" b-code "-LB-P")
(:charge/external-id (dc/entity (dc/db conn) (first (charges-of order-b)))))
"and B's own charge was re-keyed in place rather than abandoned"))))
(deftest payouts-and-shifts-are-client-scoped-too
(testing "expected deposits and cash drawer shifts are fetched per location, so two clients on
one location collide on them exactly as refunds and charges did"