notid/integreat
1
0
Fork 0
Code Issues Pull Requests 4 Actions Packages Projects Releases Wiki Activity

feat(square): backfill history so shared-location records converge

Browse Source 
Re-keying stops two client records on one Square location fighting over a
record, but it does not make their books equal. Sales orders have always
been keyed by client, so each record built its own order history from the
start. Refunds, payouts and cash-drawer shifts were not, so only ONE
record holds each of them — whichever imported it last. The migration
freezes that ownership rather than evening it out.

The record left without them shows returns from its own orders and no
refunds against them. NGBK held 158,535 orders and five refunds. On
2026-05-11 both NGBK and NGBR held the same 221 orders; NGBK had no
refunds, NGBR had two worth $2,232.29, and NGBK was out by $2,232.29 to
the cent.

Rather than manufacture copies, ask Square again. Client-scoped keys mean
each record now creates its own copy of whatever it reads, so replaying a
window converges the two histories with no code inventing a duplicate.
`backfill-history` does that for a date range across orders, payouts,
refunds and shifts. After it, all ten pairs held matching order and
refund counts.

Fixes a capped read found by doing this: the refunds import asked Square
for a location's refunds and read only the first page — no cursor, no
date range. Square pages at a hundred, so a location with more refunds
silently returned a hundred and the response looked complete. That is why
each twin held almost exactly 100 refunds and why an earlier import added
exactly 1,000 across ten locations. `refunds`/`upsert-refunds` now follow
the cursor and accept a window.

Re-measured from the same fresh restore, duplicates left active:

  before  1,191 days out of balance / $70,276.50
  after     122 days out of balance / $2,379.45   (32 material)
            1,069 into balance, 0 out, 0 already-balanced days altered

The shared records went from 423 days / $18,508.39 to 3 days / $648.84 —
NGBK and NGBR at $299.42 each (the known tender-versus-order gap) and
NGDA at $50.00 (auto-gratuity as a service charge). The zero-regression
guarantee is restored too: the six days that broke without the backfill
were tips reversed on one record whose refund sat on the twin, and all
six closed once both sides had their own copy.

This beats retiring the duplicate records, which left 279 days and
$7,790.54, and it needs no decision about whose history to abandon.

Unchanged across every run: for the 190 clients that do not share a
location, 119 days and $1,730.61, same five restaurants.

Cost: 5.9 hours for ninety days across twenty records. Every Square call
shares one 25 req/s throttle, refunds and shifts cost one API call each,
and backfill-history imports three clients at a time. Reads are not the
limit — existing-id measures 32 microseconds.

31 tests, 76 assertions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Bryce
2026-08-16 17:58:59 -07:00
parent 4a1817711d
commit 1558851c18
3 changed files with 251 additions and 85 deletions
Download Patch File Download Diff File Expand all files Collapse all files

153
docs/2026-08-15-sales-summary-reconciliation.html
View File

@@ -105,27 +105,27 @@
<div class="ledger">
<div>
<span class="side-label">Today's calculation, ninety days re-run</span>
<span class="figure num">$81,023.96</span>
<span class="subfig"><span class="num">1,451</span> days out of balance · <span class="num">92.32%</span> clean</span>
<span class="figure num">$70,276.50</span>
<span class="subfig"><span class="num">1,191</span> days out of balance · <span class="num">93.70%</span> clean</span>
</div>
<div class="arrow" aria-hidden="true">→</div>
<div>
<span class="side-label">The same ninety days, fixes on</span>
<span class="figure after num">$20,239.00</span>
<span class="subfig"><span class="num">542</span> days out of balance · <span class="num">97.13%</span> clean</span>
<span class="figure after num">$2,379.45</span>
<span class="subfig"><span class="num">122</span> days out of balance · <span class="num">99.35%</span> clean</span>
</div>
</div>
<div class="stats">
<div class="stat"><span class="k num">915</span><span class="l">client-days brought into balance</span></div>
<div class="stat"><span class="k num">6</span><span class="l">days knocked out of balance</span></div>
<div class="stat"><span class="k num">75.0%</span><span class="l">of the variance removed</span></div>
<div class="stat"><span class="k num">1,069</span><span class="l">client-days brought into balance</span></div>
<div class="stat zero"><span class="k num">0</span><span class="l">days knocked out of balance</span></div>
<div class="stat"><span class="k num">96.6%</span><span class="l">of the variance removed</span></div>
<div class="stat zero"><span class="k num">0</span><span class="l">payments shared between two clients</span></div>
</div>
<div class="measure">
<p><strong>In one sentence:</strong> a day's sales summary should show the money taken and the money earned agreeing to the penny, and on roughly one trading day in eight it did not — because two clients were fighting over the same records, tips that had been refunded were still counted as income, and service charges customers paid were credited to nothing.</p>
<p><strong>How the two figures above were produced.</strong> Both are the real nightly job, run over the same ninety days against the same restored database, writing real summaries each time — the first pass with the fixes switched off, the second with them on. Comparing a re-run against a re-run rather than against production's stored summaries is the stricter test: production's figures are in places months stale, and crediting the fixes with repairing ordinary staleness would flatter them. On that fairer footing the fixes are worth <strong>a net 909 days and $60,784.96</strong>, not the larger number a stale baseline would have shown. Both passes ran with the duplicate client records left active, which is how this will actually be deployed.</p>
<p><strong>How the two figures above were produced.</strong> Both are the real nightly job, run over the same ninety days against the same restored database, writing real summaries each time — the first pass with the fixes switched off, the second with them on. Comparing a re-run against a re-run rather than against production's stored summaries is the stricter test: production's figures are in places months stale, and crediting the fixes with repairing ordinary staleness would flatter them. On that fairer footing the fixes are worth <strong>1,069 days and $67,897.05</strong>, not the larger number a stale baseline would have shown. Both passes ran with the duplicate client records left active, which is how this will actually be deployed.</p>
<p><strong>Most of what is left is not a balancing fault at all</strong>, and the section on the fourth problem explains why deliberately leaving it unbalanced is the right call.</p>
</div>
</section>
@@ -169,7 +169,7 @@
<p>The obvious reading is that the refund simply settled on a closed day. It is the wrong one. Checking each of those days against the date its client first recorded <em>any</em> order shows <strong>140 of 171 fall before that client had a single order in the system</strong> — for seven of the nine records affected, every single one does. These are not quiet days. They are periods where the sales were never imported at all.</p>
<p><strong>Where the refunds came from.</strong> Reading the database's own ownership history settles it. A $35.35 refund dated 26 February belonged to <span class="mono">NGDG</span> that same day, and was taken over by <span class="mono">NGDU</span> on 12 August. Others flip between the two records several times a day across 12–15 August. <span class="mono">NGDU</span>'s first order is 2 August; it holds 94 refunds dated before it existed as a trading record. It never made them — it inherited seven months of the other record's refunds, because the refund key carried no client and whichever import ran last took ownership. That is fault 1, seen from the other end.</p>
<p>Across the nine records, <strong>660 refunds worth $15,237.02 sit on a record dated before that record's first order.</strong> Nothing is lost and nothing is double-counted — the money is real and the surviving record has its own copy — but it is filed against a set of books that has no sales to put it against.</p>
<p><strong>Why it is deliberately left out of balance.</strong> The day can be closed in one line: book a return equal to the day's refunds whenever the client recorded no sales. It is safe by construction — no trading day could be touched — and on this data it closes 171 of the 542 remaining days and $5,795.18. It was built, measured, and then removed, because it is the wrong thing to do. An unbalanced day is the only visible signal that a restaurant's sales are not being imported. Making the arithmetic agree would remove the alarm and leave the fire.</p>
<p><strong>Why it is deliberately left out of balance.</strong> The day can be closed in one line: book a return equal to the day's refunds whenever the client recorded no sales. It is safe by construction — no trading day could be touched — and on this data it closes 16 of the 122 remaining days and $1,227.65. It was built, measured, and then removed, because it is the wrong thing to do. An unbalanced day is the only visible signal that a restaurant's sales are not being imported. Making the arithmetic agree would remove the alarm and leave the fire.</p>
<p><span class="tech">technical</span> <code>get-returns</code> sums <code>:sales-order/returns</code> over orders scanned for the date. With no orders the sum is nil and no <code>Returns</code> line is written, while <code>get-refund-items</code> still credits <code>Card Refunds</code> from the <code>sales-refund</code> records. The imbalance is the correct output for the input; the input is what is wrong. A test now pins this behaviour in place so it is not "fixed" by someone reading only the arithmetic.</p>
</div>
</div>
@@ -371,9 +371,9 @@
<table>
<thead><tr><th>Stage</th><th class="n">Days out of balance</th><th class="n">Clean</th><th class="n">Total variance</th></tr></thead>
<tbody>
<tr><td>Today's calculation, ninety days re-run</td><td class="n">1,451</td><td class="n">92.32%</td><td class="n">$81,023.96</td></tr>
<tr><td>+ refunded tips</td><td class="n">1,199</td><td class="n">93.66%</td><td class="n">$78,522.85</td></tr>
<tr class="total"><td>+ service charges</td><td class="n good">542</td><td class="n good">97.13%</td><td class="n good">$20,239.00</td></tr>
<tr><td>Today's calculation, ninety days re-run</td><td class="n">1,191</td><td class="n">93.70%</td><td class="n">$70,276.50</td></tr>
<tr><td>+ refunded tips</td><td class="n">890</td><td class="n">95.29%</td><td class="n">$67,032.09</td></tr>
<tr class="total"><td>+ service charges</td><td class="n good">122</td><td class="n good">99.35%</td><td class="n good">$2,379.45</td></tr>
</tbody>
</table>
</div>
@@ -386,15 +386,23 @@
<table>
<thead><tr><th>Change</th><th class="n">Unchanged</th><th class="n">Into balance</th><th class="n">Out of balance</th><th class="n">Balanced days altered</th><th class="n">Money moved</th></tr></thead>
<tbody>
<tr><td>Refunded tips</td><td class="n">18,588</td><td class="n good">252</td><td class="n bad">6</td><td class="n bad">6</td><td class="n">$3,777.67</td></tr>
<tr><td>Service charges</td><td class="n">18,204</td><td class="n good">663</td><td class="n good">0</td><td class="n good">0</td><td class="n">$58,923.85</td></tr>
<tr class="total"><td>Both, end to end</td><td class="n">17,916</td><td class="n good">915</td><td class="n bad">6</td><td class="n bad">6</td><td class="n">$60,784.96</td></tr>
<tr><td>Refunded tips</td><td class="n">18,571</td><td class="n good">301</td><td class="n good">0</td><td class="n good">0</td><td class="n">$4,027.21</td></tr>
<tr><td>Service charges</td><td class="n">18,129</td><td class="n good">768</td><td class="n good">0</td><td class="n good">0</td><td class="n">$64,752.64</td></tr>
<tr class="total"><td>Both, end to end</td><td class="n">17,827</td><td class="n good">1,069</td><td class="n good">0</td><td class="n good">0</td><td class="n">$67,897.05</td></tr>
</tbody>
</table>
</div>
<div class="measure">
<p><strong>Six days were knocked out of balance, and they are worth understanding rather than hiding.</strong> Across all 18,900 client-days, 17,916 summaries came out byte-identical and 978 of the 984 that moved were already wrong. The six exceptions all have one shape: the <code>Tip</code> line drops by a round amount — $10, $15, $20, $30, $30, $50 — and the day breaks by exactly that. They are all on shared-location records (NGDG, NGEZ, NGDA, NGBK).</p>
<p>That is the tip fix working, not failing. Each is a tip that was handed back: the reversal sits on this record's order, but the refund that should offset it went to the record's twin. Before the fix the day balanced by accident, because the reversal was ignored. After it, the day correctly shows that half the transaction is filed elsewhere. The honest description is that the fix converts a hidden mis-attribution into a visible one — which is the same trade the fourth problem below makes deliberately.</p>
<p><strong>Neither fix touched a day that was already correct.</strong> Across all 18,900
client-days no balanced day was knocked out of balance, and no balanced day had a single
figure altered — 17,827 summaries came out byte-identical, and every one of the 1,073 that
moved was already wrong.</p>
<p>That claim did not hold on the first attempt, and how it was recovered is the useful part.
Measured before the historical backfill described below, six days broke — all of them a tip
reversed on one record whose refund sat on its twin, so removing the un-reversed tip left the
day short by exactly that amount. Replaying the window from Square gave both records their own
copy of every refund, and all six closed. The fix was never wrong; it was reading half a
transaction.</p>
<p>Service charges are by far the larger of the two fixes, moving $58,923.85 against the tip fix's $3,777.67.</p>
<p>That claim is stronger than a balance check, and it is the one worth insisting on: a day can stay balanced while its individual lines move, which would still be a change to the books. Every line of every summary was compared — category, debit or credit side, amount to the cent, and account — not just the day's bottom line.</p>
<p><strong>The two fixes account for every day they moved, exactly.</strong> Adding up the untendered-tip and service-charge amounts across all 984 changed days leaves a residue of <span class="mono">0.0000000002</span>. Nothing else moved those days; there is no unexplained remainder hiding a third effect, and the six that broke are accounted for by the same arithmetic as the 915 that healed.</p>
@@ -477,12 +485,12 @@
</div>
<div class="measure">
<p>The count checks are the ones that matter. If re-keying had gone wrong it would have created a second copy of every record rather than updating the existing one, and the totals would have doubled. They did not move. The payment-copy step is the exception and is meant to add records — it added 500,438, matching the number of copies it reported making. (Close, not exact: the counter increments while the transaction is being assembled, so two copies that resolve onto one entity are counted twice. It is a good check, not a proof.)</p>
<p><strong>The measurement above was taken with the duplicate client records deactivated, and that is not how this will be deployed.</strong> Leaving both records live is the intended configuration — the re-key is what separates them — but it means the deactivation that made this measurement clean will not be there. The gap that opens is narrow and specific: while any record still carries a legacy key, a second client can resolve onto it. That is why the deployment runs the migration with imports paused, and why <code>existing-id</code> now refuses to resolve a record belonging to another client.</p>
<p><strong>The measurement above was taken with both client records of each pair left live, which is how this deploys.</strong> Nothing is deactivated and no business decision about which restaurant's history survives is needed. The risk that opens is narrow and specific: while any record still carries a legacy key, a second client can resolve onto it. That is why the deployment runs the migration with imports paused, and why <code>existing-id</code> now refuses to resolve a record belonging to another client.</p>
</div>
<div class="callout">
<span class="h">The whole analysis was run again from nothing, and landed in the same place</span>
<p>Everything above was rebuilt from a fresh restore of the production backup, three times over, each time from the backup point itself rather than from a database an earlier run had touched: restore, re-key and split across all nineteen million orders, then two full ninety-day recomputes. The runs used different preparation — one deactivated the duplicate records and ran a live Square import, this one leaves the configuration exactly as production has it — so their headline figures differ, and that difference is itself the most useful measurement in this report. What did <strong>not</strong> move is the part that should not: for the 190 clients that do not share a Square location, the residue is 119 days and $1,730.61 in both runs, with the same five restaurants accounting for it. The arithmetic fixes behave identically no matter what is done to the duplicates.</p>
<p>Everything above was rebuilt from a fresh restore of the production backup, several times over, each time from the backup point itself rather than from a database an earlier run had touched: restore, re-key and split across all nineteen million orders, then two full ninety-day recomputes. The runs used deliberately different preparation — one deactivated the duplicate records, one left them live and untouched, one backfilled their history from Square — so their headline figures differ, and comparing them is how the recommendation below was reached. What did <strong>not</strong> move is the part that should not: for the 190 clients that do not share a Square location the residue is 119 days and $1,730.61 in every run, with the same five restaurants accounting for it. The arithmetic fixes behave identically no matter what is done to the duplicates, which is a stronger check on them than any single measurement.</p>
</div>
<div class="callout warn">
@@ -494,48 +502,89 @@
<section>
<h2>What is still out of balance</h2>
<div class="measure">
<p>542 client-days out of 18,900, totalling <strong>$20,239.00</strong>. Almost all of it sits
on one group of restaurants, and that is the finding:</p>
<p>122 client-days out of 18,900, totalling <strong>$2,379.45</strong> — and only 32 of
those are above ten cents.</p>
</div>
<div class="scroll">
<table>
<thead><tr><th>Where the remainder sits</th><th class="n">Days</th><th class="n">Variance</th><th class="n">Share</th></tr></thead>
<thead><tr><th>Where the remainder sits</th><th class="n">Days</th><th class="n">Variance</th></tr></thead>
<tbody>
<tr><td><strong>The twenty records that share a Square location</strong></td><td class="n">423</td><td class="n">$18,508.39</td><td class="n">91%</td></tr>
<tr class="total"><td>Every other client — 190 of the 210</td><td class="n good">119</td><td class="n good">$1,730.61</td><td class="n">9%</td></tr>
<tr><td>The twenty records that share a Square location</td><td class="n good">3</td><td class="n good">$648.84</td></tr>
<tr class="total"><td>Every other client — 190 of the 210</td><td class="n">119</td><td class="n">$1,730.61</td></tr>
</tbody>
</table>
</div>
<div class="measure">
<p><strong>For every restaurant that is not one of the ten duplicated pairs, this reproduces
to the penny.</strong> 119 days and $1,730.61, with the same five clients accounting for
almost all of it — NG4S at $1,066.61, NGMV at $259.38, NGEB at $199.09, NGPS at $172.82,
N-30012 at $30.31, and 91 further days totalling $2.40 of till rounding. Those are the
numbers an earlier run produced on a differently prepared database, which is a stronger
check on the two arithmetic fixes than any single measurement: they behave identically
whatever is done to the duplicates.</p>
<p><strong>The $18,508.39 on the twenty shared-location records is the cost of leaving both
records live.</strong> Each restaurant now keeps two sets of books, and the history behind
them was never split: refunds claimed by whichever record imported them first, orders that
went to the other, tips reversed on one side and refunded on the other. Re-keying makes that
attribution <em>stable</em> — it stops moving — but it does not make it <em>right</em>.</p>
<p>Of that remainder, 155 days and $4,567.53 are days those records had no sales imported at
all, which is the fourth problem above and is deliberately left visible. The other 268 days
are real trading days on which the two records disagree about who owns what.</p>
<p><strong>The shared-location records are now the clean part of the book.</strong> Three days
between all twenty of them: NGBK and NGBR at $299.42 each on 2026-08-06, which is the Square
tender-versus-order-total gap described below and not an attribution fault, and NGDA at $50.00,
an auto-gratuity booked as a service charge. Before the backfill those same records carried
423 days and $18,508.39.</p>
<p><strong>The other 119 days have not moved across any run of this analysis.</strong> Four
separate rebuilds — different databases, different preparation, one with the duplicates
deactivated and one without — all land on 119 days and $1,730.61, with the same five
restaurants accounting for almost all of it:</p>
</div>
<div class="scroll">
<table>
<thead><tr><th>Client</th><th class="n">Days</th><th class="n">Variance</th><th>What it is</th></tr></thead>
<tbody>
<tr><td class="mono">NG4S</td><td class="n">10</td><td class="n">$1,066.61</td><td>refunds arriving for a record with no sales imported — the fourth problem</td></tr>
<tr><td class="mono">NGMV</td><td class="n">5</td><td class="n">$259.38</td><td>late May, undiagnosed</td></tr>
<tr><td class="mono">NGEB</td><td class="n">4</td><td class="n">$199.09</td><td>ezCater fee treatment — an open question</td></tr>
<tr><td class="mono">NGPS</td><td class="n">7</td><td class="n">$172.82</td><td>undiagnosed</td></tr>
<tr><td class="mono">N-30012</td><td class="n">2</td><td class="n">$30.31</td><td>late May, undiagnosed</td></tr>
<tr class="total"><td class="dim">everyone else</td><td class="n dim">91</td><td class="n dim">$2.40</td><td class="dim">till rounding — pennies a day</td></tr>
</tbody>
</table>
</div>
<div class="measure">
<p>Sixteen of the 122 are days a record had no sales imported at all, worth $1,227.65 — the
fourth problem, still deliberately visible. The clusters on NGMV, NGPS and NGEB are
unexplained and worth a look, though at under $650 across sixteen days they are no longer
urgent.</p>
</div>
</section>
<section>
<h2>Making the duplicated restaurants match</h2>
<div class="measure">
<p>Re-keying stops the two records fighting, but on its own it does not make them equal, and
the difference is worth stating plainly because it decides whether the books close.</p>
<p><strong>Orders were always duplicated; refunds never were.</strong> A sales order's
identifier has always carried its client, so each of the two records built its own order
history from the start. Refunds, payouts and cash-drawer shifts did not, so only ONE record
holds each of them — whichever imported it last. The migration freezes that ownership rather
than evening it out. The record left without them shows returns from its own orders and no
refunds to set against them, and is out of balance by exactly the amount its twin is holding.</p>
<p>On 2026-05-11 both NGBK and NGBR held the same 221 orders. NGBK had no refunds; NGBR had
two, worth $2,232.29; and NGBK's books were out by $2,232.29 to the cent. Across the whole
database NGBK held <strong>158,535 orders and five refunds</strong>.</p>
</div>
<div class="callout">
<span class="h">The fix is to ask Square again, not to manufacture copies</span>
<p>With client-scoped keys in place, every record now creates its own copy of whatever it
reads. So replaying the window from Square is all that is needed: each record imports the same
refunds independently and the two histories converge, without any code inventing a duplicate
and having to be trusted about it. <code>backfill-history</code> does exactly that for a date
range, and after it every one of the ten pairs held matching order and refund counts.</p>
<p>It closed <strong>420 of the 423 days</strong> the shared records were carrying, and
$17,859.55 of the $18,508.39. It is also what recovered the zero-regression guarantee above.</p>
</div>
<div class="callout warn">
<span class="h">What retiring the duplicate records would be worth</span>
<p>An earlier measurement of the same window, taken with one record of each pair deactivated
and a live Square import run afterwards, left <strong>279 days and $7,790.54</strong> instead
of 542 and $20,239.00. Most of that difference is the twenty shared records: with the
duplicates retired their share fell from $18,508.39 to $6,059.93.</p>
<p>That is not an argument that the configuration is wrong — two live records is a deliberate
choice, and the re-key is what makes it safe. It is a number to weigh: leaving both active
costs roughly <strong>$12,000 of unexplained variance across 260 client-days</strong> per
ninety days, carried on ten restaurants, until the historical attribution behind them is
redistributed. The two figures are not perfectly isolated — that earlier run also included a
live import, which backfilled data this one does not have — so treat it as the right order of
magnitude rather than an exact price.</p>
<span class="h">One capped read, found by doing this</span>
<p>The refunds import asked Square for a location's refunds and read the first page of the
answer — no cursor, no date range. Square pages at a hundred, so a location with more than a
hundred refunds silently returned a hundred, and the response looked complete. That is why the
twins each held almost exactly 100 refunds, and why an earlier import added exactly 1,000
across ten locations. Following the cursor is a few lines; the reason it went unnoticed for so
long is that a capped list is indistinguishable from a short one.</p>
</div>
<div class="measure">
<p><strong>This turned out to be a better answer than retiring the duplicate records.</strong>
An earlier measurement that deactivated one record of each pair left 279 days and $7,790.54.
Backfilling instead, with both records live, leaves <strong>122 days and $2,379.45</strong> —
and it needs no business decision about which restaurant's history to abandon.</p>
</div>
</section>
@@ -704,7 +753,7 @@
<footer>
<span>Measured 2026-08-15 against <span class="mono">integreat-prod-restore</span>, restored fresh from backup point 209608347 — production as of 2026-08-14 22:52. Nothing in production was read or written. Branch <span class="mono">worktree-sales-summary-balance</span>.</span>
<span>A day counts as out of balance when money taken minus money earned is half a penny or more. "Material" means ten cents or more, the threshold below which the residual is till rounding. Of the 542 remaining days, 452 are material — but 171 are days with no sales imported rather than arithmetic faults, and 423 of the 542 sit on the twenty records that share a Square location.</span>
<span>A day counts as out of balance when money taken minus money earned is half a penny or more. "Material" means ten cents or more, the threshold below which the residual is till rounding. Of the 122 remaining days only 32 are material, and just 3 of them sit on the twenty records that share a Square location.</span>
<span>Both the baseline and the result are live captures taken straight after their own recompute, never historical reads — see the note on <code>as-of</code> above.</span>
</footer>

94
docs/2026-08-15-sales-summary-rollout-plan.md
View File

@@ -6,16 +6,17 @@ that is **off by default**, so merging and deploying this branch changes nothing
Measured on a restored copy of production (backup point `209608347`), 210 clients over
2026-05-10 → 2026-08-07, with the duplicate client records left active exactly as they will be in
production: **1,451 client-days out of balance / $81,023.96 → 542 days / $20,239.00**. 915 days
came into balance and 6 broke — all six being tips reversed on one record whose refund sits on its
twin, which the fix makes visible rather than causes.
production: **1,191 client-days out of balance / $70,276.50 → 122 days / $2,379.45**, of which only
32 are above ten cents. 1,069 days came into balance, none broke, and no already-balanced day had a
figure altered.
**Where the remaining $20,239.00 sits is the thing to read.** $18,508.39 of it — 91% — is on the
twenty records that share a Square location. For the other 190 clients the residue is 119 days and
$1,730.61, which reproduces an earlier run on a differently prepared database to the penny.
Of the $2,379.45 left, just **$648.84 across 3 days** is on the twenty shared-location records. The
other 119 days and $1,730.61 belong to ordinary clients and have not moved across any run of this
analysis.
Of the total, 171 days are not balancing faults at all — they are days where a client's sales were
never imported while its refunds were. Step 10 is about those.
Getting the shared records there needs step 5 — a historical backfill from Square. Without it they
carry 423 days and $18,508.39, because re-keying stops the two records fighting but does not give
each its own copy of the refunds.
---
@@ -40,7 +41,7 @@ duplicate to clean up. If any report or export aggregates across client records,
takings would be counted twice at that layer. Nothing in this work changes that either way.
**The only window of risk is between deploying and finishing the migration**, while legacy keys
still exist for a client to resolve. Steps 2–5 exist to make that window effectively zero.
still exist for a client to resolve. Steps 2–6 exist to make that window effectively zero.
---
@@ -61,7 +62,7 @@ See `docs/2026-08-15-remove-voided-orders-risk.md`.
## Step 2 — Pause the Square importer
**This is what makes the deploy safe, and it is easy to skip.** Steps 2 through 5 should be one
**This is what makes the deploy safe, and it is easy to skip.** Steps 2 through 6 should be one
maintenance action, not separate days' work.
While legacy keys exist, `square.core3/existing-id` falls back to them — and at a shared location
@@ -70,7 +71,7 @@ paused means no client is resolving keys while the keys are being rewritten, so
entirely rather than merely narrowing.
The migration itself takes about **38 minutes** for all 19M orders, so the pause is short — and
if you need it shorter, see step 5: you can resume imports before it finishes.
if you need it shorter, see step 6: you can resume imports before it finishes.
---
@@ -88,7 +89,7 @@ and the importer's new client-scoped keys.
**The importer reads both key schemes**, so the deploy does not depend on the migration having
finished. Two protections cover the interval before it does: imports are paused (step 2), and
`existing-id` refuses to resolve a record that already belongs to a different client. Do not remove
the legacy lookup yet — see step 9.
the legacy lookup yet — see step 10.
---
@@ -157,7 +158,7 @@ for charges, because ~283k payout stubs carry no `:charge/client` of their own a
by attribute rather than by resolving ownership. Judge completeness by `plan`.
**(b) The safety gate for the cascade** — no payment may answer to two orders, or re-enabling
`remove-voided-orders` in step 8 can delete a payment another order still needs. Check **every**
`remove-voided-orders` in step 9 can delete a payment another order still needs. Check **every**
order in the last year, with no sampling:
```clojure
@@ -181,9 +182,54 @@ lives. Completeness across all of history is check (a)'s job, not this one.
---
## Step 5 — Resume the Square importer
## Step 5 — Backfill the shared-location clients from Square
Normally: once step 4's two checks read clean. The maintenance window ends here.
**Skip this and the ten duplicated restaurants stay badly out of balance.** It is the difference
between 122 client-days out of balance and 542.
Sales orders have always been keyed by client, so both records of a pair built their own order
history. Refunds, payouts and cash-drawer shifts were not, so only ONE record holds each of them.
Re-keying freezes that ownership; it does not even it out. The record left without them shows
returns from its own orders and no refunds against them — NGBK held 158,535 orders and five
refunds — and is out of balance by exactly what its twin is holding.
Rather than manufacture copies, ask Square again. Client-scoped keys mean each record now creates
its own copy of whatever it reads, so replaying the window makes the two histories converge:
```clojure
(require '[auto-ap.square.core3 :as sq])
(require '[clj-time.core :as t])
@(apply sq/backfill-history
(t/date-time 2026 5 10) (t/date-time 2026 8 9)
["NGBK" "NGBR" "NGCD" "NGCC" "NGVG" "NGVC" "NGEZ" "NGJS" "NGDG" "NGDU"
"NGDV" "NGDS" "NGWC" "NGWN" "NGHY" "NGHA" "NGDA" "NGDL" "NGCL" "NGCT"])
```
**Verify** — every pair should hold matching order and refund counts in the window:
```clojure
;; per pair, per side: window orders and window refunds. The two sides should agree.
```
Measured on the restored copy: all ten pairs matched afterwards, and the shared records went from
423 days and $18,508.39 out of balance to 3 days and $648.84.
**Budget an overnight run.** This took **5.9 hours** for ninety days across the twenty records.
Every Square call in the process shares one 25-requests-per-second throttle, refunds and shifts cost
one API call per record, and `backfill-history` imports three clients at a time — raise its
`s/buffer` if you need it faster. Neither the database nor the transactor is the limit; reads
measured at 32 µs.
It must run **after** the migration. Run before, and it imports against legacy keys and leaves more
to migrate.
---
## Step 6 — Resume the Square importer
Normally: once step 4's two checks read clean and step 5's backfill has finished. The maintenance
window ends here.
**If the migration did not finish**, you do not have to wait for it. Resume imports once the
`::month-complete` log covers the window your importer reads — the last 75 days for payouts and
@@ -207,7 +253,7 @@ takings rather than merely duplicating a row.
---
## Step 6 — Recompute summaries, flags still off
## Step 7 — Recompute summaries, flags still off
```clojure
(require '[auto-ap.jobs.sales-summaries :as ss])
@@ -215,7 +261,7 @@ takings rather than merely duplicating a row.
```
This is the pass that banks the deduplication. **Capture the result before going further** — you
will need it as the baseline for step 7, and it cannot be reconstructed afterwards:
will need it as the baseline for step 8, and it cannot be reconstructed afterwards:
```clojure
(require '[auto-ap.tools.compare-sales-summaries :as cmp]) ; test/dev classpath
@@ -230,7 +276,7 @@ will need it as the baseline for step 7, and it cannot be reconstructed afterwar
---
## Step 7 — Turn the flag on, a few restaurants at a time
## Step 8 — Turn the flag on, a few restaurants at a time
Needs accounting sign-off first: `summary-service-charges` posts to **49000 Service Income**, chosen
so the work could be measured. It affects reporting, never whether a day balances.
@@ -243,7 +289,7 @@ so the work could be measured. It affects reporting, never whether a day balance
Start with two or three restaurants, confirm, then widen.
**Verify** against the capture from step 6:
**Verify** against the capture from step 7:
```clojure
(def after (cmp/summaries-in (d/db conn) start end))
@@ -255,18 +301,18 @@ The two numbers that matter — both were zero across all 18,900 client-days in
- `:balanced->unbalanced` must be **0**
- previously-balanced days whose lines changed must be **0**
If either is non-zero, retract the flag for the affected clients and re-run step 6. The flag is the
If either is non-zero, retract the flag for the affected clients and re-run step 7. The flag is the
rollback: removing it restores today's behaviour exactly.
---
## Step 8 — Re-enable `remove-voided-orders`
## Step 9 — Re-enable `remove-voided-orders`
Safe once step 4's gate reads zero. Keep the detach-rather-than-delete guard from step 1.
---
## Step 9 — Remove the legacy key lookup
## Step 10 — Remove the legacy key lookup
**Schedule this; do not leave it open-ended.** Both client records at a shared location stay active
permanently, so the legacy fallback in `square.core3/existing-id` is the one code path that can ever
@@ -282,7 +328,7 @@ people doing the right thing. That is the reason not to let this drift.
---
## Step 10 — Deal with the refunds that have no sales behind them
## Step 11 — Deal with the refunds that have no sales behind them
**The most important item in this document, and the only one that is not just execution.**
@@ -350,7 +396,7 @@ tender against $6,059.57 of order totals on one day), the ezCater fee question,
clusters on NGMV and NGEB.
Plus 16 days / $1,227.65 where a processing fee lands on a day with no trading — the same shape as
step 10 but from the payout side, so it needs the payout modelled rather than a rule in the summary.
step 11 but from the payout side, so it needs the payout modelled rather than a rule in the summary.
---

85
src/clj/auto_ap/square/core3.clj
View File

@@ -632,15 +632,44 @@
(log/error ::transform-payout-failed
:exception e)))))))))
(defn refunds
([client l]
(de/chain (manifold-api-call {:url (str "https://connect.squareup.com/v2/refunds?location_id=" (:square-location/square-id l))
:method :get
(defn- refund-list
"Every refund Square has for this location in `[start end]`, following the cursor to the end.
The list endpoint returns one page at a time. Reading only the first page — which is what this
did before — silently caps a location at a hundred refunds however many it actually has, and
the cap is invisible: the response looks like a complete answer. On a shared location that is
how one client record ends up holding a few refunds against a hundred and fifty thousand orders.
`start`/`end` are optional; omitting both asks for everything, which is what the nightly job
wants and what a historical backfill of more than a page needs."
([client l start end] (refund-list client l start end nil))
([client l start end cursor]
(de/chain (manifold-api-call
{:url (str "https://connect.squareup.com/v2/refunds"
"?"
(url/map->query
(cond-> {:location_id (:square-location/square-id l)
:limit 100}
start (assoc :begin_time (->square-date start))
end (assoc :end_time (->square-date end))
cursor (assoc :cursor cursor))))
:method :get
:headers (client-base-headers client)
:as :json})
:body
:refunds
(fn [result]
(log/info ::refunds-page
:count (count (:refunds result))
:more? (boolean (not-empty (:cursor result))))
(if (not-empty (:cursor result))
(de/chain (refund-list client l start end (:cursor result))
(fn [more] (concat (:refunds result) more)))
(:refunds result))))))
(defn refunds
([client l] (refunds client l nil nil))
([client l start end]
(de/chain (refund-list client l start end)
(fn [refunds]
(->> refunds
(filter (fn [r] (= "COMPLETED" (:status r))))
@@ -711,11 +740,12 @@
(for [square-location (:client/square-locations client)
:when (:square-location/client-location square-location)]
(upsert-refunds client square-location))))
([client location]
([client location] (upsert-refunds client location nil nil))
([client location start end]
(with-context-as {:source "Square refunds loading"
:client (:client/code client)} lc
(de/chain (refunds client location)
(de/chain (refunds client location start end)
(fn [refunds]
(mu/with-context lc
(try
@@ -1011,6 +1041,47 @@
(s/realize-each)
(s/reduce conj []))))
(defn backfill-history
"Re-imports orders, payouts, refunds and cash-drawer shifts for `[start end]`, one client at a
time, for every square location the client has.
This exists for the shared-location case. Sales orders have always been keyed by client, so two
client records on one Square location each built their own order history. Refunds, payouts and
shifts were not, so only ONE of the two records holds each of them — whichever imported it last
before the keys were scoped. Re-keying freezes that ownership; it does not even it out, and the
record left without them shows returns from its own orders with no refunds to offset them.
Rather than manufacture copies, this asks Square again. With client-scoped keys in place every
record now creates its own copy of what it reads, so replaying the window is what makes the two
histories match. Deliberately not part of `upsert-all`: it walks further back than the nightly
job and is meant to be run once, after the migration.
Run it AFTER `rekey-square-external-ids/migrate-all!`. Running it before would import against
legacy keys and leave more to migrate."
[start end & client-codes]
(with-context-as {:source "Square historical backfill"} lc
(->> (apply get-square-clients client-codes)
(s/->source)
(s/map (fn [client]
(with-context-as (merge lc {:client (:client/code client)}) lc
(->
(apply de/zip
(for [l (:client/square-locations client)
:when (:square-location/client-location l)]
(de/chain
(upsert client l start end)
(fn [_] (upsert-payouts client l start end))
(fn [_] (upsert-refunds client l start end))
(fn [_] (upsert-cash-shifts client l start end))
(fn [_] (log/info ::backfilled
:location (:square-location/client-location l))))))
(de/catch (fn [e]
(mu/with-context lc
(log/info ::backfill-failed :severity :error :exception e))))))))
(s/buffer 3)
(s/realize-each)
(s/reduce conj []))))
(defn do-upsert-all [& clients]
(mu/trace
::upsert-all