notid/integreat
1
0
Fork 0
Code Issues Pull Requests 4 Actions Packages Projects Releases Wiki Activity

docs: re-measure from a fresh restore with the duplicates left active

Browse Source 
The report's figures were taken with one client record of each shared
pair deactivated and a live Square import run afterwards. That is no
longer how this deploys — both records stay live and the re-key is what
separates them — so the numbers described a configuration that will not
exist.

Re-run from scratch: fresh restore of backup point 209608347 (verified
back to 16,545,495 charges and legacy keys before starting), the new
month-wise migration, then two full ninety-day recomputes.

  before  1,451 days out of balance / $81,023.96
  after     542 days out of balance / $20,239.00
            915 into balance, 6 out, 17,916 summaries untouched

Two corrections to claims that no longer hold:

- "zero days knocked out of balance" is now six. All six are the same
  shape — the Tip line falls by a round amount and the day breaks by
  exactly that — and all six are on shared-location records. Each is a
  tip reversed on one record whose refund went to its twin: the fix makes
  a hidden mis-attribution visible rather than causing one.
- the migration takes ~38 minutes, not ~13. The month-wise walk adds
  per-month index overhead, and the earlier figure predated it.

What did NOT move is the part that should not. For the 190 clients that
do not share a Square location the residue is 119 days and $1,730.61 in
both runs, with the same five restaurants accounting for it (NG4S, NGMV,
NGEB, NGPS, N-30012). The arithmetic fixes behave identically whatever is
done to the duplicates, which is a stronger check than either run alone.

$18,508.39 of the $20,239.00 — 91% — sits on the twenty shared-location
records. The report now states plainly what retiring the duplicates would
be worth (~$12,000 of variance across ~260 client-days per ninety days),
while noting the comparison is not perfectly isolated because the earlier
run also included a live import.

Migration re-measured: 16,236,839 re-keyed, 500,438 cloned, plan reports
{:total 17045933 :to-migrate 0 :already-scoped 17045933 :unscopable 0},
and the multi-parent gate reads 0 across all 5,158,470 orders of the last
year. Step 10's refunds re-measured too: 660 worth $15,237.02 dated
before their own record's first order, 140 of the 171 no-sales days
falling before it.

31 tests, 76 assertions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Bryce
2026-08-16 09:59:32 -07:00
parent 967e77e443
commit 4a1817711d
2 changed files with 98 additions and 77 deletions
Download Patch File Download Diff File Expand all files Collapse all files

131
docs/2026-08-15-sales-summary-reconciliation.html
View File

@@ -105,27 +105,27 @@
<div class="ledger">
<div>
<span class="side-label">Today's calculation, ninety days re-run</span>
<span class="figure num">$69,560.10</span>
<span class="subfig"><span class="num">1,258</span> days out of balance · <span class="num">93.34%</span> clean</span>
<span class="figure num">$81,023.96</span>
<span class="subfig"><span class="num">1,451</span> days out of balance · <span class="num">92.32%</span> clean</span>
</div>
<div class="arrow" aria-hidden="true">→</div>
<div>
<span class="side-label">The same ninety days, fixes on</span>
<span class="figure after num">$7,790.54</span>
<span class="subfig"><span class="num">279</span> days out of balance · <span class="num">98.52%</span> clean</span>
<span class="figure after num">$20,239.00</span>
<span class="subfig"><span class="num">542</span> days out of balance · <span class="num">97.13%</span> clean</span>
</div>
</div>
<div class="stats">
<div class="stat"><span class="k num">979</span><span class="l">client-days brought into balance</span></div>
<div class="stat zero"><span class="k num">0</span><span class="l">days knocked out of balance</span></div>
<div class="stat"><span class="k num">88.8%</span><span class="l">of the variance removed</span></div>
<div class="stat"><span class="k num">915</span><span class="l">client-days brought into balance</span></div>
<div class="stat"><span class="k num">6</span><span class="l">days knocked out of balance</span></div>
<div class="stat"><span class="k num">75.0%</span><span class="l">of the variance removed</span></div>
<div class="stat zero"><span class="k num">0</span><span class="l">payments shared between two clients</span></div>
</div>
<div class="measure">
<p><strong>In one sentence:</strong> a day's sales summary should show the money taken and the money earned agreeing to the penny, and on roughly one trading day in eight it did not — because two clients were fighting over the same records, tips that had been refunded were still counted as income, and service charges customers paid were credited to nothing.</p>
<p><strong>How the two figures above were produced.</strong> Both are the real nightly job, run over the same ninety days against the same restored database, writing real summaries each time — the first pass with the fixes switched off, the second with them on. Comparing a re-run against a re-run rather than against production's stored summaries is the stricter test: production's figures are in places months stale, and crediting the fixes with repairing ordinary staleness would flatter them. On that fairer footing the fixes are worth <strong>979 days and $61,769.56</strong>, not the larger number a stale baseline would have shown.</p>
<p><strong>How the two figures above were produced.</strong> Both are the real nightly job, run over the same ninety days against the same restored database, writing real summaries each time — the first pass with the fixes switched off, the second with them on. Comparing a re-run against a re-run rather than against production's stored summaries is the stricter test: production's figures are in places months stale, and crediting the fixes with repairing ordinary staleness would flatter them. On that fairer footing the fixes are worth <strong>a net 909 days and $60,784.96</strong>, not the larger number a stale baseline would have shown. Both passes ran with the duplicate client records left active, which is how this will actually be deployed.</p>
<p><strong>Most of what is left is not a balancing fault at all</strong>, and the section on the fourth problem explains why deliberately leaving it unbalanced is the right call.</p>
</div>
</section>
@@ -165,11 +165,11 @@
<div class="problem">
<h4>4. Refunds on records whose sales were never imported <span class="tech">not fixed — deliberately</span></h4>
<div class="measure">
<p><strong>This is why the duplicated restaurants looked so much worse than everyone else.</strong> Of the days still failing after the first three fixes, <strong>155 of the 158 on duplicate records had no sales orders at all</strong> — the summary consisted of nothing but refunds and their fees, with no sales for them to reduce.</p>
<p>The obvious reading is that the refund simply settled on a closed day. It is the wrong one. Checking each of those days against the date its client first recorded <em>any</em> order shows <strong>132 of 156 fall before that client had a single order in the system</strong> — for seven of the nine records affected, every single one does. These are not quiet days. They are periods where the sales were never imported at all.</p>
<p><strong>This is why the duplicated restaurants looked so much worse than everyone else.</strong> Of the days still failing after the first three fixes, <strong>155 of the 423 on shared-location records had no sales orders at all</strong> — the summary consisted of nothing but refunds and their fees, with no sales for them to reduce.</p>
<p>The obvious reading is that the refund simply settled on a closed day. It is the wrong one. Checking each of those days against the date its client first recorded <em>any</em> order shows <strong>140 of 171 fall before that client had a single order in the system</strong> — for seven of the nine records affected, every single one does. These are not quiet days. They are periods where the sales were never imported at all.</p>
<p><strong>Where the refunds came from.</strong> Reading the database's own ownership history settles it. A $35.35 refund dated 26 February belonged to <span class="mono">NGDG</span> that same day, and was taken over by <span class="mono">NGDU</span> on 12 August. Others flip between the two records several times a day across 12–15 August. <span class="mono">NGDU</span>'s first order is 2 August; it holds 94 refunds dated before it existed as a trading record. It never made them — it inherited seven months of the other record's refunds, because the refund key carried no client and whichever import ran last took ownership. That is fault 1, seen from the other end.</p>
<p>Across the nine records, <strong>659 refunds worth $15,225.24 sit on a record dated before that record's first order.</strong> Nothing is lost and nothing is double-counted — the money is real and the surviving record has its own copy — but it is filed against a set of books that has no sales to put it against.</p>
<p><strong>Why it is deliberately left out of balance.</strong> The day can be closed in one line: book a return equal to the day's refunds whenever the client recorded no sales. It is safe by construction — no trading day could be touched — and it closes 156 of the 279 remaining days and $4,820.19. It was built, measured, and then removed, because it is the wrong thing to do. An unbalanced day is the only visible signal that a restaurant's sales are not being imported. Making the arithmetic agree would remove the alarm and leave the fire.</p>
<p>Across the nine records, <strong>660 refunds worth $15,237.02 sit on a record dated before that record's first order.</strong> Nothing is lost and nothing is double-counted — the money is real and the surviving record has its own copy — but it is filed against a set of books that has no sales to put it against.</p>
<p><strong>Why it is deliberately left out of balance.</strong> The day can be closed in one line: book a return equal to the day's refunds whenever the client recorded no sales. It is safe by construction — no trading day could be touched — and on this data it closes 171 of the 542 remaining days and $5,795.18. It was built, measured, and then removed, because it is the wrong thing to do. An unbalanced day is the only visible signal that a restaurant's sales are not being imported. Making the arithmetic agree would remove the alarm and leave the fire.</p>
<p><span class="tech">technical</span> <code>get-returns</code> sums <code>:sales-order/returns</code> over orders scanned for the date. With no orders the sum is nil and no <code>Returns</code> line is written, while <code>get-refund-items</code> still credits <code>Card Refunds</code> from the <code>sales-refund</code> records. The imbalance is the correct output for the input; the input is what is wrong. A test now pins this behaviour in place so it is not "fixed" by someone reading only the arithmetic.</p>
</div>
</div>
@@ -226,9 +226,8 @@
<p>The result is pinned as the record's id on the way in, so the write lands on the existing
row regardless of which name it currently carries. The proof this worked is a count that did
not move. Every one of the 265,965 refunds, payouts and cash-drawer shifts in the database was
re-named, and afterwards there were still exactly <strong>144,688 payouts and 69,291 cash-drawer
shifts</strong> — the same figures as at the restore point. Refunds went from 50,986 to 51,986,
and all 1,000 of those came from the live Square import run afterwards, not from the renaming.
re-named, and afterwards there were still exactly <strong>50,986 refunds, 144,688 payouts and
69,291 cash-drawer shifts</strong> — the same three figures as at the restore point.
Had the fallback lookup been missing, each of these would have doubled instead.</p>
<p><strong>The fallback also has to refuse.</strong> Reading the old name is what stops
duplicates; reading <em>anyone's</em> old name is what creates them. Two clients share a Square
@@ -259,9 +258,9 @@
link it to the copy instead</pre>
</div>
<div class="measure">
<p>Run over the whole database that was <strong>9,100,314 renamed and 200,027 copied</strong>,
<p>Run over the whole database that was <strong>16,236,839 renamed and 500,438 copied</strong>,
and payments owned by two orders went from 11,469 in a 20,000-order sample to zero across
every order of the last year. The record count rose by about 200,027 — the number of copies it
every order of the last year. The record count rose by about 500,438 — the number of copies it
reported making, which is the check that it created what it meant to and nothing else.</p>
<p>One subtlety worth recording, because it bit us: the Square id has to be recovered from the
record's current owner rather than by trimming a fixed prefix. Client codes contain dashes —
@@ -333,7 +332,7 @@
(when-not (zero? amount) amount))))</pre>
</div>
<div class="measure">
<p>It works. Measured over the same ninety days it closed <strong>156 days and $4,820.19</strong>,
<p>It works. Measured over the same ninety days it closed <strong>171 days and $5,795.18</strong>,
knocked nothing out of balance, and altered no already-balanced day — the guard makes it
incapable of touching a day that traded.</p>
<p>It was removed anyway. Those days are not quiet days; they are days whose sales were never
@@ -372,9 +371,9 @@
<table>
<thead><tr><th>Stage</th><th class="n">Days out of balance</th><th class="n">Clean</th><th class="n">Total variance</th></tr></thead>
<tbody>
<tr><td>Today's calculation, ninety days re-run</td><td class="n">1,258</td><td class="n">93.34%</td><td class="n">$69,560.10</td></tr>
<tr><td>+ refunded tips</td><td class="n">971</td><td class="n">94.86%</td><td class="n">$66,414.39</td></tr>
<tr class="total"><td>+ service charges</td><td class="n good">279</td><td class="n good">98.52%</td><td class="n good">$7,790.54</td></tr>
<tr><td>Today's calculation, ninety days re-run</td><td class="n">1,451</td><td class="n">92.32%</td><td class="n">$81,023.96</td></tr>
<tr><td>+ refunded tips</td><td class="n">1,199</td><td class="n">93.66%</td><td class="n">$78,522.85</td></tr>
<tr class="total"><td>+ service charges</td><td class="n good">542</td><td class="n good">97.13%</td><td class="n good">$20,239.00</td></tr>
</tbody>
</table>
</div>
@@ -387,16 +386,18 @@
<table>
<thead><tr><th>Change</th><th class="n">Unchanged</th><th class="n">Into balance</th><th class="n">Out of balance</th><th class="n">Balanced days altered</th><th class="n">Money moved</th></tr></thead>
<tbody>
<tr><td>Refunded tips</td><td class="n">18,590</td><td class="n good">287</td><td class="n good">0</td><td class="n good">0</td><td class="n">$3,712.67</td></tr>
<tr><td>Service charges</td><td class="n">18,208</td><td class="n good">692</td><td class="n good">0</td><td class="n good">0</td><td class="n">$58,349.85</td></tr>
<tr class="total"><td>Both, end to end</td><td class="n">17,916</td><td class="n good">979</td><td class="n good">0</td><td class="n good">0</td><td class="n">$61,769.56</td></tr>
<tr><td>Refunded tips</td><td class="n">18,588</td><td class="n good">252</td><td class="n bad">6</td><td class="n bad">6</td><td class="n">$3,777.67</td></tr>
<tr><td>Service charges</td><td class="n">18,204</td><td class="n good">663</td><td class="n good">0</td><td class="n good">0</td><td class="n">$58,923.85</td></tr>
<tr class="total"><td>Both, end to end</td><td class="n">17,916</td><td class="n good">915</td><td class="n bad">6</td><td class="n bad">6</td><td class="n">$60,784.96</td></tr>
</tbody>
</table>
</div>
<div class="measure">
<p><strong>Neither fix touched a day that was already correct.</strong> Across all 18,900 client-days, no balanced day was knocked out of balance, and no balanced day had a single figure altered — 17,916 summaries came out byte-identical, and every one of the 984 that moved was already wrong. Service charges are by far the larger of the two, moving $58,349.85 against the tip fix's $3,712.67.</p>
<p><strong>Six days were knocked out of balance, and they are worth understanding rather than hiding.</strong> Across all 18,900 client-days, 17,916 summaries came out byte-identical and 978 of the 984 that moved were already wrong. The six exceptions all have one shape: the <code>Tip</code> line drops by a round amount — $10, $15, $20, $30, $30, $50 — and the day breaks by exactly that. They are all on shared-location records (NGDG, NGEZ, NGDA, NGBK).</p>
<p>That is the tip fix working, not failing. Each is a tip that was handed back: the reversal sits on this record's order, but the refund that should offset it went to the record's twin. Before the fix the day balanced by accident, because the reversal was ignored. After it, the day correctly shows that half the transaction is filed elsewhere. The honest description is that the fix converts a hidden mis-attribution into a visible one — which is the same trade the fourth problem below makes deliberately.</p>
<p>Service charges are by far the larger of the two fixes, moving $58,923.85 against the tip fix's $3,777.67.</p>
<p>That claim is stronger than a balance check, and it is the one worth insisting on: a day can stay balanced while its individual lines move, which would still be a change to the books. Every line of every summary was compared — category, debit or credit side, amount to the cent, and account — not just the day's bottom line.</p>
<p><strong>The two fixes account for the repair exactly.</strong> Adding up the untendered-tip and service-charge amounts for the 979 repaired days leaves a residue of <span class="mono">0.0000000013</span> against $61,769.56 of imbalance removed. Nothing else moved those days; there is no unexplained remainder hiding a third effect.</p>
<p><strong>The two fixes account for every day they moved, exactly.</strong> Adding up the untendered-tip and service-charge amounts across all 984 changed days leaves a residue of <span class="mono">0.0000000002</span>. Nothing else moved those days; there is no unexplained remainder hiding a third effect, and the six that broke are accounted for by the same arithmetic as the 915 that healed.</p>
</div>
</section>
@@ -465,9 +466,9 @@
<table>
<thead><tr><th>Step</th><th>Result</th></tr></thead>
<tbody>
<tr><td>Walk every order in the database</td><td class="n">19,040,785 orders</td></tr>
<tr><td>Give every order its own payment record</td><td class="n">9,100,314 re-keyed · 200,027 copied</td></tr>
<tr><td><strong>Payments owned by two orders</strong></td><td class="n good">0 <span class="dim">across every order of the last year — 5,159,787</span></td></tr>
<tr><td>Walk every order in the database, newest month first</td><td class="n">19,040,785 orders · 38 minutes</td></tr>
<tr><td>Give every order its own payment record</td><td class="n">16,236,839 re-keyed · 500,438 copied</td></tr>
<tr><td><strong>Payments owned by two orders</strong></td><td class="n good">0 <span class="dim">across every order of the last year — 5,158,470</span></td></tr>
<tr><td>Client-scope refunds, payouts and cash-drawer shifts</td><td class="n good">counts unchanged · 0 collisions</td></tr>
<tr><td>Live Square import afterwards</td><td class="n good">0 orders with duplicated payment · 0 shared payments</td></tr>
<tr><td>Ownership changes after the change</td><td class="n good">0 refunds · 0 payouts · 0 shifts</td></tr>
@@ -475,13 +476,13 @@
</table>
</div>
<div class="measure">
<p>The count checks are the ones that matter. If re-keying had gone wrong it would have created a second copy of every record rather than updating the existing one, and the totals would have doubled. They did not move. The payment-copy step is the exception and is meant to add records — it added exactly 200,027, close to the number of copies it reported making. (Close, not exact: the counter increments while the transaction is being assembled, so two copies that resolve onto one entity are counted twice. It is a good check, not a proof.)</p>
<p>The count checks are the ones that matter. If re-keying had gone wrong it would have created a second copy of every record rather than updating the existing one, and the totals would have doubled. They did not move. The payment-copy step is the exception and is meant to add records — it added 500,438, matching the number of copies it reported making. (Close, not exact: the counter increments while the transaction is being assembled, so two copies that resolve onto one entity are counted twice. It is a good check, not a proof.)</p>
<p><strong>The measurement above was taken with the duplicate client records deactivated, and that is not how this will be deployed.</strong> Leaving both records live is the intended configuration — the re-key is what separates them — but it means the deactivation that made this measurement clean will not be there. The gap that opens is narrow and specific: while any record still carries a legacy key, a second client can resolve onto it. That is why the deployment runs the migration with imports paused, and why <code>existing-id</code> now refuses to resolve a record belonging to another client.</p>
</div>
<div class="callout">
<span class="h">The whole analysis was run again from nothing, and landed in the same place</span>
<p>Everything above was rebuilt from a fresh restore of the production backup: restore, deactivate, re-key and duplicate across all nineteen million orders, re-import from Square, then two full ninety-day recomputes. At the point the two runs are comparable — before the refund-only fix below, which the earlier run did not have — the end state matched <strong>to the cent</strong>: 279 days out of balance and $7,790.54, with not one client-day differing by so much as half a penny. The recompute was then run a third time, after the database-wide re-key had finished, and produced the identical figure again. Numbers that survive being derived twice from separate restores are not artefacts of how the measurement was set up.</p>
<p>Everything above was rebuilt from a fresh restore of the production backup, three times over, each time from the backup point itself rather than from a database an earlier run had touched: restore, re-key and split across all nineteen million orders, then two full ninety-day recomputes. The runs used different preparation — one deactivated the duplicate records and ran a live Square import, this one leaves the configuration exactly as production has it — so their headline figures differ, and that difference is itself the most useful measurement in this report. What did <strong>not</strong> move is the part that should not: for the 190 clients that do not share a Square location, the residue is 119 days and $1,730.61 in both runs, with the same five restaurants accounting for it. The arithmetic fixes behave identically no matter what is done to the duplicates.</p>
</div>
<div class="callout warn">
@@ -493,37 +494,48 @@
<section>
<h2>What is still out of balance</h2>
<div class="measure">
<p>279 client-days out of 18,900, totalling <strong>$7,790.54</strong>. Most of it is not a balancing fault, and the split is the point:</p>
<p>542 client-days out of 18,900, totalling <strong>$20,239.00</strong>. Almost all of it sits
on one group of restaurants, and that is the finding:</p>
</div>
<div class="scroll">
<table>
<thead><tr><th>What the remainder actually is</th><th class="n">Days</th><th class="n">Variance</th></tr></thead>
<thead><tr><th>Where the remainder sits</th><th class="n">Days</th><th class="n">Variance</th><th class="n">Share</th></tr></thead>
<tbody>
<tr><td><strong>Sales never imported</strong> — refunds on a record with no orders at all that day</td><td class="n">156</td><td class="n">$4,988.53</td></tr>
<tr><td>Processing fee on a day with no trading — the same shape, from the payout side</td><td class="n">15</td><td class="n">$974.99</td></tr>
<tr class="total"><td>Real trading days with genuine discrepancies</td><td class="n">108</td><td class="n">$1,995.36</td></tr>
<tr><td><strong>The twenty records that share a Square location</strong></td><td class="n">423</td><td class="n">$18,508.39</td><td class="n">91%</td></tr>
<tr class="total"><td>Every other client — 190 of the 210</td><td class="n good">119</td><td class="n good">$1,730.61</td><td class="n">9%</td></tr>
</tbody>
</table>
</div>
<div class="measure">
<p><strong>Two thirds of what is left is a data-completeness problem, not an arithmetic one</strong>, and it is being left visible on purpose. Of those 156 days, 132 fall before their client had recorded a single order — they are periods with no sales in the system at all. The nine records involved hold <strong>659 refunds worth $15,225.24 dated before their own first order</strong>.</p>
<p>The 15 fee-only days are the same story from the payout side: a processing fee landing on a day with no trading. Closing them needs the payout modelled, not another rule in the summary.</p>
<p><strong>For every restaurant that is not one of the ten duplicated pairs, this reproduces
to the penny.</strong> 119 days and $1,730.61, with the same five clients accounting for
almost all of it — NG4S at $1,066.61, NGMV at $259.38, NGEB at $199.09, NGPS at $172.82,
N-30012 at $30.31, and 91 further days totalling $2.40 of till rounding. Those are the
numbers an earlier run produced on a differently prepared database, which is a stronger
check on the two arithmetic fixes than any single measurement: they behave identically
whatever is done to the duplicates.</p>
<p><strong>The $18,508.39 on the twenty shared-location records is the cost of leaving both
records live.</strong> Each restaurant now keeps two sets of books, and the history behind
them was never split: refunds claimed by whichever record imported them first, orders that
went to the other, tips reversed on one side and refunded on the other. Re-keying makes that
attribution <em>stable</em> — it stops moving — but it does not make it <em>right</em>.</p>
<p>Of that remainder, 155 days and $4,567.53 are days those records had no sales imported at
all, which is the fourth problem above and is deliberately left visible. The other 268 days
are real trading days on which the two records disagree about who owns what.</p>
</div>
<div class="scroll">
<table>
<thead><tr><th>Of the 108 real trading days</th><th class="n">Days</th><th class="n">Variance</th><th>What it is</th></tr></thead>
<tbody>
<tr><td class="mono">NGBR</td><td class="n">1</td><td class="n">$299.42</td><td rowspan="2">Square recorded $6,358.99 of tender against $6,059.57 of order totals on 2026-08-06 — the gap itself, not a summary fault</td></tr>
<tr><td class="mono">NGBK</td><td class="n">1</td><td class="n">$299.42</td></tr>
<tr><td class="mono">NGDL</td><td class="n">1</td><td class="n">$810.24</td><td>an order on one record whose payment is on the twin — the mirror image of the refund case</td></tr>
<tr><td class="mono">NGMV</td><td class="n">5</td><td class="n">$259.38</td><td>late May, undiagnosed</td></tr>
<tr><td class="mono">NGEB</td><td class="n">4</td><td class="n">$199.09</td><td>ezCater fee treatment — an open question</td></tr>
<tr class="total"><td class="dim">everyone else</td><td class="n dim">96</td><td class="n dim">$127.81</td><td class="dim">mostly till rounding — pennies a day</td></tr>
</tbody>
</table>
</div>
<div class="measure">
<p>The clusters on NGMV and NGEB are unexplained and worth a look. They are visible only because the window is ninety days; a thirty-day view does not reach them.</p>
<div class="callout warn">
<span class="h">What retiring the duplicate records would be worth</span>
<p>An earlier measurement of the same window, taken with one record of each pair deactivated
and a live Square import run afterwards, left <strong>279 days and $7,790.54</strong> instead
of 542 and $20,239.00. Most of that difference is the twenty shared records: with the
duplicates retired their share fell from $18,508.39 to $6,059.93.</p>
<p>That is not an argument that the configuration is wrong — two live records is a deliberate
choice, and the re-key is what makes it safe. It is a number to weigh: leaving both active
costs roughly <strong>$12,000 of unexplained variance across 260 client-days</strong> per
ninety days, carried on ten restaurants, until the historical attribution behind them is
redistributed. The two figures are not perfectly isolated — that earlier run also included a
live import, which backfilled data this one does not have — so treat it as the right order of
magnitude rather than an exact price.</p>
</div>
</section>
@@ -539,8 +551,8 @@
<table>
<thead><tr><th>Record type</th><th class="n">Total</th><th class="n">Client-scoped</th><th class="n">Still to rename</th><th class="n">Cannot be scoped</th></tr></thead>
<tbody>
<tr><td>Card payments</td><td class="n">17,047,142</td><td class="n good">17,047,142</td><td class="n good">0</td><td class="n good">0</td></tr>
<tr><td>Refunds</td><td class="n">51,986</td><td class="n good">51,986</td><td class="n good">0</td><td class="n good">0</td></tr>
<tr><td>Card payments</td><td class="n">17,045,933</td><td class="n good">17,045,933</td><td class="n good">0</td><td class="n good">0</td></tr>
<tr><td>Refunds</td><td class="n">50,986</td><td class="n good">50,986</td><td class="n good">0</td><td class="n good">0</td></tr>
<tr><td>Payouts</td><td class="n">144,688</td><td class="n good">144,652</td><td class="n good">0</td><td class="n">36</td></tr>
<tr><td>Cash-drawer shifts</td><td class="n">69,291</td><td class="n good">69,291</td><td class="n good">0</td><td class="n good">0</td></tr>
</tbody>
@@ -601,11 +613,12 @@
<div class="callout">
<span class="h">Re-running is safe, and that was proved at full scale</span>
<p>After the complete pass, asking the migration what it would change next returns
<strong>nothing</strong> — 17,047,142 payments examined, none to rename, none unscopable. A
<strong>nothing</strong> — 17,045,933 payments examined, none to rename, none unscopable. A
record that already carries the right name is left untouched, so the migration can be stopped,
resumed, or repeated without consequence.</p>
<p>Its speed is worth a note for whoever schedules it: the whole nineteen million orders were
walked in about <strong>thirteen minutes</strong>. An earlier attempt appeared to be
walked in about <strong>thirty-eight minutes</strong>, month by month from the current month
backwards so that stopping early leaves the recent end done. An earlier attempt appeared to be
transactor-bound and was projected at two days, which is why a previous run narrowed it to the
analysis window. That diagnosis was wrong. The bottleneck was garbage collection in the process
driving the migration — freeing held memory took an unrelated recompute from 17 client-days a
@@ -641,7 +654,7 @@
<span class="h">Two operational findings, unrelated to the summaries</span>
<p><strong>The production backup had not written a restore point since 2025-03-10</strong> — roughly seventeen months — even though data files were still uploading daily. A backup you cannot restore from is not a backup. A fresh one was taken on 2026-08-14 and is what this work used.</p>
<p><strong>The database server was sized for a toy dataset</strong>: a 2 GB cache against 27 GB of data. Worth checking what production is set to.</p>
<p><strong>Slowness here was misdiagnosed twice, in the same direction.</strong> Both a recompute crawling at 17 client-days a minute and a migration projected to take two days turned out to be garbage collection in the client process, not the database or the transactor. Freeing held memory took the recompute to 4,515 client-days a minute — a factor of 265 — and the migration finished in thirteen minutes. The lesson generalises: before concluding the transactor is the bottleneck, look at the heap of whatever is driving it.</p>
<p><strong>Slowness here was misdiagnosed twice, in the same direction.</strong> Both a recompute crawling at 17 client-days a minute and a migration projected to take two days turned out to be garbage collection in the client process, not the database or the transactor. Freeing held memory took the recompute to 4,515 client-days a minute — a factor of 265 — and the migration finished in well under an hour. The lesson generalises: before concluding the transactor is the bottleneck, look at the heap of whatever is driving it.</p>
</div>
</section>
@@ -691,7 +704,7 @@
<footer>
<span>Measured 2026-08-15 against <span class="mono">integreat-prod-restore</span>, restored fresh from backup point 209608347 — production as of 2026-08-14 22:52. Nothing in production was read or written. Branch <span class="mono">worktree-sales-summary-balance</span>.</span>
<span>A day counts as out of balance when money taken minus money earned is half a penny or more. "Material" means ten cents or more, the threshold below which the residual is till rounding. Of the 279 remaining days, 189 are material — but 171 of those are days with no sales imported, not arithmetic faults.</span>
<span>A day counts as out of balance when money taken minus money earned is half a penny or more. "Material" means ten cents or more, the threshold below which the residual is till rounding. Of the 542 remaining days, 452 are material — but 171 are days with no sales imported rather than arithmetic faults, and 423 of the 542 sit on the twenty records that share a Square location.</span>
<span>Both the baseline and the result are live captures taken straight after their own recompute, never historical reads — see the note on <code>as-of</code> above.</span>
</footer>