notid/integreat
1
0
Fork 0
Code Issues Pull Requests 4 Actions Packages Projects Releases Wiki Activity

fix(sales-summaries): stop days falling out of balance #18

Open
notid wants to merge 9 commits from sales-summary-balancing-v2 into master
pull from: sales-summary-balancing-v2
merge into: notid:master
Conversation 2 Commits 9 Files Changed 19 +98 -77
2 changed files with 98 additions and 77 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit 4a1817711d - Show all commits

131
docs/2026-08-15-sales-summary-reconciliation.html
View File

@@ -105,27 +105,27 @@
<div class="ledger">
<div>
<span class="side-label">Today's calculation, ninety days re-run</span>
<span class="figure num">$69,560.10</span>
<span class="subfig"><span class="num">1,258</span> days out of balance · <span class="num">93.34%</span> clean</span>
<span class="figure num">$81,023.96</span>
<span class="subfig"><span class="num">1,451</span> days out of balance · <span class="num">92.32%</span> clean</span>
</div>
<div class="arrow" aria-hidden="true">→</div>
<div>
<span class="side-label">The same ninety days, fixes on</span>
<span class="figure after num">$7,790.54</span>
<span class="subfig"><span class="num">279</span> days out of balance · <span class="num">98.52%</span> clean</span>
<span class="figure after num">$20,239.00</span>
<span class="subfig"><span class="num">542</span> days out of balance · <span class="num">97.13%</span> clean</span>
</div>
</div>
<div class="stats">
<div class="stat"><span class="k num">979</span><span class="l">client-days brought into balance</span></div>
<div class="stat zero"><span class="k num">0</span><span class="l">days knocked out of balance</span></div>
<div class="stat"><span class="k num">88.8%</span><span class="l">of the variance removed</span></div>
<div class="stat"><span class="k num">915</span><span class="l">client-days brought into balance</span></div>
<div class="stat"><span class="k num">6</span><span class="l">days knocked out of balance</span></div>
<div class="stat"><span class="k num">75.0%</span><span class="l">of the variance removed</span></div>
<div class="stat zero"><span class="k num">0</span><span class="l">payments shared between two clients</span></div>
</div>
<div class="measure">
<p><strong>In one sentence:</strong> a day's sales summary should show the money taken and the money earned agreeing to the penny, and on roughly one trading day in eight it did not — because two clients were fighting over the same records, tips that had been refunded were still counted as income, and service charges customers paid were credited to nothing.</p>
<p><strong>How the two figures above were produced.</strong> Both are the real nightly job, run over the same ninety days against the same restored database, writing real summaries each time — the first pass with the fixes switched off, the second with them on. Comparing a re-run against a re-run rather than against production's stored summaries is the stricter test: production's figures are in places months stale, and crediting the fixes with repairing ordinary staleness would flatter them. On that fairer footing the fixes are worth <strong>979 days and $61,769.56</strong>, not the larger number a stale baseline would have shown.</p>
<p><strong>How the two figures above were produced.</strong> Both are the real nightly job, run over the same ninety days against the same restored database, writing real summaries each time — the first pass with the fixes switched off, the second with them on. Comparing a re-run against a re-run rather than against production's stored summaries is the stricter test: production's figures are in places months stale, and crediting the fixes with repairing ordinary staleness would flatter them. On that fairer footing the fixes are worth <strong>a net 909 days and $60,784.96</strong>, not the larger number a stale baseline would have shown. Both passes ran with the duplicate client records left active, which is how this will actually be deployed.</p>
<p><strong>Most of what is left is not a balancing fault at all</strong>, and the section on the fourth problem explains why deliberately leaving it unbalanced is the right call.</p>
</div>
</section>
@@ -165,11 +165,11 @@
<div class="problem">
<h4>4. Refunds on records whose sales were never imported <span class="tech">not fixed — deliberately</span></h4>
<div class="measure">
<p><strong>This is why the duplicated restaurants looked so much worse than everyone else.</strong> Of the days still failing after the first three fixes, <strong>155 of the 158 on duplicate records had no sales orders at all</strong> — the summary consisted of nothing but refunds and their fees, with no sales for them to reduce.</p>
<p>The obvious reading is that the refund simply settled on a closed day. It is the wrong one. Checking each of those days against the date its client first recorded <em>any</em> order shows <strong>132 of 156 fall before that client had a single order in the system</strong> — for seven of the nine records affected, every single one does. These are not quiet days. They are periods where the sales were never imported at all.</p>
<p><strong>This is why the duplicated restaurants looked so much worse than everyone else.</strong> Of the days still failing after the first three fixes, <strong>155 of the 423 on shared-location records had no sales orders at all</strong> — the summary consisted of nothing but refunds and their fees, with no sales for them to reduce.</p>
<p>The obvious reading is that the refund simply settled on a closed day. It is the wrong one. Checking each of those days against the date its client first recorded <em>any</em> order shows <strong>140 of 171 fall before that client had a single order in the system</strong> — for seven of the nine records affected, every single one does. These are not quiet days. They are periods where the sales were never imported at all.</p>
<p><strong>Where the refunds came from.</strong> Reading the database's own ownership history settles it. A $35.35 refund dated 26 February belonged to <span class="mono">NGDG</span> that same day, and was taken over by <span class="mono">NGDU</span> on 12 August. Others flip between the two records several times a day across 12–15 August. <span class="mono">NGDU</span>'s first order is 2 August; it holds 94 refunds dated before it existed as a trading record. It never made them — it inherited seven months of the other record's refunds, because the refund key carried no client and whichever import ran last took ownership. That is fault 1, seen from the other end.</p>
<p>Across the nine records, <strong>659 refunds worth $15,225.24 sit on a record dated before that record's first order.</strong> Nothing is lost and nothing is double-counted — the money is real and the surviving record has its own copy — but it is filed against a set of books that has no sales to put it against.</p>
<p><strong>Why it is deliberately left out of balance.</strong> The day can be closed in one line: book a return equal to the day's refunds whenever the client recorded no sales. It is safe by construction — no trading day could be touched — and it closes 156 of the 279 remaining days and $4,820.19. It was built, measured, and then removed, because it is the wrong thing to do. An unbalanced day is the only visible signal that a restaurant's sales are not being imported. Making the arithmetic agree would remove the alarm and leave the fire.</p>
<p>Across the nine records, <strong>660 refunds worth $15,237.02 sit on a record dated before that record's first order.</strong> Nothing is lost and nothing is double-counted — the money is real and the surviving record has its own copy — but it is filed against a set of books that has no sales to put it against.</p>
<p><strong>Why it is deliberately left out of balance.</strong> The day can be closed in one line: book a return equal to the day's refunds whenever the client recorded no sales. It is safe by construction — no trading day could be touched — and on this data it closes 171 of the 542 remaining days and $5,795.18. It was built, measured, and then removed, because it is the wrong thing to do. An unbalanced day is the only visible signal that a restaurant's sales are not being imported. Making the arithmetic agree would remove the alarm and leave the fire.</p>
<p><span class="tech">technical</span> <code>get-returns</code> sums <code>:sales-order/returns</code> over orders scanned for the date. With no orders the sum is nil and no <code>Returns</code> line is written, while <code>get-refund-items</code> still credits <code>Card Refunds</code> from the <code>sales-refund</code> records. The imbalance is the correct output for the input; the input is what is wrong. A test now pins this behaviour in place so it is not "fixed" by someone reading only the arithmetic.</p>
</div>
</div>
@@ -226,9 +226,8 @@
<p>The result is pinned as the record's id on the way in, so the write lands on the existing
row regardless of which name it currently carries. The proof this worked is a count that did
not move. Every one of the 265,965 refunds, payouts and cash-drawer shifts in the database was
re-named, and afterwards there were still exactly <strong>144,688 payouts and 69,291 cash-drawer
shifts</strong> — the same figures as at the restore point. Refunds went from 50,986 to 51,986,
and all 1,000 of those came from the live Square import run afterwards, not from the renaming.
re-named, and afterwards there were still exactly <strong>50,986 refunds, 144,688 payouts and
69,291 cash-drawer shifts</strong> — the same three figures as at the restore point.
Had the fallback lookup been missing, each of these would have doubled instead.</p>
<p><strong>The fallback also has to refuse.</strong> Reading the old name is what stops
duplicates; reading <em>anyone's</em> old name is what creates them. Two clients share a Square
@@ -259,9 +258,9 @@
link it to the copy instead</pre>
</div>
<div class="measure">
<p>Run over the whole database that was <strong>9,100,314 renamed and 200,027 copied</strong>,
<p>Run over the whole database that was <strong>16,236,839 renamed and 500,438 copied</strong>,
and payments owned by two orders went from 11,469 in a 20,000-order sample to zero across
every order of the last year. The record count rose by about 200,027 — the number of copies it
every order of the last year. The record count rose by about 500,438 — the number of copies it
reported making, which is the check that it created what it meant to and nothing else.</p>
<p>One subtlety worth recording, because it bit us: the Square id has to be recovered from the
record's current owner rather than by trimming a fixed prefix. Client codes contain dashes —
@@ -333,7 +332,7 @@
(when-not (zero? amount) amount))))</pre>
</div>
<div class="measure">
<p>It works. Measured over the same ninety days it closed <strong>156 days and $4,820.19</strong>,
<p>It works. Measured over the same ninety days it closed <strong>171 days and $5,795.18</strong>,
knocked nothing out of balance, and altered no already-balanced day — the guard makes it
incapable of touching a day that traded.</p>
<p>It was removed anyway. Those days are not quiet days; they are days whose sales were never
@@ -372,9 +371,9 @@
<table>
<thead><tr><th>Stage</th><th class="n">Days out of balance</th><th class="n">Clean</th><th class="n">Total variance</th></tr></thead>
<tbody>
<tr><td>Today's calculation, ninety days re-run</td><td class="n">1,258</td><td class="n">93.34%</td><td class="n">$69,560.10</td></tr>
<tr><td>+ refunded tips</td><td class="n">971</td><td class="n">94.86%</td><td class="n">$66,414.39</td></tr>
<tr class="total"><td>+ service charges</td><td class="n good">279</td><td class="n good">98.52%</td><td class="n good">$7,790.54</td></tr>
<tr><td>Today's calculation, ninety days re-run</td><td class="n">1,451</td><td class="n">92.32%</td><td class="n">$81,023.96</td></tr>
<tr><td>+ refunded tips</td><td class="n">1,199</td><td class="n">93.66%</td><td class="n">$78,522.85</td></tr>
<tr class="total"><td>+ service charges</td><td class="n good">542</td><td class="n good">97.13%</td><td class="n good">$20,239.00</td></tr>
</tbody>
</table>
</div>
@@ -387,16 +386,18 @@
<table>
<thead><tr><th>Change</th><th class="n">Unchanged</th><th class="n">Into balance</th><th class="n">Out of balance</th><th class="n">Balanced days altered</th><th class="n">Money moved</th></tr></thead>
<tbody>
<tr><td>Refunded tips</td><td class="n">18,590</td><td class="n good">287</td><td class="n good">0</td><td class="n good">0</td><td class="n">$3,712.67</td></tr>
<tr><td>Service charges</td><td class="n">18,208</td><td class="n good">692</td><td class="n good">0</td><td class="n good">0</td><td class="n">$58,349.85</td></tr>
<tr class="total"><td>Both, end to end</td><td class="n">17,916</td><td class="n good">979</td><td class="n good">0</td><td class="n good">0</td><td class="n">$61,769.56</td></tr>
<tr><td>Refunded tips</td><td class="n">18,588</td><td class="n good">252</td><td class="n bad">6</td><td class="n bad">6</td><td class="n">$3,777.67</td></tr>
<tr><td>Service charges</td><td class="n">18,204</td><td class="n good">663</td><td class="n good">0</td><td class="n good">0</td><td class="n">$58,923.85</td></tr>
<tr class="total"><td>Both, end to end</td><td class="n">17,916</td><td class="n good">915</td><td class="n bad">6</td><td class="n bad">6</td><td class="n">$60,784.96</td></tr>
</tbody>
</table>
</div>
<div class="measure">
<p><strong>Neither fix touched a day that was already correct.</strong> Across all 18,900 client-days, no balanced day was knocked out of balance, and no balanced day had a single figure altered — 17,916 summaries came out byte-identical, and every one of the 984 that moved was already wrong. Service charges are by far the larger of the two, moving $58,349.85 against the tip fix's $3,712.67.</p>
<p><strong>Six days were knocked out of balance, and they are worth understanding rather than hiding.</strong> Across all 18,900 client-days, 17,916 summaries came out byte-identical and 978 of the 984 that moved were already wrong. The six exceptions all have one shape: the <code>Tip</code> line drops by a round amount — $10, $15, $20, $30, $30, $50 — and the day breaks by exactly that. They are all on shared-location records (NGDG, NGEZ, NGDA, NGBK).</p>
<p>That is the tip fix working, not failing. Each is a tip that was handed back: the reversal sits on this record's order, but the refund that should offset it went to the record's twin. Before the fix the day balanced by accident, because the reversal was ignored. After it, the day correctly shows that half the transaction is filed elsewhere. The honest description is that the fix converts a hidden mis-attribution into a visible one — which is the same trade the fourth problem below makes deliberately.</p>
<p>Service charges are by far the larger of the two fixes, moving $58,923.85 against the tip fix's $3,777.67.</p>
<p>That claim is stronger than a balance check, and it is the one worth insisting on: a day can stay balanced while its individual lines move, which would still be a change to the books. Every line of every summary was compared — category, debit or credit side, amount to the cent, and account — not just the day's bottom line.</p>
<p><strong>The two fixes account for the repair exactly.</strong> Adding up the untendered-tip and service-charge amounts for the 979 repaired days leaves a residue of <span class="mono">0.0000000013</span> against $61,769.56 of imbalance removed. Nothing else moved those days; there is no unexplained remainder hiding a third effect.</p>
<p><strong>The two fixes account for every day they moved, exactly.</strong> Adding up the untendered-tip and service-charge amounts across all 984 changed days leaves a residue of <span class="mono">0.0000000002</span>. Nothing else moved those days; there is no unexplained remainder hiding a third effect, and the six that broke are accounted for by the same arithmetic as the 915 that healed.</p>
</div>
</section>
@@ -465,9 +466,9 @@
<table>
<thead><tr><th>Step</th><th>Result</th></tr></thead>
<tbody>
<tr><td>Walk every order in the database</td><td class="n">19,040,785 orders</td></tr>
<tr><td>Give every order its own payment record</td><td class="n">9,100,314 re-keyed · 200,027 copied</td></tr>
<tr><td><strong>Payments owned by two orders</strong></td><td class="n good">0 <span class="dim">across every order of the last year — 5,159,787</span></td></tr>
<tr><td>Walk every order in the database, newest month first</td><td class="n">19,040,785 orders · 38 minutes</td></tr>
<tr><td>Give every order its own payment record</td><td class="n">16,236,839 re-keyed · 500,438 copied</td></tr>
<tr><td><strong>Payments owned by two orders</strong></td><td class="n good">0 <span class="dim">across every order of the last year — 5,158,470</span></td></tr>
<tr><td>Client-scope refunds, payouts and cash-drawer shifts</td><td class="n good">counts unchanged · 0 collisions</td></tr>
<tr><td>Live Square import afterwards</td><td class="n good">0 orders with duplicated payment · 0 shared payments</td></tr>
<tr><td>Ownership changes after the change</td><td class="n good">0 refunds · 0 payouts · 0 shifts</td></tr>
@@ -475,13 +476,13 @@
</table>
</div>
<div class="measure">
<p>The count checks are the ones that matter. If re-keying had gone wrong it would have created a second copy of every record rather than updating the existing one, and the totals would have doubled. They did not move. The payment-copy step is the exception and is meant to add records — it added exactly 200,027, close to the number of copies it reported making. (Close, not exact: the counter increments while the transaction is being assembled, so two copies that resolve onto one entity are counted twice. It is a good check, not a proof.)</p>
<p>The count checks are the ones that matter. If re-keying had gone wrong it would have created a second copy of every record rather than updating the existing one, and the totals would have doubled. They did not move. The payment-copy step is the exception and is meant to add records — it added 500,438, matching the number of copies it reported making. (Close, not exact: the counter increments while the transaction is being assembled, so two copies that resolve onto one entity are counted twice. It is a good check, not a proof.)</p>
<p><strong>The measurement above was taken with the duplicate client records deactivated, and that is not how this will be deployed.</strong> Leaving both records live is the intended configuration — the re-key is what separates them — but it means the deactivation that made this measurement clean will not be there. The gap that opens is narrow and specific: while any record still carries a legacy key, a second client can resolve onto it. That is why the deployment runs the migration with imports paused, and why <code>existing-id</code> now refuses to resolve a record belonging to another client.</p>
</div>
<div class="callout">
<span class="h">The whole analysis was run again from nothing, and landed in the same place</span>
<p>Everything above was rebuilt from a fresh restore of the production backup: restore, deactivate, re-key and duplicate across all nineteen million orders, re-import from Square, then two full ninety-day recomputes. At the point the two runs are comparable — before the refund-only fix below, which the earlier run did not have — the end state matched <strong>to the cent</strong>: 279 days out of balance and $7,790.54, with not one client-day differing by so much as half a penny. The recompute was then run a third time, after the database-wide re-key had finished, and produced the identical figure again. Numbers that survive being derived twice from separate restores are not artefacts of how the measurement was set up.</p>
<p>Everything above was rebuilt from a fresh restore of the production backup, three times over, each time from the backup point itself rather than from a database an earlier run had touched: restore, re-key and split across all nineteen million orders, then two full ninety-day recomputes. The runs used different preparation — one deactivated the duplicate records and ran a live Square import, this one leaves the configuration exactly as production has it — so their headline figures differ, and that difference is itself the most useful measurement in this report. What did <strong>not</strong> move is the part that should not: for the 190 clients that do not share a Square location, the residue is 119 days and $1,730.61 in both runs, with the same five restaurants accounting for it. The arithmetic fixes behave identically no matter what is done to the duplicates.</p>
</div>
<div class="callout warn">
@@ -493,37 +494,48 @@
<section>
<h2>What is still out of balance</h2>
<div class="measure">
<p>279 client-days out of 18,900, totalling <strong>$7,790.54</strong>. Most of it is not a balancing fault, and the split is the point:</p>
<p>542 client-days out of 18,900, totalling <strong>$20,239.00</strong>. Almost all of it sits
on one group of restaurants, and that is the finding:</p>
</div>
<div class="scroll">
<table>
<thead><tr><th>What the remainder actually is</th><th class="n">Days</th><th class="n">Variance</th></tr></thead>
<thead><tr><th>Where the remainder sits</th><th class="n">Days</th><th class="n">Variance</th><th class="n">Share</th></tr></thead>
<tbody>
<tr><td><strong>Sales never imported</strong> — refunds on a record with no orders at all that day</td><td class="n">156</td><td class="n">$4,988.53</td></tr>
<tr><td>Processing fee on a day with no trading — the same shape, from the payout side</td><td class="n">15</td><td class="n">$974.99</td></tr>
<tr class="total"><td>Real trading days with genuine discrepancies</td><td class="n">108</td><td class="n">$1,995.36</td></tr>
<tr><td><strong>The twenty records that share a Square location</strong></td><td class="n">423</td><td class="n">$18,508.39</td><td class="n">91%</td></tr>
<tr class="total"><td>Every other client — 190 of the 210</td><td class="n good">119</td><td class="n good">$1,730.61</td><td class="n">9%</td></tr>
</tbody>
</table>
</div>
<div class="measure">
<p><strong>Two thirds of what is left is a data-completeness problem, not an arithmetic one</strong>, and it is being left visible on purpose. Of those 156 days, 132 fall before their client had recorded a single order — they are periods with no sales in the system at all. The nine records involved hold <strong>659 refunds worth $15,225.24 dated before their own first order</strong>.</p>
<p>The 15 fee-only days are the same story from the payout side: a processing fee landing on a day with no trading. Closing them needs the payout modelled, not another rule in the summary.</p>
<p><strong>For every restaurant that is not one of the ten duplicated pairs, this reproduces
to the penny.</strong> 119 days and $1,730.61, with the same five clients accounting for
almost all of it — NG4S at $1,066.61, NGMV at $259.38, NGEB at $199.09, NGPS at $172.82,
N-30012 at $30.31, and 91 further days totalling $2.40 of till rounding. Those are the
numbers an earlier run produced on a differently prepared database, which is a stronger
check on the two arithmetic fixes than any single measurement: they behave identically
whatever is done to the duplicates.</p>
<p><strong>The $18,508.39 on the twenty shared-location records is the cost of leaving both
records live.</strong> Each restaurant now keeps two sets of books, and the history behind
them was never split: refunds claimed by whichever record imported them first, orders that
went to the other, tips reversed on one side and refunded on the other. Re-keying makes that
attribution <em>stable</em> — it stops moving — but it does not make it <em>right</em>.</p>
<p>Of that remainder, 155 days and $4,567.53 are days those records had no sales imported at
all, which is the fourth problem above and is deliberately left visible. The other 268 days
are real trading days on which the two records disagree about who owns what.</p>
</div>
<div class="scroll">
<table>
<thead><tr><th>Of the 108 real trading days</th><th class="n">Days</th><th class="n">Variance</th><th>What it is</th></tr></thead>
<tbody>
<tr><td class="mono">NGBR</td><td class="n">1</td><td class="n">$299.42</td><td rowspan="2">Square recorded $6,358.99 of tender against $6,059.57 of order totals on 2026-08-06 — the gap itself, not a summary fault</td></tr>
<tr><td class="mono">NGBK</td><td class="n">1</td><td class="n">$299.42</td></tr>
<tr><td class="mono">NGDL</td><td class="n">1</td><td class="n">$810.24</td><td>an order on one record whose payment is on the twin — the mirror image of the refund case</td></tr>
<tr><td class="mono">NGMV</td><td class="n">5</td><td class="n">$259.38</td><td>late May, undiagnosed</td></tr>
<tr><td class="mono">NGEB</td><td class="n">4</td><td class="n">$199.09</td><td>ezCater fee treatment — an open question</td></tr>
<tr class="total"><td class="dim">everyone else</td><td class="n dim">96</td><td class="n dim">$127.81</td><td class="dim">mostly till rounding — pennies a day</td></tr>
</tbody>
</table>
</div>
<div class="measure">
<p>The clusters on NGMV and NGEB are unexplained and worth a look. They are visible only because the window is ninety days; a thirty-day view does not reach them.</p>
<div class="callout warn">
<span class="h">What retiring the duplicate records would be worth</span>
<p>An earlier measurement of the same window, taken with one record of each pair deactivated
and a live Square import run afterwards, left <strong>279 days and $7,790.54</strong> instead
of 542 and $20,239.00. Most of that difference is the twenty shared records: with the
duplicates retired their share fell from $18,508.39 to $6,059.93.</p>
<p>That is not an argument that the configuration is wrong — two live records is a deliberate
choice, and the re-key is what makes it safe. It is a number to weigh: leaving both active
costs roughly <strong>$12,000 of unexplained variance across 260 client-days</strong> per
ninety days, carried on ten restaurants, until the historical attribution behind them is
redistributed. The two figures are not perfectly isolated — that earlier run also included a
live import, which backfilled data this one does not have — so treat it as the right order of
magnitude rather than an exact price.</p>
</div>
</section>
@@ -539,8 +551,8 @@
<table>
<thead><tr><th>Record type</th><th class="n">Total</th><th class="n">Client-scoped</th><th class="n">Still to rename</th><th class="n">Cannot be scoped</th></tr></thead>
<tbody>
<tr><td>Card payments</td><td class="n">17,047,142</td><td class="n good">17,047,142</td><td class="n good">0</td><td class="n good">0</td></tr>
<tr><td>Refunds</td><td class="n">51,986</td><td class="n good">51,986</td><td class="n good">0</td><td class="n good">0</td></tr>
<tr><td>Card payments</td><td class="n">17,045,933</td><td class="n good">17,045,933</td><td class="n good">0</td><td class="n good">0</td></tr>
<tr><td>Refunds</td><td class="n">50,986</td><td class="n good">50,986</td><td class="n good">0</td><td class="n good">0</td></tr>
<tr><td>Payouts</td><td class="n">144,688</td><td class="n good">144,652</td><td class="n good">0</td><td class="n">36</td></tr>
<tr><td>Cash-drawer shifts</td><td class="n">69,291</td><td class="n good">69,291</td><td class="n good">0</td><td class="n good">0</td></tr>
</tbody>
@@ -601,11 +613,12 @@
<div class="callout">
<span class="h">Re-running is safe, and that was proved at full scale</span>
<p>After the complete pass, asking the migration what it would change next returns
<strong>nothing</strong> — 17,047,142 payments examined, none to rename, none unscopable. A
<strong>nothing</strong> — 17,045,933 payments examined, none to rename, none unscopable. A
record that already carries the right name is left untouched, so the migration can be stopped,
resumed, or repeated without consequence.</p>
<p>Its speed is worth a note for whoever schedules it: the whole nineteen million orders were
walked in about <strong>thirteen minutes</strong>. An earlier attempt appeared to be
walked in about <strong>thirty-eight minutes</strong>, month by month from the current month
backwards so that stopping early leaves the recent end done. An earlier attempt appeared to be
transactor-bound and was projected at two days, which is why a previous run narrowed it to the
analysis window. That diagnosis was wrong. The bottleneck was garbage collection in the process
driving the migration — freeing held memory took an unrelated recompute from 17 client-days a
@@ -641,7 +654,7 @@
<span class="h">Two operational findings, unrelated to the summaries</span>
<p><strong>The production backup had not written a restore point since 2025-03-10</strong> — roughly seventeen months — even though data files were still uploading daily. A backup you cannot restore from is not a backup. A fresh one was taken on 2026-08-14 and is what this work used.</p>
<p><strong>The database server was sized for a toy dataset</strong>: a 2 GB cache against 27 GB of data. Worth checking what production is set to.</p>
<p><strong>Slowness here was misdiagnosed twice, in the same direction.</strong> Both a recompute crawling at 17 client-days a minute and a migration projected to take two days turned out to be garbage collection in the client process, not the database or the transactor. Freeing held memory took the recompute to 4,515 client-days a minute — a factor of 265 — and the migration finished in thirteen minutes. The lesson generalises: before concluding the transactor is the bottleneck, look at the heap of whatever is driving it.</p>
<p><strong>Slowness here was misdiagnosed twice, in the same direction.</strong> Both a recompute crawling at 17 client-days a minute and a migration projected to take two days turned out to be garbage collection in the client process, not the database or the transactor. Freeing held memory took the recompute to 4,515 client-days a minute — a factor of 265 — and the migration finished in well under an hour. The lesson generalises: before concluding the transactor is the bottleneck, look at the heap of whatever is driving it.</p>
</div>
</section>
@@ -691,7 +704,7 @@
<footer>
<span>Measured 2026-08-15 against <span class="mono">integreat-prod-restore</span>, restored fresh from backup point 209608347 — production as of 2026-08-14 22:52. Nothing in production was read or written. Branch <span class="mono">worktree-sales-summary-balance</span>.</span>
<span>A day counts as out of balance when money taken minus money earned is half a penny or more. "Material" means ten cents or more, the threshold below which the residual is till rounding. Of the 279 remaining days, 189 are material — but 171 of those are days with no sales imported, not arithmetic faults.</span>
<span>A day counts as out of balance when money taken minus money earned is half a penny or more. "Material" means ten cents or more, the threshold below which the residual is till rounding. Of the 542 remaining days, 452 are material — but 171 are days with no sales imported rather than arithmetic faults, and 423 of the 542 sit on the twenty records that share a Square location.</span>
<span>Both the baseline and the result are live captures taken straight after their own recompute, never historical reads — see the note on <code>as-of</code> above.</span>
</footer>

44
docs/2026-08-15-sales-summary-rollout-plan.md
View File

@@ -5,12 +5,17 @@ begins. The one behaviour change that alters a client's books is behind a per-cl
that is **off by default**, so merging and deploying this branch changes nothing on its own.
Measured on a restored copy of production (backup point `209608347`), 210 clients over
2026-05-10 → 2026-08-07: **1,258 client-days out of balance / $69,560.10 → 279 days / $7,790.54**,
with zero days knocked out of balance and zero already-balanced days altered.
2026-05-10 → 2026-08-07, with the duplicate client records left active exactly as they will be in
production: **1,451 client-days out of balance / $81,023.96 → 542 days / $20,239.00**. 915 days
came into balance and 6 broke — all six being tips reversed on one record whose refund sits on its
twin, which the fix makes visible rather than causes.
Of the 279 days left, **171 are not balancing faults** — they are days where a client's sales were
never imported while its refunds were. Step 10 is about those, and it is the most important
thing in this document.
**Where the remaining $20,239.00 sits is the thing to read.** $18,508.39 of it — 91% — is on the
twenty records that share a Square location. For the other 190 clients the residue is 119 days and
$1,730.61, which reproduces an earlier run on a differently prepared database to the penny.
Of the total, 171 days are not balancing faults at all — they are days where a client's sales were
never imported while its refunds were. Step 10 is about those.
---
@@ -20,7 +25,7 @@ thing in this document.
|---|---|
| Flag introduced | `summary-service-charges` — off by default |
| Migration to run once | `auto-ap.jobs.rekey-square-external-ids/migrate-all!` |
| Expected migration runtime | ~13 minutes for 19M orders on a warm cache |
| Expected migration runtime | ~38 minutes for 19M orders on a warm cache |
| Nothing here touches | invoices, payments, the ledger, or any client without the flag set |
**Client configuration is left exactly as it is.** Ten Square locations are configured against two
@@ -64,7 +69,8 @@ that is the one code path that can reach across client records. Running the migr
paused means no client is resolving keys while the keys are being rewritten, so the window closes
entirely rather than merely narrowing.
The migration itself takes about **13 minutes** for all 19M orders, so the pause is short.
The migration itself takes about **38 minutes** for all 19M orders, so the pause is short — and
if you need it shorter, see step 5: you can resume imports before it finishes.
---
@@ -104,9 +110,9 @@ Run it immediately after the deploy, while imports are still paused.
`migrate-all!` runs this same check itself, on every attribute including charges, and throws
rather than transacting if it finds one. Running it by hand first just means finding out before
the 13-minute walk rather than partway through it.
the 38-minute walk rather than partway through it.
Runs in about thirteen minutes over 19M orders. It is **idempotent and resumable** — a record that
Runs in about thirty-eight minutes over 19M orders. It is **idempotent and resumable** — a record that
already carries the right name is skipped, so it can be stopped and re-run without consequence.
**It is also ordered so that stopping early is survivable.** Refunds, payouts and cash-drawer
@@ -140,7 +146,7 @@ attributes:
```clojure
(dissoc (rk/plan (d/db conn) :charge/external-id rk/charge-prefix) :new-keys)
;; => {:total 17047142 :to-migrate 0 :already-scoped 17047142 :unscopable 0}
;; => {:total 17045933 :to-migrate 0 :already-scoped 17045933 :unscopable 0}
```
Read `:to-migrate 0` **and** `:unscopable 0`. This is the authoritative signal, and it covers all
@@ -163,7 +169,7 @@ order in the last year, with no sampling:
;; => 0
```
On the restored copy that is 5,159,787 orders — 27% of the table — via the
On the restored copy that is 5,158,470 orders — 27% of the table — via the
`:sales-order/client+date` index. A year is chosen deliberately: `remove-voided-orders` only ever
deletes orders Square reports as voided, which are recent, so that is where the destructive risk
lives. Completeness across all of history is check (a)'s job, not this one.
@@ -280,16 +286,16 @@ people doing the right thing. That is the reason not to let this drift.
**The most important item in this document, and the only one that is not just execution.**
156 of the 279 remaining days are a record carrying refunds on a day it recorded no sales at all.
132 of those fall *before that client's first ever order*. Nine records are affected, holding
**659 refunds worth $15,225.24 dated before their own first order**:
171 of the 542 remaining days are a record carrying refunds on a day it recorded no sales at all.
140 of those fall *before that client's first ever order*. Nine records are affected, holding
**660 refunds worth $15,237.02 dated before their own first order**:
| Client | First order | Refunds before it | Value |
|---|---|---:|---:|
| NGDL | 2026-07-29 | 96 | $2,751.50 |
| NGWN | 2026-07-30 | 95 | $2,218.61 |
| NGDU | 2026-08-02 | 94 | $2,188.44 |
| NG4S | 2026-05-29 | 78 | $2,168.30 |
| NG4S | 2026-05-29 | 79 | $2,180.08 |
| NGPS | 2026-05-26 | 81 | $2,167.60 |
| NGCT | 2026-08-01 | 98 | $1,789.49 |
| NGVC | 2026-08-02 | 95 | $1,594.17 |
@@ -329,7 +335,7 @@ all**, so a blanket delete would destroy the only copy of some records.
```
**Until this is resolved those days stay out of balance, on purpose.** A summary change to close
them was written and measured — it works, closes 156 days and $4,820.19, and breaks nothing — and it
them was written and measured — it works, closes 171 days and $5,795.18, and breaks nothing — and it
was removed, because an unbalanced day is the only visible signal that a restaurant's sales are not
being imported. A test asserts the day stays unbalanced so nobody closes it without reading this.
@@ -337,11 +343,13 @@ being imported. A test asserts the day stays unbalanced so nobody closes it with
## What this will not fix
The 108 real trading days still out of balance, $1,995.36 — the NGBR/NGBK tender gap ($6,358.99 of
The 371 real trading days still out of balance, $14,443.82 — of which $13,940.86 is on the twenty
shared-location records, where the two books disagree about who owns what. For every other client
it is 103 days and $502.96. Named causes: the NGBR/NGBK tender gap ($6,358.99 of
tender against $6,059.57 of order totals on one day), the ezCater fee question, and unexplained
clusters on NGMV and NGEB.
Plus 15 days / $974.99 where a processing fee lands on a day with no trading — the same shape as
Plus 16 days / $1,227.65 where a processing fee lands on a day with no trading — the same shape as
step 10 but from the payout side, so it needs the payout modelled rather than a rule in the summary.
---