Commit Graph

91 Commits

Author SHA1 Message Date
f8ef7918ef fix(sales-summaries): stop days falling out of balance
Four faults were leaving restaurant days out of balance — one in the
data, three in the arithmetic. Measured over ninety days on a restored
copy of production (210 clients, 18,900 client-days): 1,258 days out of
balance and $69,560.10 becomes 123 days and $2,970.35, of which only 33
are above ten cents.

1,135 days repaired, none knocked out of balance, and not one
already-balanced day altered — verified line by line (category, side,
amount to the cent, account), not just on each day's bottom line.

THE DATA FAULT

Ten Square locations were configured against two client records each.
Sales orders scoped their identifier by client; refunds, card payments,
payouts and cash-drawer shifts used the bare Square id. Those attributes
are :db.unique/identity, so both clients' imports resolved to a single
entity and the last writer won — 3,387 refunds, 4,069 payouts and 2,628
cash-drawer shifts changed hands over time, across 19 client pairs of
which only 10 are visible in today's configuration.

Worse, one payment could belong to two orders. :sales-order/charges is
:db/isComponent, so removing a voided order cascaded into payments the
other client still needed.

Fixes: client-scope the four key schemes; look the record up under both
schemes so the change deploys before the migration finishes; and a
migration that gives every order its own payment. Run over the whole
database that is 19,040,785 orders walked, 9,100,314 payments re-keyed
and 200,027 copied, ending with 17,047,142 payments scoped, none left to
rename, none unscopable, and no payment owned by more than one order.
Idempotent and resumable; about thirteen minutes.

THE ARITHMETIC FAULTS

- Refunded tips stayed on the books. get-tip summed tips by joining
  through :sales-order/charges, so a return-only order — no tender to
  join through — contributed nothing while its reversal sat unread on
  :sales-order/tip. Additive, not substitutive: where an order does have
  a tender the tender is the correct source.

- Service charges were collected but never earned. Nothing read
  :sales-order/service-charge. Now credited for Square orders only, both
  signs, behind summary-service-charges.

- A refund on a day with no sales had nothing to offset it. Refunds are
  credited on the day the money goes back; the return that offsets them
  is read from that day's orders. get-returns now falls back to the day's
  refunded total, but only where the client recorded no sales orders at
  all — with no orders there is no order-derived return to double-count
  and no trading day can be moved. Behind summary-refund-only-returns.

Both flags are off by default, so deploying this changes nothing until a
client is opted in. docs/2026-08-15-sales-summary-rollout-plan.md has the
steps.

SUPPORTING

- Install schema attributes before the tuples that compose them. A tuple
  in schema.edn is built from an attribute in cloud-migration-schema.edn,
  so every test fixture died in setup — very likely why sales summaries
  had no tests before this.
- Log each day's imbalance and its suspect lines.
- Bound the dirty-summary scan to one client: 1,321 ms to 5.6 ms.
- compare-sales-summaries lives in test/clj as auto-ap.tools.* — it is a
  verification harness, not part of the running application. Its
  docstring now warns that d/as-of cannot be used to compare summary
  amounts: :ledger-mapped/amount, ledger-side and account are
  :db/noHistory, so a recomputed summary reads back with its amounts
  absent and looks like a legitimate balanced day.

28 tests, 65 assertions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-15 19:19:04 -07:00
cc31d8849b Feat/Complete Sales Summaries (#5)
## Summary

Completes the automatic sales summary pipeline end-to-end: the `sales-summaries-v2` job now calculates aggregate totals, preserves manual adjustments, and automatically posts balanced journal entries to the ledger.

## What Changed

**New Datomic transaction function** (`upsert-sales-summary-ledger`)
- Transforms detailed `sales-summary-item`s into aggregated `journal-entry` lines grouped by account and ledger side
- Handles the full upsert: posts a new journal entry for summaries with mapped accounts, or retracts the orphaned entry if items no longer qualify

**Enhanced `sales-summaries-v2` job**
- Calculates and stores 13 aggregate total attributes (card/cash/food-app/gift-card payments, refunds, fees, discounts, tax, tip, returns, unknown, net)
- Preserves manual items (`manual? true`) during recalculation — only auto-calculated items are replaced

**Ledger reconciliation**
- `reconcile-ledger` now queries for sales summaries missing journal entries and repairs them via `:upsert-sales-summary-ledger`, alongside existing invoice and transaction repairs

**Schema**
- Added 13 `total-*` attributes on `sales-summary` (all `db.type/double`, no history)
- Registered the new transaction function in `tx.clj` and `datomic.clj`

**Admin UI cleanup**
- Resolved "clientize" and HTMX `client-id` TODOs in the sales summaries admin page
- `new-summary-item` now correctly passes `client-id` via `hx-vals`
- Removed stale TODO comments and placeholder code

## Files Changed (8)

| File | Purpose |
|------|---------|
| `iol_ion/.../upsert_sales_summary_ledger.clj` | New Datomic tx function |
| `iol_ion/.../tx.clj` | Register new tx function |
| `resources/schema.edn` | 13 new `total-*` attributes |
| `src/.../datomic.clj` | Load new tx namespace |
| `src/.../jobs/sales_summaries.clj` | Aggregate totals + manual item preservation |
| `src/.../ledger.clj` | Sales summary repair in `reconcile-ledger` |
| `src/.../ssr/admin/sales_summaries.clj` | UI TODO cleanup |
| `docs/plans/...plan.md` | Implementation plan document |

Co-authored-by: Bryce <bryce@integreatconsult.com>
Reviewed-on: #5
Co-authored-by: Bryce <bryce@brycecovertoperations.com>
Co-committed-by: Bryce <bryce@brycecovertoperations.com>
2026-05-16 00:16:44 -07:00
85c0e21daf Fixes minor issue with sorting 2024-09-25 23:08:22 -07:00
cf7e7619dc improves sorting on ledger. 2024-09-25 22:48:38 -07:00
db9e0dc459 makes solr ec2 based and adds resiliency to running balances. 2024-09-25 22:01:12 -07:00
Bryce
6031f15246 Adds totals 2024-04-11 13:18:45 -07:00
Bryce
65be50cf9f simplify sorting 2024-04-10 14:48:47 -07:00
930b900849 Makes logging unified 2023-10-30 12:35:18 -07:00
f997c41abd migrates accounts 2023-10-14 23:21:22 -07:00
d5565f7cf8 Revisions on grid page to make parameters easier to grok 2023-09-25 16:26:42 -07:00
45113c3144 adds better logging for queries. 2023-09-06 06:35:28 -07:00
5952c58f5a Tweak for publisher errors 2023-09-06 00:17:31 -07:00
a56d3b0b84 Should fix most of the authentication issues 2023-09-05 23:08:22 -07:00
Bryce
bc57978ceb makes it easy for bryce to trigger reload for all yodlee 2023-07-28 13:59:15 -07:00
Bryce
d2cb0763d7 Merge branch 'master' into try-tailwind 2023-06-01 14:39:10 -07:00
Bryce
a07b29a8b3 minor cleanup 2023-06-01 14:39:03 -07:00
Bryce
721bbaea53 All of my company works spame way 2023-05-30 14:36:34 -07:00
Bryce
a681b61115 Merge branch 'master' into try-tailwind
merged
2023-05-26 16:59:10 -07:00
Bryce
d6b3048ff0 Makes imports more resilient 2023-05-24 08:45:26 -07:00
Bryce
33cae26cc1 other pages 2023-05-18 22:14:23 -07:00
2845f21227 Makes cloud functions installable remotely 2023-05-02 20:50:27 -07:00
611af7f9bb making tests work again. 2023-05-01 09:42:06 -07:00
b2ad7790cf Migrates back to datomic on-prem 2023-04-29 07:12:51 -07:00
d924146f0e (cloud) implements plus for check printihng 2023-04-06 16:19:28 -07:00
2a0f736af7 makes POS page much less likely to break everything 2023-04-05 22:50:38 -07:00
c8ada656c8 starts logging in the event of an issue. 2023-03-30 11:11:17 -07:00
9d355a41b5 working on restore. 2023-03-29 06:43:24 -07:00
e6a2626312 print checks tests. 2023-03-25 15:34:26 -07:00
7562213e97 merged. 2023-03-25 15:30:46 -07:00
5e9f480790 Added check printing test 2023-03-25 15:27:43 -07:00
eb7394b42b account tests. 2023-03-24 15:34:43 -07:00
4f94602716 fixes issues with a neverending search. Also fixes client saving to match normal test. 2023-03-24 13:01:33 -07:00
7fd3528d0f (cloud) big performance fixes 2023-03-24 11:02:17 -07:00
e810612fbb (cloud) makes ledger running balances fast and smooth 2023-03-24 11:02:17 -07:00
bbb11bbf46 (cloud) fixes missing namespace 2023-03-24 11:02:17 -07:00
0af50758bd (cloud) Added running balance cache for ledger 2023-03-24 11:02:17 -07:00
ab9cb15e24 moved tx functions. 2023-03-24 11:01:09 -07:00
9260834135 scrubbed all reference to datomic.api 2023-03-24 10:59:06 -07:00
78519663ac Getting closer to datomic cloud 2023-03-24 10:57:50 -07:00
65ecee7014 making cloud runnable again. 2023-03-24 10:56:35 -07:00
d02fba2b44 Makes integreat run on datomic cloud 2023-03-24 10:56:28 -07:00
437cbdcada removing forced sleep for performance improvement. 2023-03-14 09:12:26 -07:00
63c6859d64 trying to build in a little buffer. 2022-09-26 09:12:26 -07:00
84f7e734f0 added suggestions 2022-07-26 07:01:18 -07:00
96c80853ef fixed warnings. 2022-07-26 05:56:41 -07:00
6ae5e624d8 Adds default filtering and sorting 2022-07-11 08:13:15 -07:00
Bryce Covert
f0b31fe6d1 Adds transaction->autopay linking 2021-01-05 17:57:36 -08:00
Bryce Covert
a38894a10a Adds validation for external ledger import 2020-09-29 21:19:07 -07:00
Bryce Covert
4320c129ce updating database url. 2020-09-14 11:09:59 -07:00
Bryce Covert
2f79eee111 updates for datomic 2020-09-14 08:33:48 -07:00