feat(sales): initial Parquet migration infrastructure

- Add DuckDB/S3 parquet storage layer (auto-ap.storage.parquet)
- Add sales_to_parquet migration script for historical data
- Add cleanup_sales for post-migration Datomic cleanup
- Add sales_orders_new.clj with DuckDB read layer for SSR views
- Add test scaffolding for parquet storage
- Add plan document for move-detailed-sales-to-parquet

feat(sales): redirect production and read flows to Parquet/DuckDB

- U3: Square production (upsert) now buffers to parquet via flatten-order-to-parquet!
- U3: EzCater core import-order now buffers to parquet instead of Datomic transact
- U3: EzCater XLS upload-xls now buffers to parquet instead of audit-transact
- U4: Rewrite sales_orders.clj to read from DuckDB via pq/get-sales-orders
- U5: Rewrite sales_summaries to use parquet aggregation functions
  - get-payment-items-parquet, get-discounts-parquet, get-refund-items-parquet
  - get-tax-parquet, get-tip-parquet, get-sales-parquet
- Add sum-* aggregation functions to storage/sales_summaries.clj
  - sum-discounts, sum-refunds-by-type, sum-taxes, sum-tips, sum-sales-by-category
This commit is contained in:
2026-04-25 07:43:41 -07:00
parent 297464c188
commit cc838adfac
15 changed files with 1901 additions and 290 deletions

View File

@@ -93,18 +93,14 @@
[hiccup "2.0.0-alpha2"]
;; needed for java 11
[javax.xml.bind/jaxb-api "2.4.0-b180830.0359"]
[io.forward/clojure-mail "1.0.8"]
[lambdaisland/edn-lines "1.0.10"]]
:managed-dependencies [;; explicit dependencies to get to latest versions for above
[com.fasterxml.jackson.core/jackson-core "2.12.0"]
[com.fasterxml.jackson.core/jackson-databind "2.12.0"]
[com.fasterxml.jackson.core/jackson-annotations "2.12.0"]
[com.fasterxml.jackson.dataformat/jackson-dataformat-cbor "2.12.0"]
[commons-codec "1.12"]]
:plugins [[lein-ring "0.9.7"]
;; needed for java 11
[javax.xml.bind/jaxb-api "2.4.0-b180830.0359"]
[io.forward/clojure-mail "1.0.8"]
[lambdaisland/edn-lines "1.0.10"]
[org.duckdb/duckdb_jdbc "1.1.0"]
[org.xerial/sqlite-jdbc "3.45.1.0"]
[com.fasterxml.jackson.core/jackson-core "2.12.0"]
[com.fasterxml.jackson.core/jackson-databind "2.12.0"]
[lein-cljsbuild "1.1.5"]
[lein-ancient "0.6.15"]]
:clean-targets ^{:protect false} ["resources/public/js/compiled" "target"]
@@ -144,7 +140,7 @@
[com.bhauman/rebel-readline-cljs "0.1.4" :exclusions [org.clojure/clojurescript]]
[javax.servlet/servlet-api "2.5"]]
:plugins [[lein-pdo "0.1.1"]]
:jvm-opts ["-Dconfig=config/dev.edn" "-Xms4G" "-Xmx20G" "-XX:-OmitStackTraceInFastThrow"]}
:jvm-opts ["-Dconfig=config/dev.edn" "-Xms4G" "-Xmx20G" "-XX:-OmitStackTraceInFastThrow" "-Djava.library.path=/home/noti/.local/lib"]}
:uberjar
{:java-cmd "/usr/lib/jvm/java-11-openjdk/bin/java"