it looks like there are a bbunch of orrphaned customizations for accounts, breaking indexes upsertledger - matching transaction rule might not assign an account. Other things might not assign accounts. This is an assertion that is commented out. Determine consequence of disabling Double check each job still functions in the new system Make reports just be based on running-balances Test exports Some jobs just aren't so big they need to be jobs anymore: Refreshing running balance for journal entry lines Refreshing current balance for bank accounts Closing auto invoices Running Balance Cache * Add tests for upsert-ledger * Make a new way to reset the entire cache for a client Address memory * JVM settings now and in prod Sanity checks later: * Run query Future improvements: Make reports just be based on running-balances Just use a periodic request or event instead of a job for running balance cache, and perhaps others too get rid of account-groups move to solr upsertentity Look at how address works on client save. There's agood chance that we should make saving a rel with only a temp id just resolve it to null new way to do backups Release steps: Set prod web workers to 0 Make database snapshot (run export-job) run link_sales_queries again to make sure sales queries are linked (ecs/run-task (cond-> {:capacity-provider-strategy [{:base 1 :weight 1 :capacity-provider "FARGATE_SPOT"}] :count 1 :cluster "default" :enable-ecs-managed-tags true :task-definition "XXX" :network-configuration {:aws-vpc-configuration {:subnets ["subnet-5e675761" "subnet-8519fde2" "subnet-89bab8d4"] :security-groups ["sg-004e5855310c453a3" "sg-02d167406b1082698"] :assign-public-ip AssignPublicIp/ENABLED}}} true (assoc-in [:overrides :container-overrides ] [{:name "integreat-app" :environment [{:name "args" :value (pr-str {:backup "63646188-90cd-4cec-a115-feeb7e33d54d" :starting-at "sales-order"})}]}]))) Create new database for prod-cloud (just called prod) (dc/create-database conn {:db-name "prod"}) Set this in the prod-cloud config file and prod-cloud-background-worker Restore database (ecs/run-task (cond-> {:capacity-provider-strategy [{:base 1 :weight 1 :capacity-provider "FARGATE_SPOT"}] :count 1 :cluster "default" :enable-ecs-managed-tags true :task-definition "restore_from_backup_prod_cloud:3" :network-configuration {:aws-vpc-configuration {:subnets ["subnet-5e675761" "subnet-8519fde2" "subnet-89bab8d4"] :security-groups ["sg-004e5855310c453a3" "sg-02d167406b1082698"] :assign-public-ip AssignPublicIp/ENABLED}}} true (assoc-in [:overrides :container-overrides ] [{:name "integreat-app" :environment [{:name "args" :value (pr-str {:backup "63646188-90cd-4cec-a115-feeb7e33d54d" :starting-at "sales-order"})}]}]))) Merge branch into master Rename prod-cloud to prod everywhere Release again git push deploy master regenerate all sales queries (clients.clj) (NEW) Release steps: make prod.edn configuration, and any terraform configuration, match prod-cloud as necessary run link_sales_queries again to make sure sales queries are linked X Set prod web workers to 0 X administer-system, upgrade schema X Merge branch into master X git push deploy master X (auto-ap.datomic/transact-schema auto-ap.datomic/conn) X (auto-ap.datomic/install-functions) X regenerate all sales queries (clients.clj) X (auto-ap.ledger/reset-client+account+location+date) #_(auto-ap.ledger/rebuild-running-balance-cache) X ;; sets up sales order X (doseq [[c] (dc/q '[:find ?c :where [?c :client/code]] (dc/db conn)) :let [_ (println "client " c)] batch (->> (dc/q '[:find ?so ?d ?c :in $ ?c :where [?so :sales-order/client ?c] [?so :sales-order/date ?d]] (dc/db conn) c) (map (fn [[so d c]] {:db/id so :sales-order/date d :sales-order/client c})) (partition-all 1000))] (println (count batch)) (print ".") (flush) (dc/transact conn {:tx-data batch}) ) X auser/index-solr) LATER * reset up datomic HA - needs different hostnames * Update terraform provider * make solr support graceful shutdown tini/init? Fix ledger entries: (clojure.data.csv/write-csv *out* (->> (auto-ap.datomic/pull-many (dc/as-of (dc/db conn) #inst "2023-03-31T00:00:00") '[:invoice/date {:invoice/client [:client/code] :invoice/expense-accounts [{:invoice-expense-account/account [:account/name]}] :journal-entry/_original-entity [{:journal-entry/line-items [{:journal-entry-line/account [:account/name]}]}]}] (->> bad (map first))) (map (fn [i] [(-> i :invoice/client :client/code) (-> i :invoice/date clj-time.coerce/to-date-time (auto-ap.time/unparse auto-ap.time/iso-date)) (-> i :invoice/expense-accounts first :invoice-expense-account/account :account/name) (-> i :journal-entry/_original-entity first :journal-entry/line-items (#(filter (fn [a] (not= "Accounts Payable" (:account/name (:journal-entry-line/account a)))) %)) first :journal-entry-line/account :account/name)])) )) BSG,2021-11-19,Advertising,Food Sales BSG,2021-08-26,Auto and Truck Expenses,Service or Pass Through Costs BSG,2021-12-01,Advertising,Design WGC,2021-03-01,Equipment 3,Kitchen Equipment WGC,2021-02-03,Equipment 3,Kitchen Equipment WGC,2020-12-14,Marketing Consultant,Website/ Social Media WGC,2020-12-10,Charitable Contributions,Promotional or Donation Meal Comps WGC,2020-12-15,Note Payable 3,Notes Payable - General WGC,2020-11-16,Note Payable 3,Notes Payable - General WGC,2020-11-05,Unassigned Expenses,Misc Payments WGC,2021-01-13,Marketing Consultant,Website/ Social Media WGC,2020-10-15,Note Payable 3,Notes Payable - General BSG,2021-07-20,Auto and Truck Expenses,Service or Pass Through Costs WGC,2021-02-12,Marketing Consultant,Website/ Social Media BSG,2021-12-22,Advertising,Food Sales BSG,2021-12-31,Advertising,Food Sales BSG,2021-06-15,Advertising,Design BSG,2021-12-14,Advertising,Food Sales BSG,2021-04-15,Advertising,Design WGC,2021-06-11,Marketing Consultant,Website/ Social Media BSG,2021-12-14,Food Cost,Meal Comps WGC,2021-08-05,Marketing Consultant,Website/ Social Media BSG,2021-09-30,Auto and Truck Expenses,Service or Pass Through Costs BSG,2021-08-25,Marketing,Marketing Consultant BSG,2021-12-03,Food Cost,Meal Comps BSG,2021-12-09,Food Cost,Meal Comps WGC,2021-10-08,Advertising,Website/ Social Media BSG,2021-12-09,Advertising,Food Sales WGC,2021-03-26,Equipment 3,Kitchen Equipment WGC,2021-03-26,Equipment 3,Kitchen Equipment BSG,2021-05-26,Advertising,Design WGC,2021-09-17,Advertising,Promotional or Donation Meal Comps differences: q only supports one query format in on-prem, multiple in cloud qseq takes a different format too index-pull is not chunked