it looks like there are a bbunch of orrphaned customizations for accounts, breaking indexes
upsertledger - matching transaction rule might not assign an account. Other things might not assign accounts. This is an assertion that is commented out.  Determine consequence of disabling
Double check each job still functions in the new system
Make reports just be based on running-balances
Test exports

Some jobs just aren't so big they need to be jobs anymore:
Refreshing running balance for journal entry lines
Refreshing current balance for bank accounts
Closing auto invoices

Running Balance Cache
* Add tests for upsert-ledger
* Make a new way to reset the entire cache for a client

Address memory
* JVM settings now and in prod


Sanity checks later:
* Run query 


Future improvements:
Make reports just be based on running-balances
Just use a periodic request or event instead of a job for running balance cache, and perhaps others too
get rid of account-groups
move to solr
upsertentity Look at how address works on client save. There's agood chance that we should make saving a rel with only a temp id just resolve it to null
new way to do backups



Release steps:
Set prod web workers to 0

Make database snapshot (run export-job)
run link_sales_queries again to make sure sales queries are linked
(ecs/run-task (cond-> {:capacity-provider-strategy [{:base 1 :weight 1 :capacity-provider "FARGATE_SPOT"}]
                         :count                      1
                         :cluster                    "default"
                         :enable-ecs-managed-tags    true
                         :task-definition            "XXX"
                         :network-configuration      {:aws-vpc-configuration {:subnets          ["subnet-5e675761" "subnet-8519fde2" "subnet-89bab8d4"]
                                                                              :security-groups  ["sg-004e5855310c453a3" "sg-02d167406b1082698"]
                                                                              :assign-public-ip AssignPublicIp/ENABLED}}}
                  true (assoc-in [:overrides :container-overrides ] [{:name "integreat-app" :environment [{:name "args" :value (pr-str {:backup "63646188-90cd-4cec-a115-feeb7e33d54d"
                                                                                                                                        :starting-at "sales-order"})}]}])))

Create new database for prod-cloud (just called prod)
(dc/create-database conn {:db-name "prod"})

Set this in the prod-cloud config file and prod-cloud-background-worker

Restore database
(ecs/run-task (cond-> {:capacity-provider-strategy [{:base 1 :weight 1 :capacity-provider "FARGATE_SPOT"}]
                         :count                      1
                         :cluster                    "default"
                         :enable-ecs-managed-tags    true
                         :task-definition            "restore_from_backup_prod_cloud:3"
                         :network-configuration      {:aws-vpc-configuration {:subnets          ["subnet-5e675761" "subnet-8519fde2" "subnet-89bab8d4"]
                                                                              :security-groups  ["sg-004e5855310c453a3" "sg-02d167406b1082698"]
                                                                              :assign-public-ip AssignPublicIp/ENABLED}}}
                  true (assoc-in [:overrides :container-overrides ] [{:name "integreat-app" :environment [{:name "args" :value (pr-str {:backup "63646188-90cd-4cec-a115-feeb7e33d54d"
                                                                                                                                        :starting-at "sales-order"})}]}])))

Merge branch into master
Rename prod-cloud to prod everywhere
Release again
git push deploy master
regenerate all sales queries (clients.clj)

(NEW) Release steps:
make prod.edn configuration, and any terraform configuration, match prod-cloud as necessary
run link_sales_queries again to make sure sales queries are linked
X Set prod web workers to 0
X administer-system, upgrade schema
X Merge branch into master
X git push deploy master
X (auto-ap.datomic/transact-schema auto-ap.datomic/conn)
X (auto-ap.datomic/install-functions)
X regenerate all sales queries (clients.clj)
X (auto-ap.ledger/reset-client+account+location+date)
#_(auto-ap.ledger/rebuild-running-balance-cache)
X ;; sets up sales order 
X (doseq [[c]   (dc/q '[:find ?c :where [?c :client/code]] (dc/db conn))
          :let  [_ (println "client " c)]
          batch (->> (dc/q '[:find ?so ?d ?c
                             :in $ ?c
                             :where
                             [?so :sales-order/client ?c]
                             [?so :sales-order/date ?d]]
                           (dc/db conn)
                           c)
                     (map (fn [[so d c]]
                            {:db/id              so
                             :sales-order/date   d
                             :sales-order/client c}))
                     (partition-all 1000))]

    (println (count batch))
    (print ".")
    (flush)
    (dc/transact conn {:tx-data batch})

    )
X auser/index-solr)

LATER
* reset up datomic HA - needs different hostnames
* Update terraform provider
* make solr support graceful shutdown tini/init?




Fix ledger entries:
(clojure.data.csv/write-csv
  *out*
  (->> (auto-ap.datomic/pull-many (dc/as-of (dc/db conn) #inst "2023-03-31T00:00:00") '[:invoice/date {:invoice/client                 [:client/code]
                                                                                                                                 :invoice/expense-accounts
                                                                                                                                 [{:invoice-expense-account/account [:account/name]}]
                                                                                                                                 :journal-entry/_original-entity [{:journal-entry/line-items [{:journal-entry-line/account [:account/name]}]}]}] (->> bad (map first)))
                                 (map (fn [i]
                                        [(-> i :invoice/client :client/code)
                                         (-> i :invoice/date clj-time.coerce/to-date-time (auto-ap.time/unparse auto-ap.time/iso-date))
                                         (-> i :invoice/expense-accounts first :invoice-expense-account/account :account/name)
                                         (-> i :journal-entry/_original-entity first :journal-entry/line-items
                                             (#(filter (fn [a]  (not= "Accounts Payable" (:account/name (:journal-entry-line/account a)))) %))
                                             first :journal-entry-line/account :account/name)]))
                                 ))


BSG,2021-11-19,Advertising,Food Sales
BSG,2021-08-26,Auto and Truck Expenses,Service or Pass Through Costs
BSG,2021-12-01,Advertising,Design
WGC,2021-03-01,Equipment 3,Kitchen Equipment
WGC,2021-02-03,Equipment 3,Kitchen Equipment
WGC,2020-12-14,Marketing Consultant,Website/ Social Media
WGC,2020-12-10,Charitable Contributions,Promotional or Donation Meal Comps
WGC,2020-12-15,Note Payable 3,Notes Payable - General
WGC,2020-11-16,Note Payable 3,Notes Payable - General
WGC,2020-11-05,Unassigned Expenses,Misc Payments
WGC,2021-01-13,Marketing Consultant,Website/ Social Media
WGC,2020-10-15,Note Payable 3,Notes Payable - General
BSG,2021-07-20,Auto and Truck Expenses,Service or Pass Through Costs
WGC,2021-02-12,Marketing Consultant,Website/ Social Media
BSG,2021-12-22,Advertising,Food Sales
BSG,2021-12-31,Advertising,Food Sales
BSG,2021-06-15,Advertising,Design
BSG,2021-12-14,Advertising,Food Sales
BSG,2021-04-15,Advertising,Design
WGC,2021-06-11,Marketing Consultant,Website/ Social Media
BSG,2021-12-14,Food Cost,Meal Comps
WGC,2021-08-05,Marketing Consultant,Website/ Social Media
BSG,2021-09-30,Auto and Truck Expenses,Service or Pass Through Costs
BSG,2021-08-25,Marketing,Marketing Consultant
BSG,2021-12-03,Food Cost,Meal Comps
BSG,2021-12-09,Food Cost,Meal Comps
WGC,2021-10-08,Advertising,Website/ Social Media
BSG,2021-12-09,Advertising,Food Sales
WGC,2021-03-26,Equipment 3,Kitchen Equipment
WGC,2021-03-26,Equipment 3,Kitchen Equipment
BSG,2021-05-26,Advertising,Design
WGC,2021-09-17,Advertising,Promotional or Donation Meal Comps




differences:
q only supports one query format in on-prem, multiple in cloud
qseq takes a different format too
index-pull is not chunked
