The hardest modal in the app: one wizard that both creates and edits invoices,
with a conditional middle step (basic-details → [accounts] → next-steps, where
the expense-accounts step is skipped on the default-accounts path). Migrated off
mm/* + form-cursor + the EDN snapshot onto the session-backed engine (wizard2).
Finding: the OLD basic-details "Save" was broken. It hx-puts /invoice/new/navigate,
whose `[:to {:optional true} …]` query-schema 500s on empty query-params — Ring's
wrap-params yields {} for a no-query PUT, and main-transformer's parse-empty-as-nil
decodes {} → nil, which the bare [:map] rejects. Production uses the identical
wrap-params, so it was broken there too. So e2e/invoice-new.spec.ts is an ACCEPTANCE
gate (red on the old code, green on the engine, whose submit is a POST with no
query-schema): the migration fixes a latent bug. Create semantics (default → vendor
default account, location-spread; customize → posted grid; edit → prefill + updated
row) were pinned at the REPL.
What changed:
- defrecord 4 → 0 (NewWizard2 / BasicDetailsStep / AccountsStep / NextSteps), mm/ 0,
fc/ cursor refs 0, step-params[…] field names 0.
- Conditional `:next` `(if (= :customize …) :accounts :done)` replaces mm/CustomNext +
the broken 308-to-submit. Dual-purpose new+edit = one :init-fn branching on a route
:db/id; create-wizard! seeds :init-data as per-step step-data so edit opens populated.
- The broken new-wizard-navigate route is deleted; the genuine async helpers
(account-prediction, due/scheduled-payment-date, location-select, expense total/balance,
add-row) remain but read the posted flat form (+ ws/get-all for the cross-step total).
- next-steps becomes the done-fn's returned modal (Pay now / Add another / Close).
- Dates ride as java.util.Date (#inst) in step-data so it's EDN-safe across the
non-terminal step (clj-time DateTimes break the cookie store).
Verification: full e2e suite 61/61 (58 prior + 3 new); maybe-spread-locations unit
test 6/6; create semantics + edit prefill confirmed at the REPL. Skill fed
(scorecard Phase 8, gotchas {}→nil 500 + #inst dates, form-vs-wizard conditional
:next + dual-purpose).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
232 lines
12 KiB
Markdown
232 lines
12 KiB
Markdown
# Forms vs. wizards (and the data-driven wizard engine)
|
||
|
||
## Classify first
|
||
|
||
| Signal | Classification |
|
||
|--------|----------------|
|
||
| One logical step — even with a `?mode=` toggle, $/% radio, or add/remove rows | **plain form** |
|
||
| The user genuinely advances through ordered steps, each validated before the next | **wizard** |
|
||
| In doubt | **form** |
|
||
|
||
Most "wizards" in this codebase are single-step forms wearing wizard costumes: they
|
||
implement the multi-step protocol (`mm/ModalWizardStep` + friends), serialize an EDN
|
||
snapshot into hidden fields, and register 10–20 stacked-middleware routes — all for one
|
||
step. That is pure overhead to delete.
|
||
|
||
> **Done — Transaction Edit is now a plain form.** `LinksStep`/`EditWizard` and all `mm/*`
|
||
> usage were deleted from `transaction/edit.clj`; the worked example below is realized, not
|
||
> aspirational. See "Single-step → plain form (realized)".
|
||
|
||
## The machinery being replaced
|
||
|
||
The old shape (kept here as the "before"):
|
||
|
||
```clojure
|
||
(defrecord LinksStep [linear-wizard]
|
||
mm/ModalWizardStep
|
||
(step-name [_] "Transaction Actions")
|
||
(step-key [_] :links)
|
||
(edit-path [_ _] [])
|
||
(step-schema [_] (mm/form-schema linear-wizard))
|
||
(render-step [this {{:keys [snapshot step-params]} :multi-form-state :as request}] ...))
|
||
```
|
||
|
||
…plus the snapshot round-trip: the whole accumulating form state is serialized to hidden
|
||
fields (custom EDN readers), then rebuilt every request by merging the posted pieces back
|
||
into the snapshot (`:multi-form-state :snapshot` is read ~75× in `edit.clj`). The
|
||
serialization needs custom readers, the merge is error-prone, and the payload grows each
|
||
step.
|
||
|
||
---
|
||
|
||
## Single-step → plain form
|
||
|
||
Two routes: `GET` (render) and `POST` (validate + save). State is plain form fields + an
|
||
entity id. No snapshot, no server state, no protocol.
|
||
|
||
```clojure
|
||
{::route/edit (fn [req] (html-response (render-edit-form {:entity (get-entity req)})))
|
||
::route/edit-submit (fn [req] (validate-and-save req))}
|
||
```
|
||
|
||
A `?mode=` toggle is just the `GET` re-rendering with a different query param — still a
|
||
plain form. An add-row interaction is one extra `POST` that appends a fresh row and
|
||
re-renders (the `+1` route).
|
||
|
||
### Single-step → plain form (realized: Transaction Edit)
|
||
|
||
What replacing the wizard actually looked like, end to end:
|
||
|
||
1. **Delete the records + middleware.** `EditWizard`/`LinksStep`, `mm/open-wizard-handler`,
|
||
`mm/next-handler`, `mm/submit-handler`, `mm/wrap-wizard`, `mm/wrap-decode-multi-form-state`,
|
||
and the `edit-wizard-navigate` route all go. `render-step` becomes a plain `render-form`.
|
||
2. **Rename the fields off `step-params[...]`.** Field names are now the schema path
|
||
directly (`(path->name2 :transaction/accounts 0 :transaction-account/account)` →
|
||
`transaction/accounts[0][transaction-account/account]`). They decode straight into the
|
||
form schema via the unchanged `wrap-nested-form-params` + `mc/decode` — no two-key
|
||
snapshot/step-params decode. **Strip stray keys after decode** (`select-keys` to the
|
||
schema's keys) or a non-schema input like the tab group's `method` hidden 500s the save
|
||
(see `gotchas.md`).
|
||
3. **Flat state.** `wrap-derive-state` builds a plain `{:snapshot :edit-path :step-params}`
|
||
map (not the `MultiStepFormState` record): `entity-only` fields from the entity, editable
|
||
fields from the live posted form (absent = cleared). The ~34 `:snapshot` reads keep
|
||
working.
|
||
4. **Validation/error flow without `wrap-ensure-step`.** Reuse the generic
|
||
`wrap-form-4xx-2` directly: `(-> submit-edit (wrap-form-4xx-2 render-form-response) …)`.
|
||
`submit-edit` runs `assert-schema` then dispatches the save; on a throw, `wrap-form-4xx-2`
|
||
re-renders the whole form with `:form-errors` keyed by schema paths. A `*errors*` dynamic
|
||
var (bound by `render-form`) replaces the form-cursor's `*form-errors*` for field lookups.
|
||
5. **Routes shrink to** `edit-wizard` (GET open), `edit-submit` (POST), `edit-form-changed`
|
||
(POST whole-form re-render for dependent changes), `location-select` (GET),
|
||
`unlink-payment` (POST).
|
||
|
||
---
|
||
|
||
## Genuinely multi-step → data-driven engine with session-stored step state
|
||
|
||
> **Inspiration — Django `formtools` `WizardView`.** Django's wizard does *not* round-trip
|
||
> a serialized blob of the whole form through the page. Each step's validated data is
|
||
> written to a **storage backend (the user session by default)** under that step's key,
|
||
> and the steps are combined only at the very end via `get_all_cleaned_data()`. We adopt
|
||
> the same model: **replace the EDN snapshot + piecewise merging with per-step form state
|
||
> stored in the Ring session.** A step writes its own data under its own key; nothing is
|
||
> merged into a snapshot and nothing about other steps rides through the form.
|
||
> Refs: `formtools.wizard.views.WizardView`, `SessionStorage`, `get_all_cleaned_data()`
|
||
> (https://django-formtools.readthedocs.io/en/latest/wizard.html).
|
||
|
||
A wizard is **data**:
|
||
|
||
```clojure
|
||
(def vendor-wizard-config
|
||
{:steps [{:key :info :schema info-schema :fields [...] :render render-info-step
|
||
:next (fn [data] :terms)}
|
||
{:key :terms :schema terms-schema :fields [...] :render render-terms-step
|
||
:next (fn [data] :done)}]
|
||
:init-fn (fn [req] {...})
|
||
:submit-route "/admin/vendor/wizard/submit"
|
||
:done-fn (fn [all-data req] (save! all-data) (html-response "Saved"))})
|
||
```
|
||
|
||
with a tiny engine (no protocols) whose state lives **in the session**, keyed by a wizard
|
||
instance id, each step's data under its own step key — the formtools `SessionStorage`
|
||
model. No snapshot, no custom EDN readers, no merge-into-snapshot:
|
||
|
||
```clojure
|
||
;; Storage backed by the Ring session. Path: [:wizards <wizard-id> :step-data <step-key>]
|
||
(defn create-wizard! [session config]
|
||
(let [id (str (java.util.UUID/randomUUID))]
|
||
[id (assoc-in session [:wizards id]
|
||
{:current-step (-> config :steps first :key) :step-data {}})]))
|
||
|
||
(defn put-step [session id k data] (assoc-in session [:wizards id :step-data k] data)) ; replace, not merge
|
||
(defn set-step [session id k] (assoc-in session [:wizards id :current-step] k))
|
||
(defn get-all [session id] (->> (get-in session [:wizards id :step-data]) vals (apply merge)))
|
||
(defn forget [session id] (update session :wizards dissoc id))
|
||
```
|
||
|
||
The render emits only a **reference token** (`wizard-id`, `current-step`) in the form —
|
||
never the form's state. The submit handler validates the posted step, `put-step`s it,
|
||
computes `:next`, and either advances (`set-step`) or finishes (`get-all` + `:done-fn` +
|
||
`forget`). Every fn returns the updated session for the handler to thread into the Ring
|
||
response (`(assoc resp :session session')`).
|
||
|
||
**Two routes per wizard:** open (`partial open-wizard config`) and submit
|
||
(`partial handle-step-submit config`). State is namespaced by `wizard-id` inside the
|
||
session, so multiple in-flight wizards (and browser tabs) don't collide, and it is
|
||
discarded on completion (`forget`).
|
||
|
||
### Storage lifetime (Open decision 1)
|
||
|
||
State lives in the Ring session, scoped to true multi-step wizards (plain forms hold
|
||
none). Lifetime follows the session; `forget` on completion prevents session bloat. For
|
||
long-lived wizards, confirm the session backend (in-memory vs. durable) is acceptable or
|
||
pick a durable store.
|
||
|
||
## The engine — REALIZED (Phase 6)
|
||
|
||
Built and REPL-proven in Phase 6 as two namespaces (no protocols, no defrecords):
|
||
|
||
- **`auto-ap.ssr.components.wizard-state`** — the pure session-storage layer (the skeleton
|
||
above, fleshed out): `create-wizard!` / `instance` / `exists?` / `current-step` /
|
||
`context` / `step-data` / `put-step` (replace) / `set-step` / `get-all` / `forget`. Each
|
||
is `session -> session'` (or a read); nothing mutates global state. `:context` holds
|
||
read-only data the steps need (e.g. an entity id) **outside** `:step-data`, so it never
|
||
gets merged into the combined result.
|
||
- **`auto-ap.ssr.components.wizard2`** — the engine: `open-wizard`, `render-wizard`,
|
||
`handle-step-submit`, and the `wizard-form` shell. A wizard is a **config map**:
|
||
|
||
```clojure
|
||
{:name :vendor :form-id "wizard-form" :submit-route "<resolved url>"
|
||
:init-fn (fn [request] {:context {...} :init-data {step-key data}})
|
||
:done-fn (fn [all-data request] ring-response)
|
||
:steps [{:key :info
|
||
:decode (fn [request] -> data-map) ; parse this step's posted fields
|
||
:validate (fn [data request] -> errors|nil) ; optional
|
||
:render (fn [ctx] -> hiccup) ; step body; engine wraps the <form>
|
||
:next (fn [data] -> next-step-key | :done)}
|
||
...]}
|
||
```
|
||
|
||
The step's `:render` gets `{:wizard-id :current-step :context :all-data :step-data
|
||
:errors :request :config}`. `:all-data` (every step combined so far) is exactly what a
|
||
**read-only summary/preview step** consumes. Nav buttons post a `direction` field:
|
||
`"next"` (validate+advance via `:next`), `"back"` (no validate), `"submit"` (== next, for
|
||
the last step). Only `wizard-id` + `current-step` ride in the form — **no snapshot**.
|
||
|
||
**Two routes per wizard:** `(partial open-wizard config)` (GET) and
|
||
`(partial handle-step-submit config)` (POST). No `wrap-wizard` / `wrap-decode-multi-form-state`
|
||
stack — the engine threads the session itself and `(assoc resp :session session')`.
|
||
|
||
**Proven via REPL** (lifecycle, before any modal used it): open seeds session state and
|
||
renders step 1 with no accumulated data in the form; next stores `{step-key data}` and
|
||
advances; an invalid step re-renders itself with errors (no advance); the final step's
|
||
`:done` calls `done-fn` with the combined `get-all` data and `forget`s the instance; back
|
||
navigates without validating; an unknown/expired `wizard-id` re-opens fresh instead of
|
||
500-ing. See the lifecycle eval in the Phase 6 commit message.
|
||
|
||
**Note (Phase 6 fit).** Transaction Rule itself is *edit + read-only preview of one
|
||
entity*, not a true multi-data-step flow — so it exercises the engine's render/navigation/
|
||
preview path (`:all-data` feeds the test table) but not the cross-step *merge*. The merge
|
||
(`get-all` combining independent steps) gets its real workout in Phase 7+ (Invoice Pay,
|
||
New Invoice, Vendor, Client), where steps collect genuinely different fields.
|
||
|
||
## Conditional `:next` + dual-purpose (new+edit) — New Invoice (Phase 8)
|
||
|
||
A step's `:next` is just `(fn [data] -> next-step-key | :done)`, so **branching the flow is a
|
||
one-liner** — no `CustomNext` protocol, no 308-redirect-to-submit hack:
|
||
|
||
```clojure
|
||
{:key :basic-details
|
||
:next (fn [data] (if (= :customize (:customize-accounts data)) :accounts :done))}
|
||
```
|
||
|
||
`:default` skips the expense-accounts step entirely (the done-fn uses the vendor's default
|
||
account); `:customize` routes through the grid. The old wizard expressed this with
|
||
`mm/CustomNext` returning either `navigate-handler{:to :accounts}` or a 308 to the submit
|
||
route — and the 308 path was broken (see `gotchas.md`, the `{}`→nil 500). The engine's
|
||
conditional `:next` is both simpler and correct.
|
||
|
||
**Dual-purpose (create *and* edit) = one config, one `:init-fn` that branches on a route id:**
|
||
|
||
```clojure
|
||
(defn new-init-fn [request]
|
||
(if-let [id (->db-id (get-in request [:route-params :db/id]))]
|
||
{:init-data {:basic-details (… entity prefilled, :customize-accounts :customize)
|
||
:accounts {:invoice/expense-accounts (… existing rows)}}} ; edit
|
||
{:init-data {:basic-details {:invoice/date (coerce/to-date (time/now)) ; new
|
||
:customize-accounts :default}}}))
|
||
```
|
||
|
||
`create-wizard!` stores `:init-data` **as the per-step `:step-data` map directly**, so seeding
|
||
`{:basic-details … :accounts …}` opens both steps populated — the edit case repopulates the
|
||
grid without a separate hydrate. Two open routes (`new-wizard`, `edit-wizard`) both reduce to
|
||
`(partial wizard2/open-wizard config)`; the done-fn branches on `(:db/id all-data)` to return
|
||
the next-steps modal (create) vs the swapped table row (edit).
|
||
|
||
**Async step fragments read the posted form, not multi-form-state.** The basic-details
|
||
fragments (account-prediction radio, due-date / scheduled-payment suggestions) and the
|
||
accounts totals all post the whole `#wizard-form`; in the engine that form carries the flat
|
||
`invoice/*` fields + the opaque `wizard-id`, so a fragment decodes what it needs straight from
|
||
`form-params` (and, for a cross-step value like the invoice total on the accounts step, reads
|
||
`ws/get-all` via the posted `wizard-id`). No `mm/wrap-decode-multi-form-state` stack survives.
|