notid/integreat
1
0
Fork 0
Code Issues Pull Requests 3 Actions Packages Projects Releases Wiki Activity

feat(transactions): port manual bank-transaction import to SSR #9

Merged
notid merged 2 commits from integreat-add-transaction-manual into staging 2026-06-01 21:06:52 -07:00
Conversation 0 Commits 2 Files Changed 8 +1062 -14
notid commented 2026-06-01 19:58:06 -07:00
Owner
Copy Link

Summary

Ports the master-branch manual bank-transaction import to the SSR/alpinejs/htmx stack. The route names ::external-import-page/parse/import were already declared in routes/transactions.cljc but no handler served them — this fills that gap, modeled on the existing SSR ledger import.

Flow: a dedicated admin page (/transaction2/external-import-new) where you paste the exact same Yodlee positional-column TSV as before → parsed rows render in an editable review grid with per-row error/warning badges → import. Every master validation is preserved, and the existing import.transactions engine is reused unchanged (via import.manual/import-batch).

What changed

  • New ns auto-ap.ssr.transaction.import — page, clipboard/paste form, parse handler, editable form-cursor grid, two-tier validation, and import handler. Wired into ssr/transaction.clj key->handler; admin-only via wrap-must {:activity :import :subject :transaction} + assert-admin.
  • Two-tier validation. Tier 1 (malli, positional Yodlee columns) parses shape; Tier 2 attaches [message status] per row. Hard errors (bad date, unparseable amount, unknown client/bank-account code, missing fields) block the whole batch (throw+ :field-validation re-renders the grid). Warnings (before bank-account start-date, before client locked-until, already-imported) skip just that row. Warn conditions are computed from the engine's own categorize-transaction, so the grid preview matches the import result.
  • Transactions "Import" nav entry (admin-only) in ssr/components/aside.clj.
  • Tests — e2e/transaction-import.spec.ts (3 Playwright tests, committed failing-first then made green) and ssr/transaction/import_test.clj (10 clojure.test deftests, 33 assertions, against in-memory Datomic). Deterministic bank-account code added to the test-server seed.

Validation

  • Unit/integration: 10 tests / 33 assertions green (parsing, each validation clause, clean import, block-on-hard-error, idempotent re-import → extant, warn-skip).
  • e2e (real Chromium): renders the page, paste→parse→grid→import a valid row (appears on the list), and a blocking-error batch creates nothing.
  • clj-kondo: 0 warnings. cljfmt: clean.
  • The 2 unrelated failures in the existing e2e suite (transaction-navigation date-range persistence, transaction-edit shared-location) were verified pre-existing — they fail on the baseline with this branch's changes stashed.

Residual Review Findings

From the automated code review (advisory; not blocking this PR):

Engineering follow-ups

  • [P2] client-code is an editable grid column the import never consumes (client is derived from bank-account-code) — remove or make read-only. (maintainability + correctness)
  • [P2] No unit-level test for the wrap-nested-form-params → :table coercion on the edit-then-import path (covered by e2e end-to-end, not at unit level). (testing + reliability)
  • [P2] classify-table re-runs the two full-table lookups that import-batch also runs (~4 queries/cycle); thread resolved lookups through. Minor at admin scale. (maintainability + performance)
  • [P2] classify-table DB queries lack try/catch → raw 500 on Datomic failure (mirrors ledger's existing pattern). (reliability)
  • [P3] Parse helpers partially duplicate import.manual/tabulate-data; private import-transactions helper name collides with the import.transactions (t) alias. (maintainability)

Product decisions (master-faithful as shipped)

  • [P2] Non-POSTED rows import as POSTED (status is forced, like master), so the "not posted → skip" warning sub-condition can't fire. Decide: keep master parity (drop the wording) or pass the row's real status through.
  • [P2] On import the grid is cleared even when some rows were skipped/errored; write-time errors surface only as an aggregate count (matches master's stats behavior).

Plan

docs/plans/2026-06-01-001-feat-manual-transaction-import-ssr-plan.md

🤖 Generated with Claude Code

## Summary Ports the master-branch **manual bank-transaction import** to the SSR/alpinejs/htmx stack. The route names `::external-import-page/parse/import` were already declared in `routes/transactions.cljc` but no handler served them — this fills that gap, modeled on the existing SSR ledger import. Flow: a dedicated admin page (`/transaction2/external-import-new`) where you paste the **exact same Yodlee positional-column TSV** as before → parsed rows render in an **editable review grid** with per-row error/warning badges → import. Every master validation is preserved, and the existing `import.transactions` engine is **reused unchanged** (via `import.manual/import-batch`). ## What changed - **New ns `auto-ap.ssr.transaction.import`** — page, clipboard/paste form, parse handler, editable form-cursor grid, two-tier validation, and import handler. Wired into `ssr/transaction.clj` `key->handler`; admin-only via `wrap-must {:activity :import :subject :transaction}` + `assert-admin`. - **Two-tier validation.** Tier 1 (malli, positional Yodlee columns) parses shape; Tier 2 attaches `[message status]` per row. *Hard errors* (bad date, unparseable amount, unknown client/bank-account code, missing fields) **block the whole batch** (`throw+ :field-validation` re-renders the grid). *Warnings* (before bank-account start-date, before client locked-until, already-imported) **skip just that row**. Warn conditions are computed from the engine's own `categorize-transaction`, so the grid preview matches the import result. - **Transactions "Import" nav entry** (admin-only) in `ssr/components/aside.clj`. - **Tests** — `e2e/transaction-import.spec.ts` (3 Playwright tests, committed failing-first then made green) and `ssr/transaction/import_test.clj` (10 clojure.test deftests, 33 assertions, against in-memory Datomic). Deterministic bank-account code added to the test-server seed. ## Validation - Unit/integration: 10 tests / 33 assertions green (parsing, each validation clause, clean import, block-on-hard-error, idempotent re-import → extant, warn-skip). - e2e (real Chromium): renders the page, paste→parse→grid→import a valid row (appears on the list), and a blocking-error batch creates nothing. - clj-kondo: 0 warnings. cljfmt: clean. - The 2 unrelated failures in the existing e2e suite (`transaction-navigation` date-range persistence, `transaction-edit` shared-location) were verified **pre-existing** — they fail on the baseline with this branch's changes stashed. ## Residual Review Findings From the automated code review (advisory; not blocking this PR): **Engineering follow-ups** - **[P2]** `client-code` is an editable grid column the import never consumes (client is derived from bank-account-code) — remove or make read-only. *(maintainability + correctness)* - **[P2]** No unit-level test for the `wrap-nested-form-params → :table` coercion on the edit-then-import path (covered by e2e end-to-end, not at unit level). *(testing + reliability)* - **[P2]** `classify-table` re-runs the two full-table lookups that `import-batch` also runs (~4 queries/cycle); thread resolved lookups through. Minor at admin scale. *(maintainability + performance)* - **[P2]** `classify-table` DB queries lack try/catch → raw 500 on Datomic failure (mirrors ledger's existing pattern). *(reliability)* - **[P3]** Parse helpers partially duplicate `import.manual/tabulate-data`; private `import-transactions` helper name collides with the `import.transactions` (`t`) alias. *(maintainability)* **Product decisions (master-faithful as shipped)** - **[P2]** Non-POSTED rows import as POSTED (status is forced, like master), so the "not posted → skip" warning sub-condition can't fire. Decide: keep master parity (drop the wording) or pass the row's real status through. - **[P2]** On import the grid is cleared even when some rows were skipped/errored; write-time errors surface only as an aggregate count (matches master's stats behavior). ## Plan `docs/plans/2026-06-01-001-feat-manual-transaction-import-ssr-plan.md` 🤖 Generated with [Claude Code](https://claude.com/claude-code)
notid added 2 commits 2026-06-01 19:58:06 -07:00
feat(transactions): port manual bank-transaction import to SSR a1098b28f8 
Implement the SSR/alpine/htmx manual transaction import, wiring the
already-declared but unhandled ::external-import-page/parse/import routes.
Mirrors the SSR ledger import: paste the exact master-branch Yodlee
positional-column TSV, review parsed rows in an editable grid with per-row
error/warning badges, and import. Every master validation is preserved and
the existing import.transactions engine is reused unchanged
(via import.manual/import-batch), so core components are untouched.

- New ns auto-ap.ssr.transaction.import (page, paste/parse, editable grid,
  two-tier validation, import handler) + admin-only transactions Import nav.
- Two-tier validation: fixable problems (bad date/amount, unknown client or
  bank-account code, missing fields) are hard errors that block the whole
  batch; inherent skip-conditions (non-POSTED, before start-date/locked,
  already-imported) are warnings computed from the engine's own
  categorize-transaction so the grid preview matches the import result.
- Tests: failing-first Playwright e2e (e2e/transaction-import.spec.ts) plus
  unit/integration coverage (ssr/transaction/import_test.clj, 10 tests).
- Deterministic bank-account code in the e2e seed.

Plan: docs/plans/2026-06-01-001-feat-manual-transaction-import-ssr-plan.md

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
fix(review): apply autofix feedback 9cc3418b1b 
- Alphabetize the import.clj :require block (AGENTS.md Import Formatting).
- Remove unused imports (digest, strip) flagged by clj-kondo.
- Make the client-not-found classify-table test independent: it previously
  reused the bank-account-not-found input and added zero marginal coverage;
  now seeds an orphan bank account so only the client error fires.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
notid merged commit 569e52d1c1 into staging 2026-06-01 21:06:52 -07:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
notid
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: notid/integreat#9
Delete Branch "integreat-add-transaction-manual"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?