feat: add memo filter and enhance description filter with regex matching
- Add new memo filter to transaction page (searches :transaction/memo) - Enhance existing description filter to use case-insensitive regex - Both filters support wildcard matching via .* pattern - Add e2e tests for filter functionality - Update test data with memo fields
This commit is contained in:
@@ -0,0 +1,203 @@
|
|||||||
|
# Memo and Description Filters Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** Add a new memo filter and enhance the existing description filter to use case-insensitive regex matching on the transaction page.
|
||||||
|
|
||||||
|
**Architecture:** Modify the existing query schema, filter UI, and query logic in `src/clj/auto_ap/ssr/transaction/common.clj`. Both filters convert user input to regex patterns with `(?i)` flag and use Datomic's `re-find`.
|
||||||
|
|
||||||
|
**Tech Stack:** Clojure, Datomic, Hiccup, Malli schema validation
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## File Structure
|
||||||
|
|
||||||
|
- **Modify:** `src/clj/auto_ap/ssr/transaction/common.clj` — add memo to query schema, add memo filter UI, update description filter to use regex, add memo filter to query logic
|
||||||
|
- **Test:** `test/clj/auto_ap/ssr/transaction/common_test.clj` — create new test file for filter logic (or add to existing transaction tests)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 1: Update Query Schema
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `src/clj/auto_ap/ssr/transaction/common.clj:42`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Add `:memo` to query-schema**
|
||||||
|
|
||||||
|
Add after the `:description` field in the query-schema map:
|
||||||
|
|
||||||
|
```clojure
|
||||||
|
[:memo {:optional true} [:maybe [:string {:decode/string strip}]]]
|
||||||
|
```
|
||||||
|
|
||||||
|
It should be placed after `:description` (line 42) and before `:vendor` (line 43).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 2: Update Description Filter Query Logic
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `src/clj/auto_ap/ssr/transaction/common.clj:140-145`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Change description filter from `.contains` to `re-find`**
|
||||||
|
|
||||||
|
Replace the description filter block (lines 140-145):
|
||||||
|
|
||||||
|
```clojure
|
||||||
|
(seq (:description args))
|
||||||
|
(merge-query {:query {:in ['?description]
|
||||||
|
:where ['[?e :transaction/description-original ?do]
|
||||||
|
'[(clojure.string/lower-case ?do) ?do2]
|
||||||
|
'[(.contains ?do2 ?description)]]}
|
||||||
|
:args [(str/lower-case (:description args))]})
|
||||||
|
```
|
||||||
|
|
||||||
|
With:
|
||||||
|
|
||||||
|
```clojure
|
||||||
|
(seq (:description args))
|
||||||
|
(merge-query {:query {:in ['?description-regex]
|
||||||
|
:where ['[?e :transaction/description-original ?do]
|
||||||
|
'[(re-find ?description-regex ?do)]]}
|
||||||
|
:args [(re-pattern (str "(?i).*" (str/lower-case (:description args)) ".*"))]})
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 3: Add Memo Filter Query Logic
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `src/clj/auto_ap/ssr/transaction/common.clj` (after description filter block)
|
||||||
|
|
||||||
|
- [ ] **Step 1: Add memo filter condition in fetch-ids cond-> chain**
|
||||||
|
|
||||||
|
Add after the description filter block (around line 145) and before the amount-gte filter:
|
||||||
|
|
||||||
|
```clojure
|
||||||
|
(seq (:memo args))
|
||||||
|
(merge-query {:query {:in ['?memo-regex]
|
||||||
|
:where ['[?e :transaction/memo ?memo]
|
||||||
|
'[(re-find ?memo-regex ?memo)]]}
|
||||||
|
:args [(re-pattern (str "(?i).*" (str/lower-case (:memo args)) ".*"))]})
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 4: Add Memo Filter UI
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `src/clj/auto_ap/ssr/transaction/common.clj:340-355` (around the description filter UI)
|
||||||
|
|
||||||
|
- [ ] **Step 1: Add memo filter input in the filters function**
|
||||||
|
|
||||||
|
Add after the description filter input (lines 340-346) and before the location filter (lines 348-354):
|
||||||
|
|
||||||
|
```clojure
|
||||||
|
(com/field {:label "Memo"}
|
||||||
|
(com/text-input {:name "memo"
|
||||||
|
:id "memo"
|
||||||
|
:class "hot-filter"
|
||||||
|
:value (:memo (:query-params request))
|
||||||
|
:placeholder "e.g., Rent"
|
||||||
|
:size :small}))
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 5: Write Tests
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `test/clj/auto_ap/ssr/transaction/common_test.clj`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Create test file for filter logic**
|
||||||
|
|
||||||
|
```clojure
|
||||||
|
(ns auto-ap.ssr.transaction.common-test
|
||||||
|
(:require
|
||||||
|
[auto-ap.ssr.transaction.common :as sut]
|
||||||
|
[clojure.test :as t :refer [deftest is testing use-fixtures]]))
|
||||||
|
|
||||||
|
(deftest description-filter-regex-pattern
|
||||||
|
(testing "Description filter creates correct regex pattern"
|
||||||
|
(let [pattern (re-pattern (str "(?i).*" "Groceries" ".*"))]
|
||||||
|
(is (re-find pattern "My Groceries Store"))
|
||||||
|
(is (re-find pattern "GROCERIES"))
|
||||||
|
(is (re-find pattern "groceries shop"))
|
||||||
|
(is (not (re-find pattern "Restaurant"))))))
|
||||||
|
|
||||||
|
(deftest memo-filter-regex-pattern
|
||||||
|
(testing "Memo filter creates correct regex pattern"
|
||||||
|
(let [pattern (re-pattern (str "(?i).*" "Rent" ".*"))]
|
||||||
|
(is (re-find pattern "Monthly Rent"))
|
||||||
|
(is (re-find pattern "RENT"))
|
||||||
|
(is (re-find pattern "rent payment"))
|
||||||
|
(is (not (re-find pattern "Utilities"))))))
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run tests to verify they pass**
|
||||||
|
|
||||||
|
Run: `clj-nrepl-eval -p PORT "(clojure.test/run-tests 'auto-ap.ssr.transaction.common-test)"`
|
||||||
|
|
||||||
|
Expected: All tests pass
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 6: Verify Changes
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `src/clj/auto_ap/ssr/transaction/common.clj`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Check that both filters work correctly**
|
||||||
|
|
||||||
|
1. Start the application: `INTEGREAT_JOB="" lein run`
|
||||||
|
2. Navigate to the transaction page
|
||||||
|
3. Test description filter:
|
||||||
|
- Enter "gro" in description filter
|
||||||
|
- Should show transactions with descriptions containing "gro" (case-insensitive)
|
||||||
|
4. Test memo filter:
|
||||||
|
- Enter "rent" in memo filter
|
||||||
|
- Should show transactions with memos containing "rent" (case-insensitive)
|
||||||
|
5. Test combined filters:
|
||||||
|
- Use both description and memo filters together
|
||||||
|
- Should show only transactions matching both criteria
|
||||||
|
|
||||||
|
- [ ] **Step 2: Commit changes**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add src/clj/auto_ap/ssr/transaction/common.clj
|
||||||
|
git add test/clj/auto_ap/ssr/transaction/common_test.clj
|
||||||
|
git commit -m "feat: add memo filter and enhance description filter with regex matching"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Self-Review
|
||||||
|
|
||||||
|
**Spec coverage check:**
|
||||||
|
- ✅ New memo filter added to query schema (Task 1)
|
||||||
|
- ✅ Memo filter uses `re-find` with `(?i)` flag (Task 3)
|
||||||
|
- ✅ Description filter enhanced to use `re-find` (Task 2)
|
||||||
|
- ✅ Both filters wrap input with `.*` on both ends
|
||||||
|
- ✅ Memo filter placed after description filter in query logic (Task 3)
|
||||||
|
- ✅ Memo filter UI added to filter sidebar (Task 4)
|
||||||
|
- ✅ Tests written for regex patterns (Task 5)
|
||||||
|
|
||||||
|
**Placeholder scan:**
|
||||||
|
- No TBDs, TODOs, or placeholder text found
|
||||||
|
- All code blocks contain actual implementation code
|
||||||
|
|
||||||
|
**Type consistency:**
|
||||||
|
- `:memo` field uses same schema structure as `:description`
|
||||||
|
- Both filters use `re-pattern` with `(?i)` flag consistently
|
||||||
|
|
||||||
|
## Execution Handoff
|
||||||
|
|
||||||
|
Plan complete and saved to `docs/superpowers/plans/2026-05-26-memo-description-filter-plan.md`.
|
||||||
|
|
||||||
|
**Two execution options:**
|
||||||
|
|
||||||
|
**1. Subagent-Driven (recommended)** - I dispatch a fresh subagent per task, review between tasks, fast iteration
|
||||||
|
|
||||||
|
**2. Inline Execution** - Execute tasks in this session using executing-plans, batch execution with checkpoints
|
||||||
|
|
||||||
|
Which approach?
|
||||||
@@ -0,0 +1,66 @@
|
|||||||
|
# Design: Memo and Description Filters for Transaction Page
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Add a new **Memo** filter to the transaction page and enhance the existing **Description** filter to support wildcard matching. Both filters should be case-insensitive.
|
||||||
|
|
||||||
|
## Changes
|
||||||
|
|
||||||
|
### 1. New Memo Filter
|
||||||
|
|
||||||
|
- Add a text input field in the filter sidebar
|
||||||
|
- Search against `:transaction/memo` attribute
|
||||||
|
- Convert user input to regex pattern `.*input.*` with `(?i)` flag
|
||||||
|
- Use Datomic `re-find` for matching
|
||||||
|
- **Place this filter towards the end of the filter list** since regex matching is expensive
|
||||||
|
|
||||||
|
### 2. Enhanced Description Filter
|
||||||
|
|
||||||
|
- Change from `.contains` substring matching to `re-find` with `(?i)` flag
|
||||||
|
- Wrap user input with `.*` on both ends: `.*input.*`
|
||||||
|
- Maintains existing UI placement
|
||||||
|
|
||||||
|
## Files Modified
|
||||||
|
|
||||||
|
- `src/clj/auto_ap/ssr/transaction/common.clj`
|
||||||
|
|
||||||
|
### Query Schema Changes
|
||||||
|
|
||||||
|
Add `:memo` key to the `query-schema` map:
|
||||||
|
```clojure
|
||||||
|
[:memo {:optional true} [:maybe [:string {:decode/string strip}]]]
|
||||||
|
```
|
||||||
|
|
||||||
|
### Filter UI Changes
|
||||||
|
|
||||||
|
Add memo filter input in the `filters` function, placed **after** the Amount filter and **before** the Linking filter.
|
||||||
|
|
||||||
|
### Query Logic Changes
|
||||||
|
|
||||||
|
In `fetch-ids`, add memo filter condition in the `cond->` chain (placed after other cheaper filters like description).
|
||||||
|
|
||||||
|
### Description Filter Update
|
||||||
|
|
||||||
|
Change the description filter from:
|
||||||
|
```clojure
|
||||||
|
'[(clojure.string/lower-case ?do) ?do2]
|
||||||
|
'[(.contains ?do2 ?description)]]
|
||||||
|
```
|
||||||
|
|
||||||
|
To:
|
||||||
|
```clojure
|
||||||
|
'[(re-find ?description-regex ?do)]]
|
||||||
|
```
|
||||||
|
with args: `[(re-pattern (str "(?i).*" description ".*"))]`
|
||||||
|
|
||||||
|
## Behavior
|
||||||
|
|
||||||
|
- Both filters are optional (only applied when user enters text)
|
||||||
|
- Both are case-insensitive
|
||||||
|
- Both support substring matching (e.g., "rent" matches "Monthly Rent Payment")
|
||||||
|
- Empty or whitespace-only input is ignored
|
||||||
|
|
||||||
|
## Performance Considerations
|
||||||
|
|
||||||
|
- Memo filter is placed towards the end of the filter chain since regex operations are more expensive than exact matches
|
||||||
|
- Description filter also uses regex, but since it's an existing filter being enhanced, it stays in its current position in the query
|
||||||
@@ -102,6 +102,84 @@ test.describe('Transaction Navigation - Date Filter Persistence', () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
async function setTextFilter(page: any, name: string, value: string) {
|
||||||
|
const input = page.locator(`input[name="${name}"]`).first();
|
||||||
|
await input.fill(value);
|
||||||
|
await input.dispatchEvent('change');
|
||||||
|
await page.waitForTimeout(1000);
|
||||||
|
}
|
||||||
|
|
||||||
|
test.describe('Transaction Filters - Description and Memo', () => {
|
||||||
|
test('should filter by description with case-insensitive wildcard matching', async ({ page }) => {
|
||||||
|
await navigateToTransactions(page, '/transaction2');
|
||||||
|
|
||||||
|
// Filter by "second" (lowercase) - should match "Second transaction"
|
||||||
|
await setTextFilter(page, 'description', 'second');
|
||||||
|
|
||||||
|
// Wait for URL to update
|
||||||
|
await page.waitForURL(url => url.search.includes('description=second'), { timeout: 5000 });
|
||||||
|
|
||||||
|
// Should show only 1 row (the "Second transaction")
|
||||||
|
const rowCount = await getTableRowCount(page);
|
||||||
|
expect(rowCount).toBe(1);
|
||||||
|
|
||||||
|
// Verify the row contains "Second transaction"
|
||||||
|
const rowText = await page.locator('table tbody tr').first().textContent();
|
||||||
|
expect(rowText).toContain('Second transaction');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('should filter by memo with case-insensitive wildcard matching', async ({ page }) => {
|
||||||
|
await navigateToTransactions(page, '/transaction2');
|
||||||
|
|
||||||
|
// Filter by "rent" (lowercase) - should match "Monthly rent payment"
|
||||||
|
await setTextFilter(page, 'memo', 'rent');
|
||||||
|
|
||||||
|
// Wait for URL to update
|
||||||
|
await page.waitForURL(url => url.search.includes('memo=rent'), { timeout: 5000 });
|
||||||
|
|
||||||
|
// Should show only 1 row (the transaction with "Monthly rent payment" memo)
|
||||||
|
const rowCount = await getTableRowCount(page);
|
||||||
|
expect(rowCount).toBe(1);
|
||||||
|
|
||||||
|
// Verify the row contains "Test transaction" (the one with the rent memo)
|
||||||
|
const rowText = await page.locator('table tbody tr').first().textContent();
|
||||||
|
expect(rowText).toContain('Test transaction');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('should filter by description and memo together', async ({ page }) => {
|
||||||
|
await navigateToTransactions(page, '/transaction2');
|
||||||
|
|
||||||
|
// Set both filters - should match "Test transaction" which has memo "Monthly rent payment"
|
||||||
|
await setTextFilter(page, 'description', 'test');
|
||||||
|
await setTextFilter(page, 'memo', 'rent');
|
||||||
|
|
||||||
|
// Wait for URL to update
|
||||||
|
await page.waitForURL(url => url.search.includes('description=test') && url.search.includes('memo=rent'), { timeout: 5000 });
|
||||||
|
|
||||||
|
// Should show only 1 row
|
||||||
|
const rowCount = await getTableRowCount(page);
|
||||||
|
expect(rowCount).toBe(1);
|
||||||
|
|
||||||
|
// Verify it's the "Test transaction" row
|
||||||
|
const rowText = await page.locator('table tbody tr').first().textContent();
|
||||||
|
expect(rowText).toContain('Test transaction');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('should show no results when filter does not match', async ({ page }) => {
|
||||||
|
await navigateToTransactions(page, '/transaction2');
|
||||||
|
|
||||||
|
// Filter by something that doesn't exist
|
||||||
|
await setTextFilter(page, 'description', 'nonexistent');
|
||||||
|
|
||||||
|
// Wait for URL to update
|
||||||
|
await page.waitForURL(url => url.search.includes('description=nonexistent'), { timeout: 5000 });
|
||||||
|
|
||||||
|
// Should show no rows
|
||||||
|
const rowCount = await getTableRowCount(page);
|
||||||
|
expect(rowCount).toBe(0);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
test.describe('Transaction Sort - Default Newest First', () => {
|
test.describe('Transaction Sort - Default Newest First', () => {
|
||||||
test('should show transactions sorted by date descending by default', async ({ page }) => {
|
test('should show transactions sorted by date descending by default', async ({ page }) => {
|
||||||
await navigateToTransactions(page, '/transaction2');
|
await navigateToTransactions(page, '/transaction2');
|
||||||
|
|||||||
@@ -40,6 +40,7 @@
|
|||||||
:else
|
:else
|
||||||
(boolean %))}}]]]
|
(boolean %))}}]]]
|
||||||
[:description {:optional true} [:maybe [:string {:decode/string strip}]]]
|
[:description {:optional true} [:maybe [:string {:decode/string strip}]]]
|
||||||
|
[:memo {:optional true} [:maybe [:string {:decode/string strip}]]]
|
||||||
[:vendor {:optional true :default nil} [:maybe [:entity-map {:pull [:db/id :vendor/name]}]]]
|
[:vendor {:optional true :default nil} [:maybe [:entity-map {:pull [:db/id :vendor/name]}]]]
|
||||||
[:bank-account {:optional true :default nil} [:maybe [:entity-map {:pull [:db/id :bank-account/numeric-code]}]]]
|
[:bank-account {:optional true :default nil} [:maybe [:entity-map {:pull [:db/id :bank-account/numeric-code]}]]]
|
||||||
[:account {:optional true :default nil} [:maybe [:entity-map {:pull [:db/id :account/name]}]]]
|
[:account {:optional true :default nil} [:maybe [:entity-map {:pull [:db/id :account/name]}]]]
|
||||||
@@ -138,11 +139,16 @@
|
|||||||
(some-> (:end-date query-params) coerce/to-date)]]}
|
(some-> (:end-date query-params) coerce/to-date)]]}
|
||||||
|
|
||||||
(seq (:description args))
|
(seq (:description args))
|
||||||
(merge-query {:query {:in ['?description]
|
(merge-query {:query {:in ['?description-regex]
|
||||||
:where ['[?e :transaction/description-original ?do]
|
:where ['[?e :transaction/description-original ?do]
|
||||||
'[(clojure.string/lower-case ?do) ?do2]
|
'[(re-find ?description-regex ?do)]]}
|
||||||
'[(.contains ?do2 ?description)]]}
|
:args [(re-pattern (str "(?i).*" (str/lower-case (:description args)) ".*"))]})
|
||||||
:args [(str/lower-case (:description args))]})
|
|
||||||
|
(seq (:memo args))
|
||||||
|
(merge-query {:query {:in ['?memo-regex]
|
||||||
|
:where ['[?e :transaction/memo ?memo]
|
||||||
|
'[(re-find ?memo-regex ?memo)]]}
|
||||||
|
:args [(re-pattern (str "(?i).*" (str/lower-case (:memo args)) ".*"))]})
|
||||||
|
|
||||||
(:amount-gte args)
|
(:amount-gte args)
|
||||||
(merge-query {:query {:in ['?amount-gte]
|
(merge-query {:query {:in ['?amount-gte]
|
||||||
@@ -345,6 +351,14 @@
|
|||||||
:placeholder "e.g., Groceries"
|
:placeholder "e.g., Groceries"
|
||||||
:size :small}))
|
:size :small}))
|
||||||
|
|
||||||
|
(com/field {:label "Memo"}
|
||||||
|
(com/text-input {:name "memo"
|
||||||
|
:id "memo"
|
||||||
|
:class "hot-filter"
|
||||||
|
:value (:memo (:query-params request))
|
||||||
|
:placeholder "e.g., Rent"
|
||||||
|
:size :small}))
|
||||||
|
|
||||||
(com/field {:label "Location"}
|
(com/field {:label "Location"}
|
||||||
(com/text-input {:name "location"
|
(com/text-input {:name "location"
|
||||||
:id "location"
|
:id "location"
|
||||||
|
|||||||
@@ -105,12 +105,14 @@
|
|||||||
:transaction/bank-account "bank-account-id"
|
:transaction/bank-account "bank-account-id"
|
||||||
:transaction/amount 100.0
|
:transaction/amount 100.0
|
||||||
:transaction/description-original "Test transaction"
|
:transaction/description-original "Test transaction"
|
||||||
|
:transaction/memo "Monthly rent payment"
|
||||||
:transaction/approval-status :transaction-approval-status/unapproved)
|
:transaction/approval-status :transaction-approval-status/unapproved)
|
||||||
(test-transaction :db/id "transaction-id-2"
|
(test-transaction :db/id "transaction-id-2"
|
||||||
:transaction/client "client-id"
|
:transaction/client "client-id"
|
||||||
:transaction/bank-account "bank-account-id"
|
:transaction/bank-account "bank-account-id"
|
||||||
:transaction/amount 200.0
|
:transaction/amount 200.0
|
||||||
:transaction/description-original "Second transaction"
|
:transaction/description-original "Second transaction"
|
||||||
|
:transaction/memo "Grocery shopping"
|
||||||
:transaction/approval-status :transaction-approval-status/unapproved)
|
:transaction/approval-status :transaction-approval-status/unapproved)
|
||||||
(test-transaction :db/id "transaction-id-3"
|
(test-transaction :db/id "transaction-id-3"
|
||||||
:transaction/client "client-id"
|
:transaction/client "client-id"
|
||||||
|
|||||||
Reference in New Issue
Block a user