lots of progress on input processing

This commit is contained in:
Bryce
2025-08-03 10:09:53 -07:00
parent 97545d89d2
commit b0952aee58
29 changed files with 1172 additions and 32 deletions

113
README.md
View File

@@ -1,38 +1,87 @@
Email Organizer
# Email Organizer
This repository is a self-hosted version of our paid offering for organizing your email with AI. Simply run it, connect it to your IMAP server, explain in natural language how you want your mail organized, and forget it!
A self-hosted AI-powered email organization system that automates folder sorting, prioritization, and rule recommendations through natural language configuration.
Key problems we solve:
1. No more hand writing email sieve/filter rules!
1. Separate out marketing emails from receipts / transaction confirmations.
1. Automatically star high importance emails based off learning your own preference.
1. Automatically recommend new folders/organization rules based off your own activity. For example, if you have a social media folder, and you start using tiktok, this service can recommend adding that as an organization rule.
## Core Value Proposition
- **Natural Language Rules**: Define email organization logic in plain English instead of writing sieve/filter rules
- **Smart Separation**: Automatically categorize marketing emails, receipts, and transactional messages
- **Adaptive Intelligence**: Learns your preferences to star important emails and suggest new organizational patterns
Roadmap:
1. [ ] Prototype - Setting up the base infrastructure and user experience
1. [ ] MVP - enter your imap server. You list out folders and rules, and once a day emails in the "Pending" folder will be reorganized.
1. [ ] "Import" your list of folders from your imap server
1. [ ] Polling every 5 minutes, working off recent emails since last poll
1. [ ] Label support for services like gmail
1. [ ] Generate config to automate integration into imap servers like dovecot or gmail or proton
1. [ ] Supporting auth / multi user. Admin controls the list of users, users control their folders.
1. [ ] Making a paid, hosted version
1. [ ] Auth via google / facebook / etc.
1. [ ] Automatically star based off of your habits
## Technical Overview
**Stack**:
- Backend: Flask (Python 3.10+)
- Frontend: HTMX + AlpineJS + DaisyUI (Tailwind CSS)
- Database: PostgreSQL
- AI: OpenAI-compatible API endpoints
Architectural concepts:
1. The web application is built using Flask for a web server.
1. HTMX is used to give SPA-like experience.
1. AlpineJS is used for anything
1. DaisyUI is used (on top of tailwind css) for the front end experience.
1. Everything is stored in postgres
1. OpenAI-compatible endpoints are used for email classification.
**Key Components**:
1. **Rule Engine**: Converts natural language rules → executable classification logic
2. **Email Processor**: Polls IMAP server, applies AI classifications, moves messages
3. **Recommendation System**: Analyzes usage patterns to suggest new folders/rules
How it works:
1. The user interface allows the user to specify a list of folders, and in plain language what content should go in those folders
1. When an email is considered, a prompt is generated from all of the user input, specifying the rules, as well as a detailed system prompt.
1. If an email cannot be organized, it's left in place
## Getting Started
### Prerequisites
- Python 3.10+
- PostgreSQL 14+
- IMAP server access
Initial data model:
1. Users (id | user_name | salted_password_hash | imap_server | imap_password )
1. Folders (id | user_id | organization_rule | priority)
### Setup
```bash
# Install dependencies
pip install -r requirements.txt
# Configure environment
cp .env.example .env
# (Edit .env with your DB/IMAP credentials)
# Initialize database
flask db upgrade
# Run development server
flask run
```
## Roadmap
### Milestone 1: Prototype (Current Focus)
- [ ] Core infrastructure setup
- [ ] Basic UI for rule configuration
- [ ] Mock email processing pipeline
- [ ] Database schema implementation
### Future Milestones
- [ ] MVP - enter your imap server. You list out folders and rules, and once a day emails in the "Pending" folder will be reorganized.
- [ ] "Import" your list of folders from your imap server
- [ ] Polling every 5 minutes, working off recent emails since last poll
- [ ] Label support for services like gmail
- [ ] Generate config to automate integration into imap servers like dovecot or gmail or proton
- [ ] Supporting auth / multi user. Admin controls the list of users, users control their folders.
- [ ] Making a paid, hosted version
- [ ] Auth via google / facebook / etc.
- [ ] Automatically star based off of your habits
## Data Model
**Users**
| Column | Type | Description |
|--------|------|-------------|
| id | UUID | Primary key |
| email | VARCHAR | Unique identifier |
| password_hash | BYTEA | Argon2-hashed |
| imap_config | JSONB | Encrypted server settings |
**Folders**
| Column | Type | Description |
|--------|------|-------------|
| id | UUID | Primary key |
| user_id | UUID | Foreign key |
| name | VARCHAR | Display name |
| rule_text | TEXT | Natural language rule |
| priority | INT | Processing order |
## Contributing
1. Fork the repository
2. Create your feature branch (`git checkout -b feature/AmazingFeature`)
3. Commit your changes (`git commit -m 'Add some AmazingFeature'`)
4. Push to the branch (`git push origin feature/AmazingFeature`)
5. Open a Pull Request
> **Note**: All new features require corresponding unit tests and documentation updates.