Compare commits

...

17 Commits

Author SHA1 Message Date
8dd7ae8c95 feat(dashboard): add admin-only Last Synced column
Adds a "Last Synced" column visible only to admins, positioned as the
last column. Displays the sync timestamp formatted as YYYY-MM-DD.
Includes the column in the visibility toggle modal.
2026-05-12 23:57:26 -07:00
616ffde402 fix: exclude archived projects from oldest-percent sync selection
Archived projects should not count toward the N% fraction nor be selected for syncing.
2026-05-12 23:44:52 -07:00
eb78676cdb feat: add script to backfill is_archived field on existing projects
One-time migration to set is_archived = (phase_name == 'Archived') for all projects already in Firestore.
2026-05-12 23:40:46 -07:00
3633923fa7 refactor: overhaul sync script with CLI modes, batch writes, and archive tracking
Add argparse with full/last_n/oldest_percent/hybrid/single sync modes.

Implement batch Firestore writes to reduce API overhead.

Add is_archived flag based on phase_name during sync.

Track last_synced_at on each project for incremental sync.

Improve logging with structured prefixes and worker summaries.

Remove dead code (duplicate date function, sync_single helper).
2026-05-12 23:40:25 -07:00
c62de705de feat: exclude archived projects from dashboard and query results
Add is_archived == False filter to all project queries so archived cases are hidden from users.
2026-05-12 23:39:49 -07:00
3ed260ef23 feat: add last_synced_at field to ProjectModel
Track when each project was last synced from Filevine for incremental sync strategies.
2026-05-12 23:39:30 -07:00
9df9e003c1 refactor: use requests Session for connection pooling and clean up debug output
Replace per-request requests.get/post with a shared Session for connection reuse.

Remove verbose print statements and add structured [WARN] prefixes to error logs.
2026-05-12 23:38:46 -07:00
dc81c8e2a7 fixes 2026-04-01 13:51:25 -07:00
c3263a0eaf undid 2026-02-19 20:29:51 -08:00
607e65560c resets passwords 2026-01-29 20:51:14 -08:00
86a09225e7 fixes 2026-01-28 13:10:56 -08:00
dc6c24ca6d fixes 2026-01-27 22:06:18 -08:00
6cde9ab75f progress 2026-01-03 21:28:34 -08:00
a1b836b392 fixes export 2025-12-17 16:14:02 -08:00
234578b646 supports lookup by domains 2025-12-09 22:01:06 -08:00
c3108ff68c Improvement 2025-12-05 00:10:08 -08:00
c3e943f135 final tweaks. 2025-12-04 10:55:23 -08:00
22 changed files with 1457 additions and 193 deletions

View File

@@ -0,0 +1,166 @@
# Filevine API Curl Skill
This skill enables intelligent exploration of the Filevine API by making curl requests and analyzing data patterns.
## Capabilities
1. **Discover relevant data within a project** - Search across all endpoints used in sync.py
2. **Make educated guesses** - Use patterns from sample data to infer where data might exist
3. **Query specific fields** - Search for fields like "matter number", "case number", dates, etc.
## API Endpoints Discovered
Based on analysis of `sync.py`, the following endpoints are used:
| Endpoint | Purpose | Related Code |
|----------|---------|--------------|
| `Projects/{id}` | Project details (name, URLs, dates, phase) | `fetch_project_detail()` |
| `Projects/{id}/team` | Team members and roles | `fetch_project_team()` |
| `Projects/{id}/tasks` | Project tasks | `fetch_project_tasks()` |
| `Projects/{id}/Forms/{form}` | Specific forms (datesAndDeadlines, newFileReview, etc.) | `fetch_form()` |
| `Projects/{id}/Contacts` | Project contacts | `fetch_contacts()` |
| `Contacts/{id}` | Client information | `fetch_client()` |
| `Projects/{id}/Collections/{collection}` | Collections (serviceInfo) | `fetch_collection()` |
## Sample Data Patterns
From examining `examples/*.json`, here are common patterns:
**Project-level fields (Projects/{id}):**
- `projectName` - Full project name
- `number` - Project number
- `incidentDate` - Incident date
- `projectEmailAddress` - Email address
- `projectUrl` - Link to Filevine
- `phaseName` - Current phase
- `createdDate` - Creation date
**DatesAndDeadlines form:**
- `caseNumber` - Case number
- `dateCaseFiled` - Case filed date
- `defaultDate` - Default date
- `trialDate` - Trial date
- `mSCDate` - MSC date
- `noticeExpirationDate` - Notice expiration
- `judgmentDate` - Judgment date
**NewFileReview form:**
- `noticeServiceDate` - When notice was served
- `noticeExpirationDate` - Notice expiration
- `noticeType` - Type of notice (NP, PPQ, etc.)
- `amountDemandedInNotice` - Amount demanded
**ComplaintInfo form:**
- `totalDamagesSought` - Total damages
- `complaintVerificationBy` - Verification method
## Usage
When the user asks about finding data, follow this workflow:
### 1. Parse the Query
Identify what the user wants to find:
- Field name (e.g., "matter number", "case number", "trial date")
- Context (e.g., "in a project", "in the datesAndDeadlines form")
### 2. Generate Endpoint Candidates
Based on the query, suggest multiple endpoints to check:
**For case/project identifiers:**
- `Projects/{id}` (check `number`, `caseNumber` fields)
- `Projects/{id}/Forms/datesAndDeadlines` (check `caseNumber` field)
- `Projects/{id}/Forms/matterOverview` (check `matterNumber` field)
**For dates:**
- `Projects/{id}` (check `incidentDate`, `createdDate`)
- `Projects/{id}/Forms/datesAndDeadlines` (check `trialDate`, `mSCDate`, `defaultDate`, etc.)
- `Projects/{id}/Forms/newFileReview` (check `noticeServiceDate`)
**For contact/defendant info:**
- `Projects/{id}/Contacts` (check contact list for defendant)
- `Contacts/{client_id}` (check full contact info)
**For attorney/team:**
- `Projects/{id}/team` (check `teamOrgRoles` for "Assigned Attorney", "Primary", etc.)
**For notices:**
- `Projects/{id}/Forms/newFileReview` (check `noticeType`, `noticeExpirationDate`, etc.)
- `Projects/{id}/Collections/serviceInfo` (check service dates)
### 3. Execute curl Commands
For each candidate endpoint, construct and execute curl commands using the auth headers from the codebase:
```bash
curl -X GET "https://api.filevineapp.com/fv-app/v2/Projects/{PROJECT_ID}/{ENDPOINT}" \
-H "Accept: application/json" \
-H "Authorization: Bearer {BEARER_TOKEN}" \
-H "x-fv-orgid: {ORG_ID}" \
-H "x-fv-userid: {USER_ID}"
```
Use the project ID from sample data (e.g., `15974631` or `15914808`) as a placeholder.
### 4. Analyze Results
Look for the queried field in the JSON response:
- Search field names (not values)
- Check nested objects and arrays
- Note which endpoints actually contain the data
### 5. Report Findings
Provide:
1. The field value(s) found
2. The endpoint(s) that contain it
3. Any related fields discovered in the same endpoint
4. Context about how this data is used in the application
## Common Patterns to Guess From
When the user asks "What endpoint should I hit to find [field]?", consider:
**Field Mapping:**
| Common Field Names | Likely Endpoints |
|-------------------|------------------|
| `matterNumber`, `matter number` | `Projects/{id}/Forms/matterOverview` |
| `caseNumber`, `case number` | `Projects/{id}/Forms/datesAndDeadlines` |
| `incident date`, `incidentDate` | `Projects/{id}`, `Projects/{id}/Forms/datesAndDeadlines` |
| `trial date`, `trialDate` | `Projects/{id}/Forms/datesAndDeadlines` |
| `notice service date`, `noticeServiceDate` | `Projects/{id}/Forms/newFileReview` |
| `notice expiration date`, `noticeExpirationDate` | `Projects/{id}/Forms/newFileReview` |
| `notice type` | `Projects/{id}/Forms/newFileReview` |
| `assigned attorney` | `Projects/{id}/team` (role: "Assigned Attorney") |
| `primary contact`, `staff person` | `Projects/{id}/team` (role: "Primary") |
| `defendant` | `Projects/{id}/Contacts` (filter for "Defendant" role) |
| `client` | `Contacts/{client_id}` or `Projects/{id}` (clientId field) |
## Environment Setup
When executing curl commands, use these placeholder values (user should provide actual values):
- `PROJECT_ID` - Use sample IDs like 15974631 or 15914808
- `ORG_ID` - From FILEVINE_ORG_ID env var
- `USER_ID` - From FILEVINE_USER_ID env var
- `BEARER_TOKEN` - From FilevineClient.get_bearer_token()
## Tips
1. Start with the most likely endpoints first
2. The sample data shows that many fields are duplicated across endpoints - check multiple places
3. Date fields are often in ISO format in API responses
4. Some forms (like `newFileReview`) contain multiple related date fields
5. Team roles use specific names - "Assigned Attorney", "Primary", "Secondary Paralegal"
6. Contact objects in arrays may have `personTypes` to identify their role
## Example Workflow
User: "What endpoint should I hit to find the matter number?"
1. Identify: User wants "matter number" field
2. Guess: Matter number might be in `Projects/{id}/Forms/matterOverview` based on sample data patterns
3. Check: `Projects/{id}` also has `number` field
4. Execute: curl to both endpoints with project 15974631
5. Report: Found `number` in project root and `matterNumber` in matterOverview form

5
.env
View File

@@ -9,8 +9,9 @@ GOOGLE_APPLICATION_CREDENTIALS=./rothbard-staging2-12345-firebase-adminsdk-fbsvc
# Filevine auth # Filevine auth
FILEVINE_CLIENT_ID=4F18738C-107A-4B82-BFAC-308F1B6A626A FILEVINE_CLIENT_ID=4F18738C-107A-4B82-BFAC-308F1B6A626A
FILEVINE_CLIENT_SECRET=*2{aXWvYN(9!BiYUXC_tXj^n8 FILEVINE_CLIENT_SECRET=q<}QzfD^3t_atF-7+U8(gJCgj
FILEVINE_PERSONAL_ACCESS_TOKEN=C8F5C606B834D4EE0CBF0793969496F6210037EB934523756BA80BF2D8EC1880 FILEVINE_PERSONAL_ACCESS_TOKEN=68BEBFB8437B9668642BD71EDEAF09593ACF075CE35AE4079FC7588410094210
FILEVINE_ORG_ID=9227 FILEVINE_ORG_ID=9227
FILEVINE_USER_ID=100510 FILEVINE_USER_ID=100510

1
.gitignore vendored
View File

@@ -11,3 +11,4 @@ __pycache__/**
**/*.pyc **/*.pyc
node_modules/**

139
admin.py
View File

@@ -1,7 +1,9 @@
import json import json
import os import os
import random
import string
from functools import wraps from functools import wraps
from flask import render_template, request, redirect, url_for, session, abort, jsonify from flask import render_template, request, redirect, url_for, session, abort, jsonify, flash
from firebase_init import db from firebase_init import db
from firebase_admin import auth as fb_auth from firebase_admin import auth as fb_auth
from utils import get_user_profile from utils import get_user_profile
@@ -45,8 +47,10 @@ def register_admin_routes(app):
"uid": doc.id, "uid": doc.id,
"user_email": user_data.get("user_email", ""), "user_email": user_data.get("user_email", ""),
"case_email": user_data.get("case_email", ""), "case_email": user_data.get("case_email", ""),
"case_domain_email": user_data.get("case_domain_email", ""),
"enabled": bool(user_data.get("enabled", False)), "enabled": bool(user_data.get("enabled", False)),
"is_admin": bool(user_data.get("is_admin", False)) "is_admin": bool(user_data.get("is_admin", False)),
"password_reset_required": bool(user_data.get("password_reset_required", False))
}) })
except Exception as e: except Exception as e:
@@ -78,6 +82,7 @@ def register_admin_routes(app):
"uid": uid, "uid": uid,
"user_email": user_data.get("user_email", ""), "user_email": user_data.get("user_email", ""),
"case_email": user_data.get("case_email", ""), "case_email": user_data.get("case_email", ""),
"case_domain_email": user_data.get("case_domain_email", ""),
"enabled": bool(user_data.get("enabled", False)), "enabled": bool(user_data.get("enabled", False)),
"is_admin": bool(user_data.get("is_admin", False)) "is_admin": bool(user_data.get("is_admin", False))
} }
@@ -92,17 +97,23 @@ def register_admin_routes(app):
# Get the user from Firebase Auth # Get the user from Firebase Auth
user = fb_auth.get_user(uid) user = fb_auth.get_user(uid)
# Generate password reset link using Firebase Auth
password_reset_link = fb_auth.generate_password_reset_link(user.email)
# Send password reset email using Firebase's built-in template # Generate temporary password (random word + 3 digits)
# This will send an email to the user with a link to reset their password words = ["sun", "moon", "star", "cloud", "rain", "wind", "fire", "water", "snow", "stone",
# Firebase automatically handles the email template and delivery "tree", "leaf", "flower", "bird", "wolf", "tiger", "bear", "fish", "dragon",
print(f"[INFO] Password reset link generated for {user.email}: {password_reset_link}") "magic", "quest", "light", "dark", "gold", "silver", "ruby", "pearl", "diamond"]
random_word = random.choice(words)
random_digits = ''.join(random.choices(string.digits, k=3))
temp_password = f"{random_word}{random_digits}"
# Store the password reset link in the session for display in the banner # Create user profile in Firestore with password reset required flag
session['password_reset_link'] = password_reset_link user_ref = db.collection("users").document(user.uid)
session['reset_user_email'] = user.email user_ref.set({
"password_reset_required": True
},merge=True)
flash(f"User now has a temporary password {temp_password}.", "success")
fb_auth.update_user(uid, password=temp_password)
# Redirect back to the admin users table # Redirect back to the admin users table
return redirect(url_for('admin_users')) return redirect(url_for('admin_users'))
@@ -126,11 +137,14 @@ def register_admin_routes(app):
# Update user in Firestore # Update user in Firestore
user_ref = db.collection("users").document(target_uid) user_ref = db.collection("users").document(target_uid)
user_ref.update({ # Only update fields that can be changed, excluding is_admin
update_data = {
"enabled": data.get("enabled", False), "enabled": data.get("enabled", False),
"is_admin": data.get("is_admin", False), "case_email": data.get("case_email", ""),
"case_email": data.get("case_email", "") "case_domain_email": data.get("case_domain_email", "")
}) }
# Never allow changing is_admin field during updates - admin status can only be set during creation
user_ref.update(update_data)
return jsonify({"success": True}) return jsonify({"success": True})
@@ -147,7 +161,7 @@ def register_admin_routes(app):
@app.route("/admin/users/create", methods=["POST"]) @app.route("/admin/users/create", methods=["POST"])
@admin_required @admin_required
def create_user(): def create_user():
"""Create a new user""" """Create a new user with temporary password"""
try: try:
# Get form data # Get form data
user_email = request.form.get("user_email") user_email = request.form.get("user_email")
@@ -158,22 +172,37 @@ def register_admin_routes(app):
if "@" not in user_email: if "@" not in user_email:
abort(400, "Invalid email format") abort(400, "Invalid email format")
# Create user in Firebase Authentication # Generate temporary password (random word + 3 digits)
words = ["sun", "moon", "star", "cloud", "rain", "wind", "fire", "water", "snow", "stone",
"tree", "leaf", "flower", "bird", "wolf", "tiger", "bear", "fish", "dragon",
"magic", "quest", "light", "dark", "gold", "silver", "ruby", "pearl", "diamond"]
random_word = random.choice(words)
random_digits = ''.join(random.choices(string.digits, k=3))
temp_password = f"{random_word}{random_digits}"
# Create user in Firebase Authentication with temporary password
user_record = fb_auth.create_user( user_record = fb_auth.create_user(
email=user_email, email=user_email,
email_verified=False, email_verified=False,
disabled=not request.form.get("enabled", False) disabled=not request.form.get("enabled", False),
password=temp_password
) )
# Create user profile in Firestore # Create user profile in Firestore with password reset required flag
user_ref = db.collection("users").document(user_record.uid) user_ref = db.collection("users").document(user_record.uid)
user_ref.set({ user_ref.set({
"user_email": user_email, "user_email": user_email,
"case_email": request.form.get("case_email", ""), "case_email": request.form.get("case_email", ""),
"case_domain_email": request.form.get("case_domain_email", ""),
"enabled": bool(request.form.get("enabled", False)), "enabled": bool(request.form.get("enabled", False)),
"is_admin": bool(request.form.get("is_admin", False)) "is_admin": bool(request.form.get("is_admin", False)),
"password_reset_required": True
}) })
# Display success message with temporary password
flash(f"User created successfully. Temporary password: {temp_password}", "success")
print(f"[INFO] Created user {user_email} with temp password: {temp_password}")
# Redirect to admin users page # Redirect to admin users page
return redirect(url_for("admin_users")) return redirect(url_for("admin_users"))
@@ -183,3 +212,73 @@ def register_admin_routes(app):
except Exception as e: except Exception as e:
print(f"[ERR] Failed to create user: {e}") print(f"[ERR] Failed to create user: {e}")
abort(500, "Failed to create user") abort(500, "Failed to create user")
@app.route("/admin/users/<uid>/become-user", methods=["POST"])
@admin_required
def become_user(uid):
"""Allow admin to impersonate another user by replacing session UID"""
try:
# Verify the target user exists
target_user_doc = db.collection("users").document(uid).get()
if not target_user_doc.exists:
abort(404, "User not found")
# Store original admin UID and set impersonation flags
session['original_uid'] = session.get('uid')
session['impersonating'] = True
# Replace session UID with target user's UID
session['uid'] = uid
print(f"[INFO] Admin {session['original_uid']} is now impersonating user {uid}")
# Redirect to dashboard
return redirect(url_for('dashboard'))
except Exception as e:
print(f"[ERR] Failed to become user {uid}: {e}")
abort(500, "Failed to impersonate user")
@app.route("/admin/users/revert")
def revert_user():
"""Revert back to the original admin session.
Only accessible if the original session owner (before impersonation) was an admin.
This check must happen before @admin_required decorator to verify the original UID,
not the currently impersonated user's UID.
"""
try:
# Check if user is currently impersonating
if 'original_uid' not in session:
# Not impersonating - check if current user is admin
uid = session.get('uid')
if not uid:
return redirect(url_for('login'))
profile = get_user_profile(uid)
if profile.get('is_admin'):
return redirect(url_for('admin_users'))
# Not an admin, deny access
abort(403, "Access denied. Admin privileges required.")
# Verify the original session owner was an admin (not the impersonated user)
original_uid = session.get('original_uid')
if not original_uid:
abort(403, "Access denied. Invalid session state.")
original_profile = get_user_profile(original_uid)
if not original_profile.get('is_admin'):
abort(403, "Access denied. Only admins can revert from impersonation.")
# Restore original admin UID
session['uid'] = original_uid
session.pop('impersonating', None)
session.pop('original_uid', None)
print(f"[INFO] Reverted from impersonation back to admin")
# Redirect to admin users page
return redirect(url_for('admin_users'))
except Exception as e:
print(f"[ERR] Failed to revert user: {e}")
abort(500, "Failed to revert session")

219
app.py
View File

@@ -29,6 +29,82 @@ def login_required(view):
return view(*args, **kwargs) return view(*args, **kwargs)
return wrapped return wrapped
def projects_for(profile, case_email_match, per_page, offset):
"""
Filter projects based on user profile and case_email query string argument.
Args:
profile (dict): User profile containing 'enabled', 'is_admin', 'case_email', and 'case_domain_email' fields
case_email_match (str): Case email from query string argument, or None
Returns:
list: List of project dictionaries that match the filtering criteria
"""
is_admin = profile.get("is_admin", False)
if not profile.get("enabled"):
return ([], 0)
# Query Firestore for projects where case_email is in viewing_emails array
try:
cnt = 0
if is_admin:
if case_email_match:
case_email_match_lower = case_email_match.lower().strip()
# Check if case_email_match is a valid email address (contains @)
if '@' in case_email_match_lower and not case_email_match_lower.startswith('@'):
# If it's a complete email address, filter by exact match in viewing_emails
projects_ref = db.collection("projects").where("viewing_emails", "array_contains", case_email_match_lower).where("is_archived", "==", False)
cnt = int(projects_ref.count().get()[0][0].value)
projects = []
for doc in projects_ref.order_by("matter_description").limit(per_page).offset(offset).stream():
projects.append(doc.to_dict())
return (projects, cnt)
else:
# If no @ sign, treat as domain search
# Also handle cases like "@gmail.com" by extracting the domain
domain_search = case_email_match_lower
if domain_search.startswith('@'):
domain_search = domain_search[1:] # Remove the @ sign
# Filter by domain match in viewing_emails
projects_ref = db.collection("projects").where("viewing_domains", "array_contains", domain_search).where("is_archived", "==", False)
print("HERE domain", domain_search)
cnt = int(projects_ref.count().get()[0][0].value)
projects = []
for doc in projects_ref.order_by("matter_description").limit(per_page).offset(offset).stream():
projects.append(doc.to_dict())
return (projects, cnt)
else:
projects_ref = db.collection("projects").where("is_archived", "==", False)
else:
# For non-admin users, check if they have domain email or specific case email
case_domain_email = profile.get("case_domain_email", "")
case_email = profile.get("case_email", "")
if case_domain_email:
# Use exact match on viewing_domains field
domain_lower = case_domain_email.lower()
projects_ref = db.collection("projects").where("viewing_domains", "array_contains", domain_lower).where("is_archived", "==", False)
elif case_email:
# Use the original logic for specific case email match
projects_ref = db.collection("projects").where("viewing_emails", "array_contains", case_email.lower()).where("is_archived", "==", False)
else:
return ([], 0)
cnt = int(projects_ref.count().get()[0][0].value)
projects = []
for doc in projects_ref.order_by("matter_description").limit(per_page).offset(offset).stream():
projects.append(doc.to_dict())
return (projects, cnt)
except Exception as e:
print(f"[ERROR] Failed to query projects: {e}")
return ([], 0)
@app.context_processor @app.context_processor
@@ -63,7 +139,7 @@ def index():
if not uid: if not uid:
return redirect(url_for("login")) return redirect(url_for("login"))
profile = get_user_profile(uid) profile = get_user_profile(uid)
if profile.get("enabled") and profile.get("case_email"): if profile.get("enabled") and (profile.get("case_email") or profile.get("case_domain_email")):
return redirect(url_for("dashboard")) return redirect(url_for("dashboard"))
return redirect(url_for("welcome")) return redirect(url_for("welcome"))
@@ -94,6 +170,12 @@ def session_login():
# Optional: short session # Optional: short session
session["expires_at"] = (datetime.utcnow() + timedelta(hours=8)).isoformat() session["expires_at"] = (datetime.utcnow() + timedelta(hours=8)).isoformat()
# Check if user needs password reset
user_profile = get_user_profile(uid)
if user_profile.get("password_reset_required"):
return jsonify({"requires_password_reset": True})
print(f"logged in as {uid} - user_profile.email")
return jsonify({"ok": True}) return jsonify({"ok": True})
except Exception as e: except Exception as e:
print("[ERR] session_login:", e) print("[ERR] session_login:", e)
@@ -106,6 +188,52 @@ def logout():
return redirect(url_for("login")) return redirect(url_for("login"))
@app.route("/require-password-reset")
@login_required
def require_password_reset():
"""Show password reset page for users who need to reset their password"""
return render_template("require_password_reset.html")
@app.route("/reset-password-submit", methods=["POST"])
@login_required
def reset_password_submit():
"""Handle password reset form submission"""
uid = session.get("uid")
profile = get_user_profile(uid)
new_password = request.form.get("new_password")
confirm_password = request.form.get("confirm_password")
# Validate passwords match
if new_password != confirm_password:
flash("Passwords do not match", "error")
return redirect(url_for("require_password_reset"))
# Validate password length
if len(new_password) < 6:
flash("Password must be at least 6 characters", "error")
return redirect(url_for("require_password_reset"))
# Update password in Firebase Auth
try:
fb_auth.update_user(uid, password=new_password)
print(db.collection("users").document(uid))
# Clear the password reset required flag in Firestore
db.collection("users").document(uid).set({"password_reset_required": False},merge=True)
print(db.collection("users").document(uid))
print(f"[INFO] Password reset successful for user {uid}")
except Exception as e:
print(f"[ERR] Failed to reset password for {uid}: {e}")
flash("Failed to reset password. Please try again.", "error")
return redirect(url_for("require_password_reset"))
# Allow user to login now
return redirect(url_for("login"))
@app.route("/welcome") @app.route("/welcome")
@login_required @login_required
def welcome(): def welcome():
@@ -127,72 +255,33 @@ def dashboard(page=1):
return redirect(url_for("welcome")) return redirect(url_for("welcome"))
is_admin = profile.get("is_admin") is_admin = profile.get("is_admin")
case_email = None
if not is_admin:
case_email = profile.get("case_email")
if not case_email:
return redirect(url_for("welcome"))
if is_admin and request.args.get('case_email'):
case_email = request.args.get('case_email').lower()
# Validate email format
if '@' not in case_email:
return abort(400, "Invalid email format")
# Pagination settings # Pagination settings
per_page = int(request.args.get('per_page', 25)) per_page = int(request.args.get('per_page', 25))
offset = (page - 1) * per_page offset = (page - 1) * per_page
query = None case_email_match = None
if is_admin and request.args.get('case_email'):
# Get total count efficiently using a count aggregation query case_email_match = request.args.get('case_email')
try: if not is_admin and (not profile.get('case_email') and not profile.get('case_domain_email')):
# Firestore doesn't have a direct count() method, so we need to count documents return redirect(url_for("welcome"))
import time paginated_rows, total_projects = projects_for(profile, case_email_match, per_page, offset)
start_time = time.time()
projects_ref = db.collection("projects")
# Filter projects where case_email is in viewing_emails array
if case_email:
query = projects_ref.where("viewing_emails", "array_contains", case_email.lower())
else:
query = projects_ref
total_projects = int(query.count().get()[0][0].value)
end_time = time.time()
print(f"Filtered projects count: {total_projects} (took {end_time - start_time:.2f}s)")
except Exception as e:
print(f"[WARN] Failed to get filtered count: {e}")
total_projects = 0
# Calculate pagination # Calculate pagination
total_pages = (total_projects + per_page - 1) // per_page # Ceiling division total_pages = (total_projects + per_page - 1) // per_page # Ceiling division
# Read only the current page from Firestore using limit() and offset() # Read only the current page from Firestore using limit() and offset()
import time import time
start_time = time.time() print(f"Retrieved {len(paginated_rows)} projects from Firestore")
# Filter projects where case_email is in viewing_emails array
if case_email:
projects_ref = db.collection("projects").where("viewing_emails", "array_contains", case_email.lower()).order_by("matter_description").limit(per_page).offset(offset)
else:
projects_ref = db.collection("projects").order_by("matter_description").limit(per_page).offset(offset)
docs = projects_ref.stream()
paginated_rows = []
for doc in docs:
paginated_rows.append(doc.to_dict())
end_time = time.time()
print(f"Retrieved {len(paginated_rows)} projects from Firestore (page {page} of {total_pages}) in {end_time - start_time:.2f}s")
from pprint import pprint
pprint([p['property_contacts'] for p in paginated_rows if p['property_contacts'].get('propertyManager1', None)])
pprint([p['ProjectId'] for p in paginated_rows ])
# Render table with pagination data # Render table with pagination data
return render_template("dashboard.html", return render_template("dashboard.html",
rows=paginated_rows, rows=paginated_rows,
case_email=case_email, case_email=case_email_match,
current_page=page, current_page=page,
total_pages=total_pages, total_pages=total_pages,
total_projects=total_projects, total_projects=total_projects,
per_page=per_page) per_page=per_page,
is_admin=is_admin)
@app.route("/dashboard/export_xls") @app.route("/dashboard/export_xls")
@@ -206,33 +295,15 @@ def dashboard_export_xls():
is_admin = profile.get("is_admin") is_admin = profile.get("is_admin")
case_email = None case_email = None
if not is_admin: if not is_admin and (not profile.get('case_email') and not profile.get('case_domain_email')):
case_email = profile.get("case_email")
if not case_email:
return redirect(url_for("welcome")) return redirect(url_for("welcome"))
if is_admin and request.args.get('case_email'):
case_email = request.args.get('case_email').lower()
# Validate email format
if '@' not in case_email:
return abort(400, "Invalid email format")
# Get all projects without pagination # Get all projects without pagination
try: try:
projects_ref = db.collection("projects") all_rows, cnt = projects_for(profile, case_email, 10000, 0)
# Filter projects where case_email is in viewing_emails array # Filter projects where case_email is in viewing_emails array
if case_email:
projects_ref = projects_ref.where("viewing_emails", "array_contains", case_email.lower())
# Order by matter_description to maintain consistent ordering # Order by matter_description to maintain consistent ordering
projects_ref = projects_ref.order_by("matter_description") print(f"Retrieved {cnt} projects from Firestore for XLS export")
docs = projects_ref.stream()
all_rows = []
for doc in docs:
all_rows.append(doc.to_dict())
print(f"Retrieved {len(all_rows)} projects from Firestore for XLS export")
# Create workbook and worksheet # Create workbook and worksheet
wb = Workbook() wb = Workbook()
@@ -362,7 +433,7 @@ def dashboard_export_xls():
elif header == 'Notice Expir. Date': elif header == 'Notice Expir. Date':
field_name = 'notice_expiration_date' field_name = 'notice_expiration_date'
elif header == 'Date Case Filed': elif header == 'Date Case Filed':
field_name = 'case_field_date' field_name = 'case_filed_date'
elif header == 'Daily Rent Damages': elif header == 'Daily Rent Damages':
field_name = 'daily_rent_damages' field_name = 'daily_rent_damages'
elif header == 'Default Date': elif header == 'Default Date':

61
backfill_is_archived.py Normal file
View File

@@ -0,0 +1,61 @@
#!/usr/bin/env python3
"""
One-off script to backfill is_archived field on all projects in Firestore.
This sets is_archived = True for projects where phase_name == "Archived",
and is_archived = False for all other projects.
Usage:
python backfill_is_archived.py
"""
import os
from firebase_admin import credentials, initialize_app, firestore
# Path to your staging service account JSON
CREDENTIALS_PATH = "./rothbard-staging2-12345-firebase-adminsdk-fbsvc-7f95268383.json"
def main():
# Initialize Firebase Admin with staging credentials
cred = credentials.Certificate(CREDENTIALS_PATH)
app = initialize_app(cred, name='backfill-is-archived')
db = firestore.client(app=app)
projects_ref = db.collection("projects")
docs = list(projects_ref.stream())
total = len(docs)
archived_count = 0
updated_count = 0
batch_size = 500
print(f"Found {total} projects. Processing in batches of {batch_size}...")
for i in range(0, total, batch_size):
batch = db.batch()
batch_docs = docs[i:i + batch_size]
for doc in batch_docs:
data = doc.to_dict()
phase_name = data.get("phase_name", "")
is_archived = (phase_name == "Archived")
if is_archived:
archived_count += 1
# Only update if the field is missing or different
if data.get("is_archived") != is_archived:
ref = projects_ref.document(doc.id)
batch.update(ref, {"is_archived": is_archived})
updated_count += 1
batch.commit()
print(f" Committed batch {i//batch_size + 1}/{(total + batch_size - 1)//batch_size}")
print(f"\nDone!")
print(f" Total projects: {total}")
print(f" Projects with phase_name == 'Archived': {archived_count}")
print(f" Documents updated: {updated_count}")
if __name__ == "__main__":
main()

View File

@@ -11,14 +11,21 @@ FV_USER_ID = os.environ.get("FILEVINE_USER_ID")
class FilevineClient: class FilevineClient:
def __init__(self, bearer_token: str = None): def __init__(self, bearer_token: str = None):
self.bearer_token = bearer_token
self.base_url = "https://api.filevineapp.com/fv-app/v2" self.base_url = "https://api.filevineapp.com/fv-app/v2"
self.bearer_token = bearer_token
self.headers = { self.headers = {
"Accept": "application/json", "Accept": "application/json",
"Authorization": f"Bearer {self.bearer_token}", "Authorization": f"Bearer {self.bearer_token}",
"x-fv-orgid": str(FV_ORG_ID), "x-fv-orgid": str(FV_ORG_ID),
"x-fv-userid": str(FV_USER_ID), "x-fv-userid": str(FV_USER_ID),
} }
self.session = requests.Session()
self.session.headers.update({
"Accept": "application/json",
"x-fv-orgid": str(FV_ORG_ID),
"x-fv-userid": str(FV_USER_ID),
})
self.get_bearer_token()
def get_bearer_token(self) -> str: def get_bearer_token(self) -> str:
"""Get a new bearer token using Filevine credentials""" """Get a new bearer token using Filevine credentials"""
@@ -32,13 +39,12 @@ class FilevineClient:
} }
headers = {"Accept": "application/json"} headers = {"Accept": "application/json"}
print(data) resp = self.session.post(url, data=data, headers=headers, timeout=30)
resp = requests.post(url, data=data, headers=headers, timeout=30)
resp.raise_for_status() resp.raise_for_status()
js = resp.json() js = resp.json()
token = js.get("access_token") token = js.get("access_token")
print(f"Got bearer js", js)
self.bearer_token = token self.bearer_token = token
self.session.headers["Authorization"] = f"Bearer {token}"
self.headers["Authorization"] = f"Bearer {token}" self.headers["Authorization"] = f"Bearer {token}"
return token return token
@@ -58,7 +64,6 @@ class FilevineClient:
while True: while True:
cnt = len(results) cnt = len(results)
print(f"list try {tries}, starting at {offset}, previous count {last_count}, currently at {cnt}")
tries += 1 tries += 1
url = base url = base
params = {} params = {}
@@ -70,7 +75,7 @@ class FilevineClient:
if latest_activity_since: if latest_activity_since:
params["latestActivitySince"] = latest_activity_since params["latestActivitySince"] = latest_activity_since
r = requests.get(url, headers=self.headers, params=params, timeout=30) r = self.session.get(url, headers=self.headers, params=params, timeout=30)
r.raise_for_status() r.raise_for_status()
page = r.json() page = r.json()
items = page.get("items", []) items = page.get("items", [])
@@ -87,35 +92,35 @@ class FilevineClient:
def fetch_project_detail(self, project_id_native: int) -> Dict[str, Any]: def fetch_project_detail(self, project_id_native: int) -> Dict[str, Any]:
"""Fetch detailed information for a specific project""" """Fetch detailed information for a specific project"""
url = f"{self.base_url}/Projects/{project_id_native}" url = f"{self.base_url}/Projects/{project_id_native}"
r = requests.get(url, headers=self.headers, timeout=30) r = self.session.get(url, headers=self.headers, timeout=30)
r.raise_for_status() r.raise_for_status()
return r.json() return r.json()
def fetch_project_team(self, project_id_native: int) -> List[Dict[str, Any]]: def fetch_project_team(self, project_id_native: int) -> List[Dict[str, Any]]:
"""Fetch team members for a specific project""" """Fetch team members for a specific project"""
url = f"{self.base_url}/Projects/{project_id_native}/team?limit=1000" url = f"{self.base_url}/Projects/{project_id_native}/team?limit=1000"
r = requests.get(url, headers=self.headers, timeout=30) r = self.session.get(url, headers=self.headers, timeout=30)
r.raise_for_status() r.raise_for_status()
return r.json().get('items') or [] return r.json().get('items') or []
def fetch_project_tasks(self, project_id_native: int) -> Dict[str, Any]: def fetch_project_tasks(self, project_id_native: int) -> Dict[str, Any]:
"""Fetch tasks for a specific project""" """Fetch tasks for a specific project"""
url = f"{self.base_url}/Projects/{project_id_native}/tasks" url = f"{self.base_url}/Projects/{project_id_native}/tasks"
r = requests.get(url, headers=self.headers, timeout=30) r = self.session.get(url, headers=self.headers, timeout=30)
r.raise_for_status() r.raise_for_status()
return r.json() return r.json()
def fetch_client(self, client_id_native: int) -> Dict[str, Any]: def fetch_client(self, client_id_native: int) -> Dict[str, Any]:
"""Fetch client information by client ID""" """Fetch client information by client ID"""
url = f"{self.base_url}/contacts/{client_id_native}" url = f"{self.base_url}/contacts/{client_id_native}"
r = requests.get(url, headers=self.headers, timeout=30) r = self.session.get(url, headers=self.headers, timeout=30)
r.raise_for_status() r.raise_for_status()
return r.json() return r.json()
def fetch_contacts(self, project_id_native: int) -> Optional[List[Dict[str, Any]]]: def fetch_contacts(self, project_id_native: int) -> Optional[List[Dict[str, Any]]]:
"""Fetch contacts for a specific project""" """Fetch contacts for a specific project"""
url = f"{self.base_url}/projects/{project_id_native}/contacts" url = f"{self.base_url}/projects/{project_id_native}/contacts"
r = requests.get(url, headers=self.headers, timeout=30) r = self.session.get(url, headers=self.headers, timeout=30)
r.raise_for_status() r.raise_for_status()
return r.json().get("items") return r.json().get("items")
@@ -123,20 +128,20 @@ class FilevineClient:
"""Fetch a specific form for a project""" """Fetch a specific form for a project"""
try: try:
url = f"{self.base_url}/Projects/{project_id_native}/Forms/{form}" url = f"{self.base_url}/Projects/{project_id_native}/Forms/{form}"
r = requests.get(url, headers=self.headers, timeout=30) r = self.session.get(url, headers=self.headers, timeout=30)
r.raise_for_status() r.raise_for_status()
return r.json() return r.json()
except Exception as e: except Exception as e:
print(e) print(f"[WARN] Failed to fetch form '{form}' for project {project_id_native}: {e}")
return {} return {}
def fetch_collection(self, project_id_native: int, collection: str) -> List[Dict[str, Any]]: def fetch_collection(self, project_id_native: int, collection: str) -> List[Dict[str, Any]]:
"""Fetch a collection for a project""" """Fetch a collection for a project"""
try: try:
url = f"{self.base_url}/Projects/{project_id_native}/Collections/{collection}" url = f"{self.base_url}/Projects/{project_id_native}/Collections/{collection}"
r = requests.get(url, headers=self.headers, timeout=30) r = self.session.get(url, headers=self.headers, timeout=30)
r.raise_for_status() r.raise_for_status()
return [x.get('dataObject') for x in r.json().get("items")] return [x.get('dataObject') for x in r.json().get("items")]
except Exception as e: except Exception as e:
print(e) print(f"[WARN] Failed to fetch collection '{collection}' for project {project_id_native}: {e}")
return {} return {}

View File

@@ -31,6 +31,7 @@ class ProjectModel:
pending_tasks: List[Dict[str, Any]] = None, pending_tasks: List[Dict[str, Any]] = None,
notice_service_date: str = "", notice_service_date: str = "",
notice_expiration_date: str = "", notice_expiration_date: str = "",
case_filed_date: str = "",
case_field_date: str = "", case_field_date: str = "",
daily_rent_damages: str = "", daily_rent_damages: str = "",
default_date: str = "", default_date: str = "",
@@ -69,7 +70,9 @@ class ProjectModel:
project_name: str = "", project_name: str = "",
project_url: str = "", project_url: str = "",
property_contacts: Dict[str, Any] = None, property_contacts: Dict[str, Any] = None,
viewing_emails: List[str] = None viewing_emails: List[str] = None,
viewing_domains: List[str] = None,
last_synced_at: str = ""
): ):
self.client = client self.client = client
@@ -88,6 +91,7 @@ class ProjectModel:
self.pending_tasks = pending_tasks or [] self.pending_tasks = pending_tasks or []
self.notice_service_date = notice_service_date self.notice_service_date = notice_service_date
self.notice_expiration_date = notice_expiration_date self.notice_expiration_date = notice_expiration_date
self.case_filed_date = case_filed_date or case_field_date
self.case_field_date = case_field_date self.case_field_date = case_field_date
self.daily_rent_damages = daily_rent_damages self.daily_rent_damages = daily_rent_damages
self.default_date = default_date self.default_date = default_date
@@ -127,6 +131,8 @@ class ProjectModel:
self.project_url = project_url self.project_url = project_url
self.property_contacts = property_contacts or {} self.property_contacts = property_contacts or {}
self.viewing_emails = viewing_emails or [] self.viewing_emails = viewing_emails or []
self.viewing_domains = viewing_domains or []
self.last_synced_at = last_synced_at
def to_dict(self) -> Dict[str, Any]: def to_dict(self) -> Dict[str, Any]:
"""Convert the ProjectModel to a dictionary for Firestore storage.""" """Convert the ProjectModel to a dictionary for Firestore storage."""
@@ -147,6 +153,7 @@ class ProjectModel:
"pending_tasks": self.pending_tasks, "pending_tasks": self.pending_tasks,
"notice_service_date": self.notice_service_date, "notice_service_date": self.notice_service_date,
"notice_expiration_date": self.notice_expiration_date, "notice_expiration_date": self.notice_expiration_date,
"case_filed_date": self.case_filed_date,
"case_field_date": self.case_field_date, "case_field_date": self.case_field_date,
"daily_rent_damages": self.daily_rent_damages, "daily_rent_damages": self.daily_rent_damages,
"default_date": self.default_date, "default_date": self.default_date,
@@ -185,7 +192,9 @@ class ProjectModel:
"ProjectName": self.project_name, "ProjectName": self.project_name,
"ProjectUrl": self.project_url, "ProjectUrl": self.project_url,
"property_contacts": self.property_contacts, "property_contacts": self.property_contacts,
"viewing_emails": self.viewing_emails "viewing_emails": self.viewing_emails,
"viewing_domains": self.viewing_domains,
"last_synced_at": self.last_synced_at
} }
@classmethod @classmethod
@@ -208,6 +217,7 @@ class ProjectModel:
pending_tasks=data.get("pending_tasks", []), pending_tasks=data.get("pending_tasks", []),
notice_service_date=data.get("notice_service_date", ""), notice_service_date=data.get("notice_service_date", ""),
notice_expiration_date=data.get("notice_expiration_date", ""), notice_expiration_date=data.get("notice_expiration_date", ""),
case_filed_date=data.get("case_filed_date", ""),
case_field_date=data.get("case_field_date", ""), case_field_date=data.get("case_field_date", ""),
daily_rent_damages=data.get("daily_rent_damages", ""), daily_rent_damages=data.get("daily_rent_damages", ""),
default_date=data.get("default_date", ""), default_date=data.get("default_date", ""),
@@ -246,5 +256,7 @@ class ProjectModel:
project_name=data.get("ProjectName", ""), project_name=data.get("ProjectName", ""),
project_url=data.get("ProjectUrl", ""), project_url=data.get("ProjectUrl", ""),
property_contacts=data.get("property_contacts", {}), property_contacts=data.get("property_contacts", {}),
viewing_emails=data.get("viewing_emails", []) viewing_emails=data.get("viewing_emails", []),
viewing_domains=data.get("viewing_domains", []),
last_synced_at=data.get("last_synced_at", "")
) )

71
package-lock.json generated Normal file
View File

@@ -0,0 +1,71 @@
{
"name": "rothbard",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"dependencies": {
"@playwright/test": "1.58.0"
}
},
"node_modules/@playwright/test": {
"version": "1.58.0",
"resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.58.0.tgz",
"integrity": "sha512-fWza+Lpbj6SkQKCrU6si4iu+fD2dD3gxNHFhUPxsfXBPhnv3rRSQVd0NtBUT9Z/RhF/boCBcuUaMUSTRTopjZg==",
"license": "Apache-2.0",
"dependencies": {
"playwright": "1.58.0"
},
"bin": {
"playwright": "cli.js"
},
"engines": {
"node": ">=18"
}
},
"node_modules/fsevents": {
"version": "2.3.2",
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
"integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
"hasInstallScript": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
}
},
"node_modules/playwright": {
"version": "1.58.0",
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.58.0.tgz",
"integrity": "sha512-2SVA0sbPktiIY/MCOPX8e86ehA/e+tDNq+e5Y8qjKYti2Z/JG7xnronT/TXTIkKbYGWlCbuucZ6dziEgkoEjQQ==",
"license": "Apache-2.0",
"dependencies": {
"playwright-core": "1.58.0"
},
"bin": {
"playwright": "cli.js"
},
"engines": {
"node": ">=18"
},
"optionalDependencies": {
"fsevents": "2.3.2"
}
},
"node_modules/playwright-core": {
"version": "1.58.0",
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.58.0.tgz",
"integrity": "sha512-aaoB1RWrdNi3//rOeKuMiS65UCcgOVljU46At6eFcOFPFHWtd2weHRRow6z/n+Lec0Lvu0k9ZPKJSjPugikirw==",
"license": "Apache-2.0",
"bin": {
"playwright-core": "cli.js"
},
"engines": {
"node": ">=18"
}
}
}
}

5
package.json Normal file
View File

@@ -0,0 +1,5 @@
{
"dependencies": {
"@playwright/test": "1.58.0"
}
}

209
query_projects.py Executable file
View File

@@ -0,0 +1,209 @@
#!/usr/bin/env python3
"""
CLI script to query Firebase for projects associated with a user email.
Usage:
python scripts/query_projects.py --email user@example.com
python scripts/query_projects.py --email user@example.com --limit 10
python scripts/query_projects.py --email @gmail.com # Domain search
"""
import argparse
import json
import sys
from typing import List, Dict, Any
# Add parent directory to path for imports
sys.path.insert(0, "..")
from dotenv import load_dotenv
load_dotenv()
from firebase_init import db
from firebase_admin import auth as fb_auth
from utils import get_user_profile
def get_uid_from_email(email: str) -> str | None:
"""Get Firebase Auth UID from user email."""
try:
user = fb_auth.get_user_by_email(email)
return user.uid
except fb_auth.UserNotFound:
return None
except Exception as e:
print(f"Error looking up user: {e}")
return None
def query_projects_for_user(
uid: str,
case_email: str | None = None,
case_domain_email: str | None = None,
limit: int = 100
) -> tuple[List[Dict[str, Any]], int]:
"""
Query Firestore for projects associated with a user.
Args:
uid: Firebase user UID
case_email: Specific case email to filter by (optional)
case_domain_email: Domain to filter by (optional)
limit: Maximum number of projects to return
Returns:
Tuple of (projects list, total count)
"""
profile = get_user_profile(uid)
if not profile.get("enabled"):
print(f"Warning: User is not enabled")
# Determine which filter to use
filter_email = case_email or profile.get("case_email")
filter_domain = case_domain_email or profile.get("case_domain_email")
if not filter_email and not filter_domain:
print("Error: No case_email or case_domain_email configured for this user")
return ([], 0)
try:
if filter_domain:
# Domain-based search
domain_lower = filter_domain.lower()
projects_ref = db.collection("projects").where(
"viewing_domains", "array_contains", domain_lower
).where("is_archived", "==", False)
else:
# Email-based search
email_lower = filter_email.lower()
projects_ref = db.collection("projects").where(
"viewing_emails", "array_contains", email_lower
).where("is_archived", "==", False)
# Get total count
total_count = int(projects_ref.count().get()[0][0].value)
# Get paginated results
projects = []
for doc in projects_ref.order_by("matter_description").limit(limit).stream():
projects.append(doc.to_dict())
return (projects, total_count)
except Exception as e:
print(f"Error querying projects: {e}")
return ([], 0)
def format_project(project: Dict[str, Any]) -> str:
"""Format a project dictionary for display."""
lines = [
f"Matter #: {project.get('number', 'N/A')}",
f"Description: {project.get('matter_description', 'N/A')}",
f"Client: {project.get('client', 'N/A')}",
f"Defendant 1: {project.get('defendant_1', 'N/A')}",
f"Case #: {project.get('case_number', 'N/A')}",
f"Premises: {project.get('premises_address', 'N/A')}, {project.get('premises_city', 'N/A')}",
f"Assigned Attorney: {project.get('responsible_attorney', 'N/A')}",
f"Primary Contact: {project.get('staff_person', 'N/A')}",
f"Matter Stage: {project.get('phase_name', 'N/A')}",
f"Documents: {project.get('documents_url', 'N/A')}",
]
return "\n".join(lines)
def main():
parser = argparse.ArgumentParser(
description="Query Firebase for projects associated with a user email"
)
parser.add_argument(
"--email", "-e",
required=True,
help="User email address (e.g., user@example.com or @domain.com for domain search)"
)
parser.add_argument(
"--limit", "-l",
type=int,
default=100,
help="Maximum number of projects to return (default: 100)"
)
parser.add_argument(
"--json", "-j",
action="store_true",
help="Output results as JSON"
)
parser.add_argument(
"--verbose", "-v",
action="store_true",
help="Show detailed project information"
)
parser.add_argument(
"--case-email",
help="Override case email filter (use user's case_email by default)"
)
parser.add_argument(
"--case-domain",
help="Override case domain filter (use user's case_domain_email by default)"
)
args = parser.parse_args()
# Get UID from email
uid = get_uid_from_email(args.email)
if not uid:
print(f"Error: User not found with email '{args.email}'")
sys.exit(1)
# Get user profile
profile = get_user_profile(uid)
print(f"\nUser: {profile.get('user_email', 'N/A')}")
print(f"UID: {uid}")
print(f"Enabled: {profile.get('enabled', False)}")
print(f"Is Admin: {profile.get('is_admin', False)}")
print(f"Case Email: {profile.get('case_email', 'N/A')}")
print(f"Case Domain Email: {profile.get('case_domain_email', 'N/A')}")
# Query projects
projects, total_count = query_projects_for_user(
uid=uid,
case_email=args.case_email,
case_domain_email=args.case_domain,
limit=args.limit
)
print(f"\nFound {total_count} projects (showing {min(len(projects), args.limit)})")
if args.json:
# Output as JSON
result = {
"user": {
"email": profile.get("user_email"),
"uid": uid,
"enabled": profile.get("enabled"),
"is_admin": profile.get("is_admin"),
"case_email": profile.get("case_email"),
"case_domain_email": profile.get("case_domain_email")
},
"total_count": total_count,
"projects": projects
}
print(json.dumps(result, indent=2, default=str))
else:
# Human-readable output
for i, project in enumerate(projects, 1):
print(f"\n{'='*60}")
print(f"Project {i}:")
print('='*60)
if args.verbose:
print(format_project(project))
else:
print(f" Matter #: {project.get('number', 'N/A')}")
print(f" Description: {project.get('matter_description', 'N/A')}")
print(f" Client: {project.get('client', 'N/A')}")
print(f" Case #: {project.get('case_number', 'N/A')}")
if __name__ == "__main__":
main()

222
sync.py
View File

@@ -9,7 +9,7 @@ import os
import concurrent.futures import concurrent.futures
import threading import threading
from typing import List, Dict, Any, Optional from typing import List, Dict, Any, Optional
from datetime import datetime from datetime import datetime, timedelta
import pytz import pytz
from dotenv import load_dotenv from dotenv import load_dotenv
load_dotenv() load_dotenv()
@@ -17,6 +17,34 @@ load_dotenv()
# Add the current directory to the Python path so we can import app and models # Add the current directory to the Python path so we can import app and models
sys.path.append(os.path.dirname(os.path.abspath(__file__))) sys.path.append(os.path.dirname(os.path.abspath(__file__)))
def batch_write_to_firestore(db, collection_name: str, documents: List[tuple], batch_size: int = 500):
"""Write documents to Firestore in batches from the main thread.
Args:
db: Firestore client
collection_name: Name of the collection
documents: List of (doc_id, data) tuples
batch_size: Number of documents per batch
"""
collection = db.collection(collection_name)
total = len(documents)
written = 0
for i in range(0, total, batch_size):
batch = documents[i:i + batch_size]
try:
write_batch = db.batch()
for doc_id, data in batch:
ref = collection.document(str(doc_id))
write_batch.set(ref, data)
write_batch.commit()
written += len(batch)
print(f"[BATCH] Wrote {written}/{total} documents")
except Exception as e:
print(f"[ERROR] Batch write failed: {e}")
print(f"[BATCH] Completed writing {written} documents to Firestore")
def convert_to_pacific_time(date_str): def convert_to_pacific_time(date_str):
"""Convert UTC date string to Pacific Time and format as YYYY-MM-DD. """Convert UTC date string to Pacific Time and format as YYYY-MM-DD.
@@ -40,11 +68,33 @@ def convert_to_pacific_time(date_str):
pacific_time = utc_time.astimezone(pytz.timezone('America/Los_Angeles')) pacific_time = utc_time.astimezone(pytz.timezone('America/Los_Angeles'))
# Format as YYYY-MM-DD # Format as YYYY-MM-DD
return pacific_time.strftime('%m/%d/%Y') return pacific_time.strftime('%Y-%m-%d')
except (ValueError, AttributeError) as e: except (ValueError, AttributeError) as e:
print(f"[WARN] Date conversion failed for '{date_str}': {e}") print(f"[WARN] Date conversion failed for '{date_str}': {e}")
return '' return ''
def extract_domains_from_emails(emails: List[str]) -> List[str]:
"""Extract unique domains from a list of email addresses.
Args:
emails (List[str]): List of email addresses
Returns:
List[str]: List of unique domains extracted from the emails
"""
if not emails:
return []
domains = set()
for email in emails:
if email and '@' in email:
# Extract domain part after @
domain = email.split('@')[1].lower()
domains.add(domain)
return sorted(list(domains))
from models.project_model import ProjectModel from models.project_model import ProjectModel
from filevine_client import FilevineClient from filevine_client import FilevineClient
@@ -73,18 +123,19 @@ def process_project(index: int, total: int, project_data: dict, client: Filevine
p = project_data p = project_data
pid = (p.get("projectId") or {}).get("native") pid = (p.get("projectId") or {}).get("native")
print(f"Working on {pid} ({index}/{total})")
client = get_filevine_client() client = get_filevine_client()
if pid is None: if pid is None:
print(f"[SKIP] Missing projectId for item {index}")
return {} return {}
project_name = p.get("projectName", "")
try: try:
c = client.fetch_client((p.get("clientId") or {}).get("native")) c = client.fetch_client((p.get("clientId") or {}).get("native"))
cs = client.fetch_contacts(pid) cs = client.fetch_contacts(pid)
detail = client.fetch_project_detail(pid) detail = client.fetch_project_detail(pid)
except Exception as e: except Exception as e:
print(f"[WARN] Failed to fetch essential data for {pid}: {e}") print(f"[ERROR] Failed to fetch essential data for project {pid} '{project_name}': {e}")
return {} return {}
defendant_one = next((c.get('orgContact', {}) for c in cs if "Defendant" in c.get('orgContact', {}).get('personTypes', [])), {}) defendant_one = next((c.get('orgContact', {}) for c in cs if "Defendant" in c.get('orgContact', {}).get('personTypes', [])), {})
@@ -178,12 +229,8 @@ def process_project(index: int, total: int, project_data: dict, client: Filevine
# Extract attorney fees and costs # Extract attorney fees and costs
attorney_fees = fees_and_costs.get("totalAttorneysFees") or '' attorney_fees = fees_and_costs.get("totalAttorneysFees") or ''
costs = fees_and_costs.get("totalCosts") or '' costs = fees_and_costs.get("totalCosts") or ''
from pprint import pprint
property_managers = [property_contacts.get('propertyManager1'), property_contacts.get('propertyManager2'), property_contacts.get('propertyManager3'), property_contacts.get('propertyManager4')] property_managers = [property_contacts.get('propertyManager1'), property_contacts.get('propertyManager2'), property_contacts.get('propertyManager3'), property_contacts.get('propertyManager4')]
import itertools
# valid_property_managers = list(itertools.chain(*))
valid_property_managers = [e.get('address').lower() for pm in property_managers if pm and pm.get('emails') for e in pm.get('emails') if e and e.get('address')] valid_property_managers = [e.get('address').lower() for pm in property_managers if pm and pm.get('emails') for e in pm.get('emails') if e and e.get('address')]
pprint(valid_property_managers)
row = ProjectModel( row = ProjectModel(
@@ -204,6 +251,7 @@ def process_project(index: int, total: int, project_data: dict, client: Filevine
notice_service_date=notice_service_date, notice_service_date=notice_service_date,
notice_expiration_date=notice_expiration_date, notice_expiration_date=notice_expiration_date,
case_field_date=case_filed_date, case_field_date=case_filed_date,
case_filed_date=case_filed_date,
daily_rent_damages=daily_rent_damages, daily_rent_damages=daily_rent_damages,
default_date=default_date, default_date=default_date,
demurrer_hearing_date=demurrer_hearing_date, demurrer_hearing_date=demurrer_hearing_date,
@@ -235,85 +283,175 @@ def process_project(index: int, total: int, project_data: dict, client: Filevine
service_attempt_date_1=convert_to_pacific_time(next(iter(service_info), {}).get('serviceDate')), service_attempt_date_1=convert_to_pacific_time(next(iter(service_info), {}).get('serviceDate')),
contacts=cs, contacts=cs,
project_email_address=p.get("projectEmailAddress", ""), project_email_address=p.get("projectEmailAddress", ""),
number=p.get("number", ""), number=p.get("number", "") or matter_overview.get('matterNumber', ''),
incident_date=convert_to_pacific_time(p.get("incidentDate") or detail.get("incidentDate")), incident_date=convert_to_pacific_time(p.get("incidentDate") or detail.get("incidentDate")),
project_id=pid, project_id=pid,
project_name=p.get("projectName") or detail.get("projectName"), project_name=p.get("projectName") or detail.get("projectName"),
project_url=p.get("projectUrl") or detail.get("projectUrl"), project_url=p.get("projectUrl") or detail.get("projectUrl"),
#property_contacts=property_contacts #property_contacts=property_contacts
viewing_emails = valid_property_managers viewing_emails = valid_property_managers,
viewing_domains = extract_domains_from_emails(valid_property_managers),
last_synced_at=datetime.now(pytz.UTC).isoformat()
) )
# Store the results in Firestore print(f"[{index}/{total}] Saved: {pid} | Matter {row.number} | {project_name}")
from app import db # Import db from app
projects_ref = db.collection("projects")
from pprint import pprint
# pprint([p.get("number"), property_info, new_file_review])
# Add new projects
project_id = row.project_id
if project_id:
projects_ref.document(str(project_id)).set(row.to_dict())
print(f"Finished on {pid} Matter {row.number} ({index}/{total})")
return row.to_dict() return row.to_dict()
except Exception as e: except Exception as e:
print(f"[ERROR] Processing failed for {pid}: {e}") print(f"[ERROR] Failed to process project {pid} '{project_name}': {e}")
import traceback
traceback.print_exc()
return {} return {}
def process_projects_parallel(projects: List[dict], client: FilevineClient, max_workers: int = 9) -> List[Dict[str, Any]]: def process_projects_parallel(projects: List[dict], client: FilevineClient, max_workers: int = 10) -> List[Dict[str, Any]]:
""" """
Process projects in parallel using a worker pool. Process projects in parallel using a worker pool.
Args: Args:
projects: List of project data dictionaries projects: List of project data dictionaries
client: FilevineClient instance client: FilevineClient instance
max_workers: Number of concurrent workers (default 9) max_workers: Number of concurrent workers (default 10)
Returns: Returns:
List of processed project dictionaries List of processed project dictionaries
""" """
# Create a thread pool with specified number of workers
total = len(projects) total = len(projects)
success_count = 0
fail_count = 0
print(f"[WORKERS] Starting parallel processing of {total} projects with {max_workers} workers...")
with concurrent.futures.ThreadPoolExecutor(max_workers=max_workers, initializer=worker_init, initargs=(client,)) as executor: with concurrent.futures.ThreadPoolExecutor(max_workers=max_workers, initializer=worker_init, initargs=(client,)) as executor:
# Submit all tasks to the executor
future_to_project = {executor.submit(process_project, indx, total, project, client): project for indx, project in enumerate(projects)} future_to_project = {executor.submit(process_project, indx, total, project, client): project for indx, project in enumerate(projects)}
# Collect results as they complete
results = [] results = []
for future in concurrent.futures.as_completed(future_to_project): for future in concurrent.futures.as_completed(future_to_project):
try: try:
result = future.result() result = future.result()
if result and result.get('ProjectId'):
success_count += 1
else:
fail_count += 1
results.append(result) results.append(result)
except Exception as e: except Exception as e:
print(f"[ERROR] Processing failed: {e}") fail_count += 1
# Add empty dict or handle error appropriately print(f"[ERROR] Worker thread failed: {e}")
results.append({}) results.append({})
print(f"[WORKERS] Completed: {success_count} succeeded, {fail_count} failed, {total} total")
return results return results
def get_oldest_unsynced_projects(db, fraction: float = 0.2) -> List[int]:
"""Get the oldest fraction of projects by last_synced_at from Firestore.
Args:
db: Firestore client
fraction: Fraction of projects to return (default 0.2 = 1/5th)
Returns:
List of project IDs (native) that need syncing
"""
try:
projects_ref = db.collection("projects")
all_docs = list(projects_ref.stream())
# Exclude archived projects from the sync pool
active_docs = [doc for doc in all_docs if not doc.to_dict().get("is_archived")]
total = len(active_docs)
count_to_sync = max(1, int(total * fraction))
# Sort by last_synced_at ascending (empty strings first, then oldest timestamps)
sorted_docs = sorted(active_docs, key=lambda doc: doc.to_dict().get("last_synced_at", ""))
selected_docs = sorted_docs[:count_to_sync]
result_ids = [int(doc.id) for doc in selected_docs if doc.id and doc.id != "None"]
print(f"[SYNC STRATEGY] {total} active projects in Firestore, will sync oldest {len(result_ids)} ({fraction*100:.0f}%)")
if selected_docs:
sample = selected_docs[0].to_dict()
print(f"[SYNC STRATEGY] Oldest: ID={result_ids[0]}, last_synced_at='{sample.get('last_synced_at', 'N/A')}'")
if len(selected_docs) > 1:
sample = selected_docs[-1].to_dict()
print(f"[SYNC STRATEGY] Cutoff: ID={result_ids[-1]}, last_synced_at='{sample.get('last_synced_at', 'N/A')}'")
return result_ids
except Exception as e:
print(f"[ERROR] Failed to get oldest unsynced projects: {e}")
import traceback
traceback.print_exc()
return []
def main(): def main():
"""Main function to fetch and sync projects""" """Main function to fetch and sync projects"""
print("Starting project sync...") import argparse
parser = argparse.ArgumentParser(description='Sync Filevine projects to Firestore')
parser.add_argument('--mode', choices=['full', 'last_n', 'oldest_percent', 'hybrid', 'single'],
default='hybrid', help='Sync mode: full=all projects, last_n=recently active, oldest_percent=oldest by last_synced_at, hybrid=last_n+oldest_percent, single=one project')
parser.add_argument('--days', type=int, default=14, help='Number of days for last_n mode (default: 14)')
parser.add_argument('--percent', type=float, default=20.0, help='Percentage for oldest_percent mode (default: 20)')
parser.add_argument('--project-id', type=int, help='Project ID for single mode (required when mode=single)')
args = parser.parse_args()
if args.mode == 'single' and not args.project_id:
parser.error("--project-id is required when mode is 'single'")
print(f"[SYNC] Starting sync - mode={args.mode}, workers=10")
try: try:
# Initialize Filevine client
client = FilevineClient() client = FilevineClient()
bearer = client.get_bearer_token() client.get_bearer_token()
from app import db
# List projects (all pages) with filter for projects updated in the last 7 days if args.mode == 'full':
from datetime import datetime, timedelta print("[MODE] Full sync - fetching all projects")
seven_days_ago = (datetime.now() - timedelta(days=14)).strftime('%Y-%m-%d') projects = client.list_all_projects()
projects = client.list_all_projects(latest_activity_since=seven_days_ago)
#projects = [p for p in projects if (p.get("projectId") or {}).get("native") == 15914808] elif args.mode == 'last_n':
#projects = projects[:10] days_ago = (datetime.now() - timedelta(days=args.days)).strftime('%Y-%m-%d')
print(f"[MODE] Last {args.days} days - fetching active since {days_ago}")
projects = client.list_all_projects(latest_activity_since=days_ago)
elif args.mode == 'oldest_percent':
fraction = args.percent / 100.0
oldest_ids = get_oldest_unsynced_projects(db, fraction=fraction)
print(f"[MODE] Oldest {args.percent}% - fetching {len(oldest_ids)} projects")
all_projects = client.list_all_projects()
projects = [p for p in all_projects if p.get("projectId", {}).get("native") in set(oldest_ids)]
elif args.mode == 'single':
print(f"[MODE] Single project - fetching project {args.project_id}")
project_detail = client.fetch_project_detail(args.project_id)
projects = [project_detail] if project_detail else []
elif args.mode == 'hybrid':
print("[MODE] Hybrid - active + oldest")
days_ago = (datetime.now() - timedelta(days=args.days)).strftime('%Y-%m-%d')
active_projects = client.list_all_projects(latest_activity_since=days_ago)
active_ids = {p.get("projectId", {}).get("native") for p in active_projects}
print(f"[SYNC] {len(active_projects)} active since {days_ago}")
fraction = args.percent / 100.0
oldest_ids = get_oldest_unsynced_projects(db, fraction=fraction)
all_ids_to_sync = active_ids.union(set(oldest_ids))
print(f"[SYNC] {len(all_ids_to_sync)} total unique to sync")
all_projects = client.list_all_projects()
projects = [p for p in all_projects if p.get("projectId", {}).get("native") in all_ids_to_sync]
# Process projects in parallel # Process projects in parallel
detailed_rows = process_projects_parallel(projects, client, 9) detailed_rows = process_projects_parallel(projects, client, max_workers=10)
# Batch write all results to Firestore
documents = []
for row in detailed_rows:
if row.get('ProjectId'):
row['is_archived'] = (row.get('phase_name') == 'Archived')
documents.append((row.get('ProjectId'), row))
batch_write_to_firestore(db, "projects", documents)
print(f"Successfully synced {len(detailed_rows)} projects to Firestore") print(f"[SYNC] Complete - {len(documents)} projects saved to Firestore")
except Exception as e: except Exception as e:
print(f"Error during sync: {e}") print(f"Error during sync: {e}")

View File

@@ -40,6 +40,14 @@
<p class="mt-1 text-sm text-slate-500">The email address used for project access.</p> <p class="mt-1 text-sm text-slate-500">The email address used for project access.</p>
</div> </div>
<div>
<label for="case_domain_email" class="block text-sm font-medium text-slate-700">Case Domain Email</label>
<input type="text" id="case_domain_email" name="case_domain_email"
value=""
class="mt-1 block w-full px-3 py-2 border border-slate-300 rounded-md shadow-sm focus:outline-none focus:ring-blue-500 focus:border-blue-500">
<p class="mt-1 text-sm text-slate-500">All cases with property contacts in this domain will be viewable to the user</p>
</div>
<div class="flex justify-end space-x-3 pt-4"> <div class="flex justify-end space-x-3 pt-4">
<button type="button" onclick="window.location.href='/admin/users'" <button type="button" onclick="window.location.href='/admin/users'"
class="px-4 py-2 text-sm font-medium text-slate-700 bg-gray-100 hover:bg-gray-200 rounded-md transition-colors"> class="px-4 py-2 text-sm font-medium text-slate-700 bg-gray-100 hover:bg-gray-200 rounded-md transition-colors">

View File

@@ -28,9 +28,13 @@
<div class="mt-1 flex items-center"> <div class="mt-1 flex items-center">
<input type="checkbox" id="is_admin" name="is_admin" <input type="checkbox" id="is_admin" name="is_admin"
{% if user.is_admin %}checked{% endif %} {% if user.is_admin %}checked{% endif %}
{% if not user.is_admin %}disabled{% endif %}
class="h-4 w-4 text-blue-600 focus:ring-blue-500 border-slate-300 rounded"> class="h-4 w-4 text-blue-600 focus:ring-blue-500 border-slate-300 rounded">
<label for="is_admin" class="ml-2 block text-sm text-slate-700">Check to make this user an admin</label> <label for="is_admin" class="ml-2 block text-sm text-slate-700">Check to make this user an admin</label>
</div> </div>
{% if not user.is_admin %}
<p class="mt-1 text-sm text-slate-500">Admin status can only be set during user creation.</p>
{% endif %}
</div> </div>
<div> <div>
@@ -38,7 +42,15 @@
<input type="email" id="case_email" name="case_email" <input type="email" id="case_email" name="case_email"
value="{{ user.case_email }}" value="{{ user.case_email }}"
class="mt-1 block w-full px-3 py-2 border border-slate-300 rounded-md shadow-sm focus:outline-none focus:ring-blue-500 focus:border-blue-500"> class="mt-1 block w-full px-3 py-2 border border-slate-300 rounded-md shadow-sm focus:outline-none focus:ring-blue-500 focus:border-blue-500">
<p class="mt-1 text-sm text-slate-500">The email address used for project access.</p> <p class="mt-1 text-sm text-slate-500">All cases with this email will be viewable by the user</p>
</div>
<div>
<label for="case_domain_email" class="block text-sm font-medium text-slate-700">Case Domain Email</label>
<input type="text" id="case_domain_email" name="case_domain_email"
value="{{ user.case_domain_email }}"
class="mt-1 block w-full px-3 py-2 border border-slate-300 rounded-md shadow-sm focus:outline-none focus:ring-blue-500 focus:border-blue-500">
<p class="mt-1 text-sm text-slate-500">All cases with property contacts in this domain will be viewable to the user</p>
</div> </div>
<div class="flex justify-end space-x-3 pt-4"> <div class="flex justify-end space-x-3 pt-4">
@@ -62,8 +74,8 @@ document.getElementById('userForm').addEventListener('submit', function(e) {
const userData = { const userData = {
uid: '{{ user.uid }}', uid: '{{ user.uid }}',
enabled: formData.get('enabled') === 'on', enabled: formData.get('enabled') === 'on',
is_admin: formData.get('is_admin') === 'on', case_email: formData.get('case_email'),
case_email: formData.get('case_email') case_domain_email: formData.get('case_domain_email')
}; };
fetch('/admin/users/update', { fetch('/admin/users/update', {

View File

@@ -1,6 +1,16 @@
{% extends 'base.html' %} {% extends 'base.html' %}
{% block content %} {% block content %}
<div class="h-full flex flex-col"> <div class="h-full flex flex-col">
{% with messages = get_flashed_messages(with_categories=true) %}
{% if messages %}
{% for category, message in messages %}
<div class="bg-{% if category == 'success' %}green{% else %}blue{% endif %}-50 border border-{% if category == 'success' %}green{% else %}blue{% endif %}-200 text-{% if category == 'success' %}green{% else %}blue{% endif %}-800 px-4 py-3 rounded-md mb-4">
<p>{{ message }}</p>
</div>
{% endfor %}
{% endif %}
{% endwith %}
{% if session.get('password_reset_link') %} {% if session.get('password_reset_link') %}
<div class="bg-blue-50 border border-blue-200 text-blue-800 px-4 py-3 rounded-md mb-4"> <div class="bg-blue-50 border border-blue-200 text-blue-800 px-4 py-3 rounded-md mb-4">
<p class="font-medium">Please send an email to {{ session.get('reset_user_email') }}</p> <p class="font-medium">Please send an email to {{ session.get('reset_user_email') }}</p>
@@ -67,6 +77,14 @@
</td> </td>
<td class="px-4 py-3 text-sm text-slate-800">{{ user.case_email }}</td> <td class="px-4 py-3 text-sm text-slate-800">{{ user.case_email }}</td>
<td class="px-4 py-3 text-sm text-slate-800"> <td class="px-4 py-3 text-sm text-slate-800">
<div class="flex flex-col space-y-1">
<form method="POST" action="/admin/users/{{ user.uid }}/become-user" style="display: inline;">
<button type="submit"
class="text-green-600 hover:text-green-800 text-sm font-medium underline"
onclick="return confirm('Are you sure you want to view as {{ user.user_email }}? This will replace your current session.')">
Become User
</button>
</form>
<form method="POST" action="/admin/users/{{ user.uid }}/reset-password" style="display: inline;"> <form method="POST" action="/admin/users/{{ user.uid }}/reset-password" style="display: inline;">
<button type="submit" <button type="submit"
class="text-blue-600 hover:text-blue-800 text-sm font-medium underline" class="text-blue-600 hover:text-blue-800 text-sm font-medium underline"
@@ -74,6 +92,7 @@
Reset Password Reset Password
</button> </button>
</form> </form>
</div>
</td> </td>
</tr> </tr>
{% else %} {% else %}

View File

@@ -18,11 +18,15 @@
<a href="/" class="font-semibold">Rothbard Law Group - Cases</a> <a href="/" class="font-semibold">Rothbard Law Group - Cases</a>
<nav class="space-x-4"> <nav class="space-x-4">
{% if session.uid %} {% if session.uid %}
{% if session.impersonating %}
<a href="/admin/users/revert" class="text-sm text-orange-600 hover:text-orange-900 font-medium">Revert to Admin</a>
{% else %}
<a href="/dashboard" class="text-sm text-slate-600 hover:text-slate-900">Dashboard</a> <a href="/dashboard" class="text-sm text-slate-600 hover:text-slate-900">Dashboard</a>
{% set profile = get_user_profile(session.uid) %} {% set profile = get_user_profile(session.uid) %}
{% if profile.is_admin %} {% if profile.is_admin %}
<a href="/admin/users" class="text-sm text-slate-600 hover:text-slate-900">Admin Users</a> <a href="/admin/users" class="text-sm text-slate-600 hover:text-slate-900">Admin Users</a>
{% endif %} {% endif %}
{% endif %}
<a href="/logout" class="text-sm text-slate-600 hover:text-slate-900">Logout</a> <a href="/logout" class="text-sm text-slate-600 hover:text-slate-900">Logout</a>
{% else %} {% else %}
<a href="/login" class="text-sm text-slate-600 hover:text-slate-900">Login</a> <a href="/login" class="text-sm text-slate-600 hover:text-slate-900">Login</a>

View File

@@ -1,7 +1,24 @@
{% extends 'base.html' %} {% extends 'base.html' %}
{% block content %} {% block content %}
<div class="h-full flex flex-col" x-data="columnConfig()"> <div class="h-full flex flex-col" x-data="columnConfig()">
{% if session.impersonating %}
{% set impersonated_profile = get_user_profile(session.uid) %}
<div class="bg-orange-50 border border-orange-200 text-orange-800 px-4 py-3 rounded-md mb-4 flex justify-between items-center">
<div>
<p class="font-medium">Viewing as: {{ impersonated_profile.user_email }}</p>
<p class="text-sm mt-1">You are impersonating this user. Click "Revert to Admin" in the navigation to return to your admin account.</p>
</div>
<a href="/admin/users/revert" class="inline-flex items-center px-3 py-1.5 text-sm font-medium text-orange-700 bg-orange-100 hover:bg-orange-200 rounded-md transition-colors">
Revert to Admin
</a>
</div>
{% endif %}
{% if case_email %}
<h1 class="text-xl font-semibold mb-4">Projects for {{ case_email }}</h1> <h1 class="text-xl font-semibold mb-4">Projects for {{ case_email }}</h1>
{% else %}
<h1 class="text-xl font-semibold mb-4">All projects</h1>
{% endif %}
<div class="flex justify-between"> <div class="flex justify-between">
{% set profile = get_user_profile(session.uid) %} {% set profile = get_user_profile(session.uid) %}
@@ -150,6 +167,9 @@
<th style="background-color: rgb(89, 121, 142);" class="px-4 py-3 w-32 sticky top-0 z-[40]" :class="{'hidden': !isColumnVisible('Date Possession Recovered')}">Date Possession Recovered</th> <th style="background-color: rgb(89, 121, 142);" class="px-4 py-3 w-32 sticky top-0 z-[40]" :class="{'hidden': !isColumnVisible('Date Possession Recovered')}">Date Possession Recovered</th>
<th style="background-color: rgb(89, 121, 142);" class="px-4 py-3 w-32 sticky top-0 z-[40]" :class="{'hidden': !isColumnVisible('Attorney\'s Fees')}">Attorney's Fees</th> <th style="background-color: rgb(89, 121, 142);" class="px-4 py-3 w-32 sticky top-0 z-[40]" :class="{'hidden': !isColumnVisible('Attorney\'s Fees')}">Attorney's Fees</th>
<th style="background-color: rgb(89, 121, 142);" class="px-4 py-3 w-32 sticky top-0 z-[40]" :class="{'hidden': !isColumnVisible('Costs')}">Costs</th> <th style="background-color: rgb(89, 121, 142);" class="px-4 py-3 w-32 sticky top-0 z-[40]" :class="{'hidden': !isColumnVisible('Costs')}">Costs</th>
{% if is_admin %}
<th style="background-color: rgb(89, 121, 142);" class="px-4 py-3 w-32 sticky top-0 z-[40]" :class="{'hidden': !isColumnVisible('Last Synced')}">Last Synced</th>
{% endif %}
</tr> </tr>
</thead> </thead>
<tbody class="bg-slate-100 divide-y divide-slate-300"> <tbody class="bg-slate-100 divide-y divide-slate-300">
@@ -375,9 +395,8 @@
{% endcall %} {% endcall %}
</td> </td>
<td class="px-4 py-3 text-sm" :class="{'hidden': !isColumnVisible('Date Case Filed')}"> <td class="px-4 py-3 text-sm" :class="{'hidden': !isColumnVisible('Date Case Filed')}">
{% call expander() %} {% call expander() %}
{{ r.case_field_date }} {% if r.case_filed_date %}{{ r.case_filed_date }}{% else %}{{ r.case_field_date }}{% endif %}
{% endcall %} {% endcall %}
</td> </td>
<td class="px-4 py-3 text-sm" :class="{'hidden': !isColumnVisible('Daily Rent Damages')}"> <td class="px-4 py-3 text-sm" :class="{'hidden': !isColumnVisible('Daily Rent Damages')}">
@@ -526,6 +545,13 @@
{{ r.costs }} {{ r.costs }}
{% endcall %} {% endcall %}
</td> </td>
{% if is_admin %}
<td class="px-4 py-3 text-sm" :class="{'hidden': !isColumnVisible('Last Synced')}">
{% call expander() %}
{% if r.last_synced_at %}{{ r.last_synced_at.split('T')[0] }}{% endif %}
{% endcall %}
</td>
{% endif %}
</tr> </tr>
{% else %} {% else %}
<tr> <tr>
@@ -594,7 +620,8 @@
'Matter Gate or Entry Code', 'Matter Gate or Entry Code',
'Date Possession Recovered', 'Date Possession Recovered',
'Attorney\'s Fees', 'Attorney\'s Fees',
'Costs' 'Costs',
'Last Synced'
], ],
selectAll: true, selectAll: true,
visibleColumns: [], visibleColumns: [],

View File

@@ -4,6 +4,9 @@
<div class="flex justify-center py-8"> <div class="flex justify-center py-8">
<div class="w-full max-w-md p-8 space-y-6 bg-white rounded-xl shadow-lg"> <div class="w-full max-w-md p-8 space-y-6 bg-white rounded-xl shadow-lg">
<h1 class="text-2xl font-bold text-center text-gray-800">Secure Access</h1> <h1 class="text-2xl font-bold text-center text-gray-800">Secure Access</h1>
<div class="bg-blue-50 border border-blue-200 text-blue-700 px-4 py-3 rounded-lg mb-4">
<p>If you don't have a user account, or need to reset your password, send an email to <a href="mailto:office@rothbardlawgroup.com" class="underline">office@rothbardlawgroup.com</a>.</p>
</div>
<form id="login-form" class="space-y-4"> <form id="login-form" class="space-y-4">
<div> <div>
<label class="block text-sm font-medium text-gray-700 mb-1">Email Address</label> <label class="block text-sm font-medium text-gray-700 mb-1">Email Address</label>
@@ -28,18 +31,14 @@
<script> <script>
// Initialize Firebase configuration from template // Initialize Firebase configuration from template
window.FIREBASE_CONFIG = {{ firebase_config|tojson }}; window.FIREBASE_CONFIG = {{ firebase_config|tojson }};
// Initialize Firebase app and auth
const app = firebase.initializeApp(window.FIREBASE_CONFIG || {}); const app = firebase.initializeApp(window.FIREBASE_CONFIG || {});
const auth = firebase.auth(); const auth = firebase.auth();
// Get form and input elements
const form = document.getElementById('login-form'); const form = document.getElementById('login-form');
const email = document.getElementById('email'); const email = document.getElementById('email');
const password = document.getElementById('password'); const password = document.getElementById('password');
const err = document.getElementById('error'); const err = document.getElementById('error');
// Handle form submission
form.addEventListener('submit', async (e) => { form.addEventListener('submit', async (e) => {
e.preventDefault(); e.preventDefault();
err.classList.add('hidden'); err.classList.add('hidden');
@@ -51,10 +50,17 @@
headers: { 'Content-Type': 'application/json' }, headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ idToken }) body: JSON.stringify({ idToken })
}); });
if(!res.ok){
if (!res.ok) {
throw new Error('Session exchange failed'); throw new Error('Session exchange failed');
} }
const data = await res.json();
if (data.requires_password_reset) {
window.location.href = '/require-password-reset';
} else {
window.location.href = '/'; window.location.href = '/';
}
} catch (e) { } catch (e) {
err.textContent = e.message || 'Authentication failed'; err.textContent = e.message || 'Authentication failed';
err.classList.remove('hidden'); err.classList.remove('hidden');

View File

@@ -0,0 +1,47 @@
{% extends 'base.html' %}
{% block content %}
<div class="h-full flex flex-col max-w-md mx-auto">
<h1 class="text-xl font-semibold mb-6">Password Reset Required</h1>
<form method="POST" action="/reset-password-submit" class="space-y-6">
<div>
<label for="new_password" class="block text-sm font-medium text-slate-700">New Password</label>
<input type="password" id="new_password" name="new_password"
value=""
required
class="mt-1 block w-full px-3 py-2 border border-slate-300 rounded-md shadow-sm focus:outline-none focus:ring-blue-500 focus:border-blue-500"
minlength="6">
<p class="mt-1 text-sm text-slate-500">Password must be at least 6 characters</p>
</div>
<div>
<label for="confirm_password" class="block text-sm font-medium text-slate-700">Confirm Password</label>
<input type="password" id="confirm_password" name="confirm_password"
value=""
required
class="mt-1 block w-full px-3 py-2 border border-slate-300 rounded-md shadow-sm focus:outline-none focus:ring-blue-500 focus:border-blue-500">
</div>
{% with messages = get_flashed_messages(with_categories=true) %}
{% if messages %}
{% for category, message in messages %}
<div class="bg-red-50 border border-red-200 text-red-800 px-4 py-3 rounded-md">
{{ message }}
</div>
{% endfor %}
{% endif %}
{% endwith %}
<div class="flex justify-end space-x-3 pt-4">
<a href="/login"
class="px-4 py-2 text-sm font-medium text-slate-700 bg-gray-100 hover:bg-gray-200 rounded-md transition-colors text-center inline-block">
Cancel
</a>
<button type="submit"
class="px-4 py-2 text-sm font-medium text-white bg-blue-600 hover:bg-blue-700 rounded-md transition-colors">
Reset Password
</button>
</div>
</form>
</div>
{% endblock %}

152
terraform/main.tf Normal file
View File

@@ -0,0 +1,152 @@
terraform {
required_providers {
google-beta = {
source = "hashicorp/google-beta"
version = "~> 6.0"
}
}
}
provider "google" {
project = var.gcp_project_id
region = var.gcp_region
}
# Firebase Project Setup
resource "google_firebase_project" "default" {
provider = google-beta
project = var.gcp_project_id
}
# Firebase Web App
resource "google_firebase_web_app" "rothbard_portal" {
provider = google-beta
project = google_firebase_project.default.project
display_name = "Rothbard Client Portal"
app_urls = ["https://${var.domain_name}"]
}
# Firestore Database
resource "google_firestore_database" "default" {
provider = google-beta
project = var.gcp_project_id
name = "(default)"
location_id = var.firestore_location
type = "FIRESTORE_NATIVE"
delete_protection_state = "DELETE_PROTECTION_DISABLED"
}
# Firebase Authentication - Complete Configuration
resource "google_firebase_auth_config" "default" {
provider = google-beta
project = var.gcp_project_id
sign_in_options {
email {
enabled = true
password_required = true
}
# Disable other providers for security
phone {
enabled = false
}
google {
enabled = var.enable_google_signin
}
facebook {
enabled = false
}
apple {
enabled = false
}
}
# Email configuration
email {
reset_password_template {
from_email_address = var.auth_from_email
from_display_name = var.auth_from_name
reply_to = var.auth_reply_to
subject = "Reset your Rothbard Law Group password"
html = file("${path.module}/templates/reset_password.html")
text = file("${path.module}/templates/reset_password.txt")
}
email_verification_template {
from_email_address = var.auth_from_email
from_display_name = var.auth_from_name
reply_to = var.auth_reply_to
subject = "Verify your Rothbard Law Group account"
html = file("${path.module}/templates/email_verification.html")
text = file("${path.module}/templates/email_verification.txt")
}
}
# Security settings
sign_in {
allow_duplicate_emails = false
}
# Multi-factor authentication (disabled for simplicity)
multi_factor_auth {
enabled = false
}
# Anonymous user access (disabled)
anonymous {
enabled = false
}
}
# Service Account for the Flask App
resource "google_service_account" "flask_app" {
account_id = "rothbard-flask-app"
display_name = "Rothbard Flask App Service Account"
}
# IAM permissions for the Flask App
resource "google_project_iam_member" "firestore_access" {
project = var.gcp_project_id
role = "roles/datastore.user"
member = "serviceAccount:${google_service_account.flask_app.email}"
}
resource "google_project_iam_member" "firebase_admin" {
project = var.gcp_project_id
role = "roles/firebase.admin"
member = "serviceAccount:${google_service_account.flask_app.email}"
}
# Firestore Security Rules - Note: Firestore security policies are managed through Firestore rules
# This section is commented out as google_firestore_security_policy is not supported
# Security rules should be managed through firestore.rules file or Firebase console
# Firebase Hosting (optional - for static assets)
resource "google_firebase_hosting_site" "default" {
provider = google-beta
project = var.gcp_project_id
site_id = "rothbard-portal"
}
# Output important values
output "firebase_web_app_id" {
description = "Firebase Web App ID"
value = google_firebase_web_app.rothbard_portal.app_id
}
output "firebase_project_id" {
description = "Firebase Project ID"
value = google_firebase_project.default.project
}
output "service_account_email" {
description = "Service account email for Flask app"
value = google_service_account.flask_app.email
}

View File

@@ -0,0 +1,144 @@
# Cloud Run Service for Flask App
resource "google_cloud_run_service" "flask_app" {
name = "${var.app_name}-service"
location = var.gcp_region
template {
spec {
containers {
image = var.container_image
# Environment variables for the Flask app
env {
name = "FLASK_SECRET_KEY"
value = var.flask_secret_key
}
env {
name = "FIREBASE_PROJECT_ID"
value = var.firebase_project_id
}
env {
name = "GOOGLE_APPLICATION_CREDENTIALS"
value = "/etc/secrets/service-account.json"
}
# Filevine API credentials
env {
name = "FILEVINE_CLIENT_ID"
value = var.filevine_client_id
}
env {
name = "FILEVINE_CLIENT_SECRET"
value = var.filevine_client_secret
}
env {
name = "FILEVINE_PERSONAL_ACCESS_TOKEN"
value = var.filevine_pat
}
env {
name = "FILEVINE_ORG_ID"
value = var.filevine_org_id
}
env {
name = "FILEVINE_USER_ID"
value = var.filevine_user_id
}
# Memory and CPU limits
resources {
limits = {
cpu = "1000m"
memory = "512Mi"
}
}
# Mount service account key
volume_mount {
name = "service-account-key"
mount_path = "/etc/secrets"
read_only = true
}
}
# Service account for the container
service_account_name = var.service_account_email
# Volumes
volumes {
name = "service-account-key"
secret {
secret_name = google_secret_manager_secret.service_account_key.secret_id
items {
key = "latest"
path = "service-account.json"
}
}
}
# Allow unauthenticated access
container_concurrency = 100
timeout_seconds = 300
}
# Traffic settings
metadata {
annotations = {
"autoscaling.knative.dev/maxScale" = "10"
"autoscaling.knative.dev/minScale" = "1"
"run.googleapis.com/ingress" = "all"
}
}
}
traffic {
percent = 100
latest_revision = true
}
depends_on = [google_secret_manager_secret_version.service_account_key]
}
# Make Cloud Run service publicly accessible
resource "google_cloud_run_service_iam_member" "public" {
location = google_cloud_run_service.flask_app.location
project = google_cloud_run_service.flask_app.project
service = google_cloud_run_service.flask_app.name
role = "roles/run.invoker"
member = "allUsers"
}
# Store service account key in Secret Manager
resource "google_secret_manager_secret" "service_account_key" {
project = var.gcp_project_id
secret_id = "${var.app_name}-service-account-key"
replication {
automatic {}
}
}
resource "google_secret_manager_secret_version" "service_account_key" {
secret = google_secret_manager_secret.service_account_key.id
secret_data = var.service_account_key_data
}
# Cloud Storage bucket for container storage (if needed)
resource "google_storage_bucket" "app_storage" {
name = "${var.app_name}-storage-${var.gcp_project_id}"
location = var.gcp_region
force_destroy = true
uniform_bucket_level_access = true
}
# Output the service URL
output "service_url" {
description = "Cloud Run service URL"
value = google_cloud_run_service.flask_app.status[0].url
}

View File

@@ -3,7 +3,7 @@ from firebase_init import db
from firebase_admin import auth as fb_auth from firebase_admin import auth as fb_auth
def get_user_profile(uid: str): def get_user_profile(uid: str):
"""Fetch user's Firestore profile: users/{uid} => { enabled, case_email, is_admin, user_email }""" """Fetch user's Firestore profile: users/{uid} => { enabled, case_email, is_admin, user_email, password_reset_required }"""
doc_ref = db.collection("users").document(uid) doc_ref = db.collection("users").document(uid)
snap = doc_ref.get() snap = doc_ref.get()
if not snap.exists: if not snap.exists:
@@ -19,18 +19,24 @@ def get_user_profile(uid: str):
"enabled": False, "enabled": False,
"is_admin": False, "is_admin": False,
"user_email": user_email, "user_email": user_email,
"case_email": user_email "case_email": user_email,
"case_domain_email": "",
"password_reset_required": True
}, merge=True) }, merge=True)
return { return {
"enabled": False, "enabled": False,
"is_admin": False, "is_admin": False,
"user_email": user_email, "user_email": user_email,
"case_email": user_email "case_email": user_email,
"case_domain_email": "",
"password_reset_required": True
} }
data = snap.to_dict() or {} data = snap.to_dict() or {}
return { return {
"enabled": bool(data.get("enabled", False)), "enabled": bool(data.get("enabled", False)),
"is_admin": bool(data.get("is_admin", False)), "is_admin": bool(data.get("is_admin", False)),
"user_email": data.get("user_email"), "user_email": data.get("user_email"),
"case_email": data.get("case_email") "case_email": data.get("case_email"),
"case_domain_email": data.get("case_domain_email", ""),
"password_reset_required": bool(data.get("password_reset_required", False))
} }