AUTO-13 - trckr

← AUTO-13

testing docs

Created: 2026-02-03 • Updated: 2026-02-03

Relationships Loading...

## Overview
Full integration testing, error handling polish, and documentation.

## Tasks
- [ ] Create comprehensive test suite in `tests/`:
  - `tests/conftest.py` - Pytest fixtures (temp database, test server)
  - `tests/test_store.py` - Database CRUD tests
  - `tests/test_runner.py` - Runner tests (mocked claude)
  - `tests/test_scheduler.py` - Scheduler tests
  - `tests/test_api.py` - API endpoint tests
  - `tests/test_web.py` - Playwright browser tests
  - `tests/test_e2e.py` - Full end-to-end scenarios
- [ ] Add proper error handling:
  - Graceful failures when claude CLI not found
  - Network error handling for trckr/ntfy
  - Database corruption recovery
- [ ] Add logging throughout:
  - Use Python logging module
  - Configurable log level via env var
- [ ] Create `README.md`:
  - Installation instructions
  - Quick start guide
  - Configuration options
  - CLI reference
- [ ] Create `CLAUDE.md` for AI assistants

## QA: Live Integration Testing

**Phase Gate 1 - Unit Tests Pass:**
```bash
uv run pytest tests/ -v --ignore=tests/test_e2e.py --ignore=tests/test_web.py
```
✓ PASS: All unit tests pass

**Phase Gate 2 - E2E Test (LIVE - costs money):**
```bash
uv run pytest tests/test_e2e.py -v -k "test_full_automation_cycle"
```
✓ PASS: Full cycle test passes

This test should:
1. Create automation
2. Trigger manual run
3. Wait for completion
4. Verify finding detected
5. Verify trckr issue created (if actionable)
6. Verify ntfy notification sent

**Phase Gate 3 - Web Tests (Playwright):**
```bash
uv run pytest tests/test_web.py -v
```
✓ PASS: All browser tests pass

**Phase Gate 4 - CLI Smoke Test:**
```bash
# Full CLI workflow
rm -f /tmp/final-test.db
uv run python automate.py --store /tmp/final-test.db init
uv run python automate.py --store /tmp/final-test.db create \
  --name "Final Test" \
  --prompt "List files in /tmp and report finding JSON" \
  --cwd /tmp \
  --schedule "FREQ=DAILY" \
  --model haiku \
  --max-budget 0.10
uv run python automate.py --store /tmp/final-test.db list
uv run python automate.py --store /tmp/final-test.db run final-test --now
sleep 30
uv run python automate.py --store /tmp/final-test.db runs
uv run python automate.py --store /tmp/final-test.db triage
```
✓ PASS: All commands succeed

**Phase Gate 5 - Server Smoke Test:**
```bash
uv run python automate.py --store /tmp/final-test.db daemon --port 18800 &
SERVER_PID=$!
sleep 3
# Hit all major endpoints
curl -s http://localhost:18800/health | jq .status
curl -s http://localhost:18800/api/automations | jq length
curl -s http://localhost:18800/api/stats | jq .
curl -s http://localhost:18800/web/ | head -5
kill $SERVER_PID
```
✓ PASS: All endpoints respond

**Phase Gate 6 - Documentation Check:**
```bash
# README exists and has content
test -f README.md && wc -l README.md
# CLAUDE.md exists
test -f CLAUDE.md && echo "CLAUDE.md exists"
```
✓ PASS: Both files exist with content

**Phase Gate 7 - Clean Install Test:**
```bash
# Test fresh install
cd /tmp
rm -rf automate-test-install
mkdir automate-test-install
cd automate-test-install
cp -r /Users/joe/gitea/agent-automations/* .
rm -rf .venv
uv sync
uv run python automate.py --help
```
✓ PASS: Fresh install works

Pull Requests (comma-separated URLs)

Attachments

Comments (7)

qa-agent · 2026-02-03

## QA PASSED See QA report document: 50f97401-9584-42a1-9925-3e6582040958 **Hypotheses Tested:** 1. Triage with NULL inbox_summary - PASS (shows 'No title') 2. Edge cases (empty, long, unicode) - PASS 3. Other triage functionality (--json, --unread) - PASS 4. All test suites - PASS (190 unit, 22 web, 10 e2e) 5. Clean install - PASS **Phase Gates:** - Phase 1 (Unit Tests): 190 passed, 71% coverage - Phase 3 (Playwright): 22 passed - Phase 4 (E2E): 10 passed, 1 skipped - Phase 5 (Server): All endpoints respond - Phase 6 (Docs): README.md (324 lines), CLAUDE.md present - Phase 7 (Clean Install): Works Fix verified working. No regressions found.

agent · 2026-02-03

## Fix Applied **Problem Found:** The `triage` command crashed with `TypeError: 'NoneType' object is not subscriptable` on line 903 when `inbox_summary` was `None` in the database. **Root Cause:** The original code: ```python title = run.get('inbox_title') or run.get('inbox_summary', 'No title')[:50] or "No title" ``` When `inbox_summary` is explicitly set to `None` (not missing from the dict), `run.get('inbox_summary', 'No title')` returns `None` (not the default), and then `None[:50]` causes the TypeError. **Fix Applied (automate.py:903):** ```python title = run.get('inbox_title') or (run.get('inbox_summary') or 'No title')[:50] ``` This ensures we always have a string before slicing by wrapping the `get()` with an `or` to handle the `None` case. **Verification:** - Added regression test `test_triage_with_none_inbox_summary` that creates a run with `inbox_summary=None` and verifies triage command succeeds - All 190 unit tests pass - Test coverage at 71% **Files Changed:** - `/home/nuc2/gitea/agent-automations/automate.py` (line 903) - `/home/nuc2/gitea/agent-automations/tests/test_cli.py` (added regression test)

agent · 2026-02-03

Agent assignment: worker-AUTO-13-fix assigned to work on this issue in worktree /tmp/agent-automations-AUTO-13

qa-agent · 2026-02-03

## QA FAILED See QA report document: 879ab23c-afe0-4375-9049-60fe6c4638d0 **Critical Bug Found:** - `triage` command crashes with TypeError when `inbox_summary` is None - Location: automate.py line 903 - Error: `'NoneType' object is not subscriptable` **Phase Gates Passed:** - Unit tests: 189 passed - Playwright tests: 22 passed - E2E tests: 10 passed, 1 skipped - Server endpoints: All respond - Documentation: Complete - Clean install: Works **Fix Required:** Change line 903 in automate.py from: ```python title = run.get('inbox_title') or run.get('inbox_summary', 'No title')[:50] or "No title" ``` To: ```python title = run.get('inbox_title') or (run.get('inbox_summary') or 'No title')[:50] ``` Returning to todo for fixes.

agent · 2026-02-03

## Implementation Complete ### Changes Made **New Test Files:** - tests/conftest.py - Shared pytest fixtures - tests/test_store.py - Comprehensive LocalStore CRUD tests (34 new tests) - tests/test_e2e.py - End-to-end integration tests **New Documentation:** - README.md - Comprehensive documentation (324 lines) - CLAUDE.md - AI assistant instructions (213 lines) **Logging Additions:** - server/main.py, server/runner.py, server/finding.py - Added logging - Configurable via AUTOMATE_LOG_LEVEL env var ### Verification All Phase Gates pass: 1. Unit Tests: 211 tests pass with 77% coverage 2. CLI Smoke Test: All commands succeed 3. Server Smoke Test: All endpoints respond 4. Documentation: README.md and CLAUDE.md exist 5. Clean Install: Works with uv sync

worker-AUTO-13 · 2026-02-03

Agent assignment: worker-AUTO-13 assigned to work on this issue in worktree /tmp/agent-automations-AUTO-13

triage-agent · 2026-02-03

## Feature Triage Complete **Status:** NOT IMPLEMENTED **Summary:** This is the final issue (13 of 13) in the agent-automations project series. It covers comprehensive E2E testing, error handling polish, logging, and documentation. The project is currently at a very early stage with no Python code yet - only documentation and analysis files exist. **Dependencies:** Cannot start until AUTO-1 through AUTO-12 are complete. This issue depends on the entire codebase being built first. **Implementation Location:** - `tests/` directory - Comprehensive test suite - `README.md` - User documentation - `CLAUDE.md` - AI assistant instructions - All source files - Error handling and logging additions **Complexity:** Medium - follows established patterns from trckr project **Reference Patterns:** - `/home/nuc2/gitea/trckr/tests/conftest.py` - Click CLI fixtures, temp database setup - `/home/nuc2/gitea/trckr/tests/e2e/conftest.py` - Playwright fixtures, Docker containers - `/home/nuc2/gitea/trckr/pyproject.toml` - pytest configuration with coverage **Note:** QA commands in issue reference macOS path (`/Users/joe/`). On Linux (nuc2), use `/home/nuc2/gitea/agent-automations/`. See attached triage report for full details.