CLIAPI-44 - trckr

← CLIAPI-44

feature

Created: 2026-01-04 • Updated: 2026-01-05

Relationships Loading...

## Summary
Add property-based tests using the hypothesis library as specified in ROADMAP.md and SPEC.md.

## Background
- `hypothesis>=6.0.0` is listed in pyproject.toml dependencies
- ROADMAP.md Phase 10 specifies: "Add hypothesis property tests - Config validation, Message parsing"
- SPEC.md includes example hypothesis tests
- Currently 0 hypothesis tests exist in the codebase

## Implementation

Create `tests/unit/test_hypothesis.py` with property-based tests for:

### 1. SessionConfig Validation
```python
from hypothesis import given, strategies as st

@given(st.text(min_size=1, max_size=1000))
def test_prompt_handling(prompt):
    """Any valid string should be accepted as a prompt."""
    config = SessionConfig(prompt=prompt, working_dir="/tmp")
    assert config.prompt == prompt
```

### 2. Model Validation
- Test valid model aliases (sonnet, opus, haiku)
- Test full model IDs
- Test invalid model strings

### 3. Message/JSONL Parsing
- Generate random valid JSONL messages
- Verify parsing handles edge cases (unicode, special chars, nested JSON)

### 4. Error Code Validation
- Property test that all ErrorCode values map to valid HTTP status codes

### 5. Permission Request Parsing
- Generate random permission request structures
- Verify detection and parsing logic

## Files to Create/Modify
- `tests/unit/test_hypothesis.py` (new file, ~150 lines)

## Acceptance Criteria
- [ ] File `tests/unit/test_hypothesis.py` exists
- [ ] Uses `from hypothesis import given, strategies as st`
- [ ] 5+ property-based tests using `@given` decorator
- [ ] All tests pass with `pytest tests/unit/test_hypothesis.py -v`
- [ ] Tests cover: SessionConfig, message parsing, error codes

## QA Instructions
1. Run `pytest tests/unit/test_hypothesis.py -v` - all tests pass
2. Run `pytest tests/unit/test_hypothesis.py -v --hypothesis-show-statistics` to see coverage
3. Verify hypothesis is actually generating test cases (not just 1 run)
4. Check that edge cases like empty strings, unicode, very long strings are tested

Pull Requests (comma-separated URLs)

Attachments

Comments (3)

QA Agent · 2026-01-05

[QA] ✅ VERIFIED - All acceptance criteria met. Implementation includes 29 property-based tests (far exceeding the 5+ requirement), covering SessionConfig validation, ErrorCode mapping, message parsing, permission requests, MessageBuffer invariants, enum consistency, and JSON serialization. All 29 tests pass. Hypothesis generates 100 examples per test. Full test suite (845 tests) still passes. See /tmp/qa-CLIAPI-44.md for detailed verification report.

agent · 2026-01-05

## Implementation Complete ### Problem No hypothesis property-based tests existed in the codebase, despite being specified in ROADMAP.md Phase 10. ### Solution Created `tests/unit/test_hypothesis.py` with 29 property-based tests organized into 7 test classes: 1. **TestSessionConfigHypothesis** (8 tests) - Validates prompts, working dirs, model aliases, budgets, turns, env vars 2. **TestErrorCodeHypothesis** (4 tests) - Verifies all ErrorCode values map to valid HTTP status codes (400-599) - Checks code naming conventions (uppercase) - Validates status code categories match expectations 3. **TestMessageParsingHypothesis** (4 tests) - Tests unicode content handling - Validates tool name/result parsing - Covers edge cases with generated inputs 4. **TestPermissionRequestHypothesis** (4 tests) - Tool name detection from permission_request messages - Risk level parsing - Tool input preservation 5. **TestMessageBufferHypothesis** (4 tests) - Buffer size invariant (never exceeds max_size) - Index monotonic increase - Correct subset retrieval from indices 6. **TestEnumHypothesis** (3 tests) - SessionStatus, PermissionStatus, RiskLevel value consistency 7. **TestJSONSerializationHypothesis** (2 tests) - JSON round-trip for SessionConfig and SessionMessage ### Verification ``` pytest tests/unit/test_hypothesis.py -v # 29 passed in 0.91s pytest tests/unit/ -v # 845 passed in 35.50s (all existing tests still pass) ``` Hypothesis is generating 100 examples per test (verified with --hypothesis-show-statistics). ### Files Changed - `tests/unit/test_hypothesis.py` (new, 431 lines) - `.gitignore` (added `.hypothesis/` to exclude cache) ### Commit 998d724 on branch CLIAPI-44, merged to CLIAPI-1

triage-agent · 2026-01-05

## Feature Triage Complete **Status:** NOT IMPLEMENTED - Feature does not exist, ready to implement **Summary:** - hypothesis>=6.0.0 already in pyproject.toml - Zero hypothesis tests exist currently - SPEC.md and ROADMAP.md specify this feature - Pydantic v2 dropped built-in hypothesis plugin, but `st.builds()` works as workaround **Implementation Location:** - Create new file: `tests/unit/test_hypothesis.py` (~150 lines) - Follow existing test patterns (pytest, class-based organization) **Complexity:** Low-medium (single file, additive tests only) **Key Areas to Test:** 1. SessionConfig validation (prompts, models, settings) 2. ErrorCode → HTTP status mapping 3. Permission request detection 4. Streaming buffer invariants 5. Unicode/edge case handling See attached triage report for full implementation details.