fast-next-template/backend/docs/COVERAGE_REPORT.md

Test Coverage Analysis Report

Date: 2025-11-02 (Updated) Current Coverage: 88% (2,157/2,455 lines) Previous Coverage: 79% (1,932/2,439 lines) Target Coverage: 95% Gap: ~175 lines needed to reach 95%

Executive Summary

This report documents the successful resolution of the coverage tracking issue and the path to reach the 95% coverage target.

Current Status

• Total Tests: 598 passing ✅
• Overall Coverage: 88% (up from 79%)
• Lines Covered: 2,157 / 2,455
• Lines Missing: 298 (down from 507)
• Improvement: +9 percentage points (+225 lines covered)

✅ RESOLVED: Coverage Tracking Issue

Problem: Pytest-cov was not properly recording coverage for FastAPI route files executed through httpx's ASGITransport, despite:

1. Tests passing successfully (598/598 ✓)
2. Manual verification showing code paths ARE being executed
3. Correct responses being returned from endpoints

Root Cause Identified: Coverage.py was not configured to track async code execution through ASGI transport's greenlet-based concurrency model.

Solution: Added concurrency = thread,greenlet to .coveragerc

[run]
source = app
concurrency = thread,greenlet  # ← THIS WAS THE FIX!
omit = ...

Results After Fix:

• admin.py: 46% → 98% (+52 points!)
• auth.py: 79% → 95% (+16 points)
• sessions.py: 49% → 84% (+35 points)
• users.py: 60% → 93% (+33 points)
• Overall: 79% → 88% (+9 points)

Detailed Coverage Breakdown (Post-Fix)

Files with Excellent Coverage (95%+) ✅

• app/crud/session.py: 100%
• app/utils/security.py: 100%
• app/schemas/sessions.py: 100%
• app/schemas/errors.py: 100%
• app/services/email_service.py: 100%
• app/services/session_cleanup.py: 100%
• app/api/main.py: 100%
• app/api/routes/admin.py: 98% (was 46%!)
• app/core/config.py: 98%
• app/schemas/common.py: 97%
• app/utils/device.py: 97%
• app/auth.py: 95%
• app/core/exceptions.py: 95%

Files Requiring Coverage Improvement (to reach 95%)

1. app/api/routes/organizations.py - Priority: CRITICAL ⚠️

• Coverage: 35% (23/66 lines)
• Missing Lines: 43
• Impact: Largest remaining gap, NO TESTS EXIST

Missing Coverage Areas:

Lines 54-83     : List organizations endpoint (entire function)
Lines 103-128   : Get organization by ID (entire function)
Lines 150-172   : Add member to organization (entire function)
Lines 193-221   : Remove member from organization (entire function)

Required Tests: Create tests/api/test_organizations.py with ~12-15 tests

---

2. app/crud/base.py - Priority: HIGH

• Coverage: 73% (164/224 lines)
• Missing Lines: 60
• Impact: Foundation class for all CRUD operations

Missing Coverage Areas:

Lines 77-78     : Exception handling in get()
Lines 119-120   : Exception handling in get_multi()
Lines 130-152   : Advanced filtering logic in get_multi()
Lines 254-296   : Pagination, sorting, filtering in get_multi_with_total()
Lines 342-343   : Exception handling in update()
Lines 383-384   : Exception handling in remove()
Lines 143-144   : User creation success logging
Lines 146-147   : User creation error handling (ValueError)
Lines 170-175   : Get user NotFoundError
Lines 194-208   : Update user success + error paths
Lines 226-252   : Delete user (success, self-check, errors)
Lines 270-288   : Activate user (success + errors)
Lines 306-332   : Deactivate user (success, self-check, errors)
Lines 375-396   : Bulk actions (activate/deactivate/delete) + results
Lines 427-452   : List organizations with pagination + member counts
Lines 475-489   : Create organization success + response building
Lines 492-493   : Create organization ValueError
Lines 516-533   : Get organization + member count
Lines 552-578   : Update organization success + member count
Lines 596-614   : Delete organization success
Lines 634-664   : List organization members with pagination
Lines 689-731   : Add member to organization (success + errors)
Lines 750-786   : Remove member from organization (success + errors)

Tests Created (not reflected in coverage):

• 20 new tests covering all the above scenarios
• All tests pass successfully
• Manual verification confirms endpoints return correct data

Recommended Actions:

1. Run coverage with single-process mode: pytest -n 0 --cov
2. Use coverage HTML report: pytest --cov=app --cov-report=html
3. Investigate pytest-cov source mode vs trace mode
4. Consider running coverage separately from test execution

---

2. app/api/routes/users.py - Priority: MEDIUM

• Coverage: 63% (58/92 lines) - Improved from 58%!
• Missing Lines: 34

Missing Coverage Areas:

Lines 87-100    : List users pagination (superuser endpoint)
Lines 150-154   : Dead code - UserUpdate schema doesn't include is_superuser
                  (MARKED with pragma: no cover)
Lines 163-164   : Update current user success logging
Lines 211-217   : Get user by ID NotFoundError + return
Lines 262-286   : Update user by ID (NotFound, auth check, success, errors)
Lines 270-275   : Dead code - is_superuser validation unreachable
                  (MARKED with pragma: no cover)
Lines 377-396   : Delete user by ID (NotFound, success, errors)

Tests Created:

• 10 new tests added
• Improved coverage from 58% → 63%
• Marked unreachable code with # pragma: no cover

Remaining Work:

• Lines 87-100: List users endpoint needs superuser fixture
• Lines 163-164: Success path logging
• Lines 211-217: Get user endpoint error path
• Lines 377-396: Delete user endpoint paths

---

3. app/api/routes/sessions.py - Priority: MEDIUM

• Coverage: 49% (33/68 lines)
• Missing Lines: 35

Missing Coverage Areas:

Lines 69-106    : List sessions (auth header parsing, response building, error)
Lines 149-183   : Revoke session (NotFound, auth check, success, errors)
Lines 226-236   : Cleanup sessions (success logging, error + rollback)

Existing Tests: Comprehensive test suite already exists in test_sessions.py

• 4 test classes with ~30 tests
• Tests appear complete but coverage not being recorded

Recommended Actions:

1. Verify test execution is actually hitting the routes
2. Check if rate limiting is affecting coverage
3. Re-run with coverage HTML to visualize hit/miss lines

---

4. app/api/routes/organizations.py - Priority: HIGH

• Coverage: 35% (23/66 lines)
• Missing Lines: 43

Missing Coverage Areas:

Lines 54-83     : List organizations endpoint (entire function)
Lines 103-128   : Get organization by ID (entire function)
Lines 150-172   : Add member to organization (entire function)
Lines 193-221   : Remove member from organization (entire function)

Status: NO TESTS EXIST for this file

Required Tests:

1. List user's organizations (with/without filters)
2. Get organization by ID (success + NotFound)
3. Add member to organization (success, already member, permission errors)
4. Remove member from organization (success, not a member, permission errors)

Estimated Effort: 12-15 tests needed

---

5. app/crud/organization.py - Priority: MEDIUM

• Coverage: 80% (160/201 lines)
• Missing Lines: 41

Missing Coverage Areas:

Lines 33-35     : Create organization ValueError exception
Lines 57-62     : Create organization general Exception + rollback
Lines 114-116   : Update organization exception handling
Lines 130-132   : Update organization rollback
Lines 207-209   : Delete organization (remove) exception
Lines 258-260   : Add user ValueError (already member)
Lines 291-294   : Add user Exception + rollback
Lines 326-329   : Remove user Exception + rollback
Lines 385-387   : Update user role ValueError
Lines 409-411   : Update user role Exception + rollback
Lines 466-468   : Get organization members Exception
Lines 491-493   : Get member count Exception

Pattern: All missing lines are exception handlers

Required Tests:

• Mock database errors for each CRUD operation
• Test ValueError paths (business logic violations)
• Test Exception paths (unexpected errors)
• Verify rollback is called on failures

Estimated Effort: 12 tests to cover all exception paths

---

6. app/crud/base.py - Priority: MEDIUM

• Coverage: 73% (164/224 lines)
• Missing Lines: 60

Missing Coverage Areas:

Lines 77-78     : Get method exception handling
Lines 119-120   : Get multi exception handling
Lines 130-152   : Get multi with filters (complex filtering logic)
Lines 254-296   : Get multi with total (pagination, sorting, filtering, search)
Lines 342-343   : Update method exception handling
Lines 383-384   : Remove method exception handling

Key Uncovered Features:

• Advanced filtering with filters parameter
• Sorting functionality (sort_by, sort_order)
• Search across multiple fields
• Pagination parameter validation

Required Tests:

1. Test filtering with various field types
2. Test sorting (ASC/DESC, different fields)
3. Test search across text fields
4. Test pagination edge cases (negative skip, limit > 1000)
5. Test exception handlers for all methods

Estimated Effort: 15-20 tests

---

7. app/api/dependencies/permissions.py - Priority: MEDIUM

• Coverage: 53% (23/43 lines)
• Missing Lines: 20

Missing Coverage Areas:

Lines 52-57     : Organization owner check (NotFound, success)
Lines 98-120    : Organization admin check (multiple error paths)
Lines 154-157   : Organization member check NotFoundError
Lines 174-189   : Can manage member check (permission logic)

Status: Limited testing of permission dependencies

Required Tests:

1. Test each permission level: owner, admin, member
2. Test permission denials
3. Test with non-existent organizations
4. Test with users not in organization

Estimated Effort: 12-15 tests

---

8. app/init_db.py - Priority: LOW

• Coverage: 72% (29/40 lines)
• Missing Lines: 11

Missing Coverage Areas:

Lines 71-88     : Initialize database (create tables, seed superuser)

Note: This is initialization code that runs once. May not need testing if it's manual/setup code.

Recommended: Either test or exclude from coverage with # pragma: no cover

---

9. app/core/auth.py - Priority: LOW

• Coverage: 93% (53/57 lines)
• Missing Lines: 4

Missing Coverage Areas:

Lines 151       : decode_token exception path
Lines 209, 212  : refresh_token_response edge cases
Lines 232       : verify_password constant-time comparison path

Status: Already excellent coverage, minor edge cases remain

---

10. app/schemas/validators.py - Priority: MEDIUM

• Coverage: 62% (16/26 lines)
• Missing Lines: 10

Missing Coverage Areas:

Lines 115       : Phone number validation edge case
Lines 119       : Phone number regex validation
Lines 148       : Password validation edge case
Lines 170-183   : Password strength validation (length, uppercase, lowercase, digit, special)

Required Tests:

1. Invalid phone numbers (wrong format, too short, etc.)
2. Weak passwords (missing uppercase, digits, special chars)
3. Edge cases (empty strings, None values)

Estimated Effort: 8-10 tests

---

---

UPDATED Path to 95% Coverage (Post-Fix)

Current State: 88% → Target: 95% (Need to cover ~175 more lines)

Breakdown by Priority:

File	Current	Missing Lines	Priority	Estimated Tests Needed
organizations.py (routes)	35%	43	CRITICAL	12-15 tests
base.py (crud)	73%	60	HIGH	15-20 tests
organization.py (crud)	80%	41	MEDIUM	12 tests
permissions.py (deps)	53%	20	MEDIUM	12-15 tests
main.py	80%	16	LOW	5-8 tests
database.py (core)	78%	14	LOW	5-8 tests
validators.py (schemas)	62%	10	LOW	8-10 tests

Quick Win Strategy (Estimated 15-20 hours):

1. Phase 1 (5h): Create tests/api/test_organizations.py → +43 lines (+1.8%)
2. Phase 2 (6h): Test base CRUD advanced features → +60 lines (+2.4%)
3. Phase 3 (4h): Test organization CRUD exceptions → +41 lines (+1.7%)
4. Phase 4 (3h): Test permission dependencies → +20 lines (+0.8%)
5. Phase 5 (2h): Misc coverage (validators, database utils) → +20 lines (+0.8%)

Expected Result: 88% + 7.5% = 95.5% ✅

---

Path to 95% Coverage (Historical - Pre-Fix)

Recommended Prioritization

Phase 1: Fix Coverage Tracking (CRITICAL)

Estimated Time: 2-4 hours

1. Investigate pytest-cov configuration:

   # Try different coverage modes
   pytest --cov=app --cov-report=html -n 0
   pytest --cov=app --cov-report=term-missing --no-cov-on-fail
   

2. Generate HTML coverage report:

   IS_TEST=True pytest --cov=app --cov-report=html -n 0
   open htmlcov/index.html
   

3. Verify route tests are actually running:

   • Add debug logging to route handlers
   • Check if mocking is preventing actual code execution
   • Verify dependency overrides are working

4. Consider coverage configuration changes:

   • Update .coveragerc to use source-based coverage
   • Disable xdist for coverage runs (use -n 0)
   • Try coverage run instead of pytest --cov

Phase 2: Test Organization Routes (HIGH IMPACT)

Estimated Time: 3-4 hours Coverage Gain: ~43 lines (1.8%)

Create tests/api/test_organizations.py with:

• List organizations endpoint
• Get organization endpoint
• Add member endpoint
• Remove member endpoint

Phase 3: Test Organization CRUD Exceptions (MEDIUM IMPACT)

Estimated Time: 2-3 hours Coverage Gain: ~41 lines (1.7%)

Enhance tests/crud/test_organization.py with:

• Mock database errors for all CRUD operations
• Test ValueError paths
• Verify rollback calls

Phase 4: Test Base CRUD Advanced Features (MEDIUM IMPACT)

Estimated Time: 4-5 hours Coverage Gain: ~60 lines (2.5%)

Enhance tests/crud/test_base.py with:

• Complex filtering tests
• Sorting tests (ASC/DESC)
• Search functionality tests
• Pagination validation tests

Phase 5: Test Permission Dependencies (MEDIUM IMPACT)

Estimated Time: 2-3 hours Coverage Gain: ~20 lines (0.8%)

Create comprehensive permission tests for all roles.

Phase 6: Test Validators (LOW IMPACT)

Estimated Time: 1-2 hours Coverage Gain: ~10 lines (0.4%)

Test phone and password validation edge cases.

Phase 7: Review and Exclude Untestable Code (LOW IMPACT)

Estimated Time: 1 hour Coverage Gain: ~11 lines (0.5%)

Mark initialization and setup code with # pragma: no cover.

---

Summary of Potential Coverage Gains

Phase	Target	Lines	Coverage Gain	Cumulative
Current	-	1,932	79.0%	79.0%
Fix Tracking	Admin routes	+100	+4.1%	83.1%
Fix Tracking	Sessions routes	+35	+1.4%	84.5%
Fix Tracking	Users routes	+20	+0.8%	85.3%
Phase 2	Organizations routes	+43	+1.8%	87.1%
Phase 3	Organization CRUD	+41	+1.7%	88.8%
Phase 4	Base CRUD	+60	+2.5%	91.3%
Phase 5	Permissions	+20	+0.8%	92.1%
Phase 6	Validators	+10	+0.4%	92.5%
Phase 7	Exclusions	+11	+0.5%	93.0%

Total Potential: 93% coverage (achievable) With Admin Fix: Could reach 95%+ if coverage tracking is resolved

---

Critical Action Items (UPDATED)

✅ Completed

1. ✅ RESOLVED: Coverage tracking issue - Added concurrency = thread,greenlet to .coveragerc
2. ✅ Generated HTML coverage report - Visualized actual vs missing coverage
3. ✅ Ran coverage in single-process mode - Confirmed xdist was not the issue
4. ✅ Achieved 88% coverage - Up from 79% (+9 percentage points)

High Priority (Path to 95%)

1. ⬜ Create organization routes tests - Highest uncovered file (35%, 43 lines missing)

   • Estimated: 12-15 tests, 5 hours
   • Impact: +1.8% coverage

2. ⬜ Test base CRUD advanced features - Foundation for all CRUD operations (73%, 60 lines)

   • Estimated: 15-20 tests, 6 hours
   • Impact: +2.4% coverage

3. ⬜ Complete organization CRUD exception tests - Exception handling (80%, 41 lines)

   • Estimated: 12 tests, 4 hours
   • Impact: +1.7% coverage

Medium Priority

4. ⬜ Test permission dependencies thoroughly - Security-critical (53%, 20 lines)
   • Estimated: 12-15 tests, 3 hours
   • Impact: +0.8% coverage

Low Priority

5. ⬜ Miscellaneous coverage - Validators, database utils, main.py (~40 lines total)
   • Estimated: 15-20 tests, 2 hours
   • Impact: +1.6% coverage

---

Known Issues and Blockers (UPDATED)

✅ RESOLVED: Coverage Not Being Recorded for Routes

Problem: Coverage.py was not tracking async code execution through httpx's ASGITransport

Solution: Added concurrency = thread,greenlet to .coveragerc

Result: Coverage jumped from 79% → 88%, with route files now properly tracked:

• admin.py: 46% → 98%
• auth.py: 79% → 95%
• sessions.py: 49% → 84%
• users.py: 60% → 93%

Remaining Issue: Dead Code in users.py

Issue: Lines 150-154 and 270-275 check for is_superuser field in UserUpdate, but the schema doesn't include this field.

Solution: ✅ Marked with # pragma: no cover

Recommendation: Remove dead code or add is_superuser to UserUpdate schema with proper validation.

---

Test Files Status

Created/Enhanced in This Session

1. ✅ tests/api/test_admin_error_handlers.py - Added 20 success path tests
2. ✅ tests/api/test_users.py - Added 10 tests, improved 58% → 63%
3. ✅ app/api/routes/users.py - Marked dead code with pragma

Existing Comprehensive Tests

1. ✅ tests/api/test_sessions.py - Excellent coverage (but not recorded)
2. ✅ tests/crud/test_session_db_failures.py - 100% session CRUD coverage
3. ✅ tests/crud/test_base_db_failures.py - Base CRUD exception handling

Missing Test Files

1. ⬜ tests/api/test_organizations.py - NEEDS CREATION
2. ⬜ Enhanced tests/crud/test_organization.py - Needs exception tests
3. ⬜ Enhanced tests/crud/test_base.py - Needs advanced feature tests
4. ⬜ tests/api/test_permissions.py - NEEDS CREATION
5. ⬜ tests/schemas/test_validators.py - NEEDS CREATION

---

Recommendations

Short Term (This Week)

1. Fix coverage tracking - Highest priority blocker
2. Create organization routes tests - Biggest gap
3. Test organization CRUD exceptions - Quick win

Medium Term (Next Sprint)

1. Comprehensive base CRUD testing - Foundation for all operations
2. Permission dependency tests - Security critical
3. Validator tests - Data integrity

Long Term (Future)

1. Consider integration tests - End-to-end workflows
2. Performance testing - Load testing critical paths
3. Security testing - Penetration testing, SQL injection, XSS

---

Conclusion (UPDATED)

✅ Coverage tracking issue RESOLVED! Coverage improved from 79% → 88% by adding concurrency = thread,greenlet to .coveragerc.

Current coverage is 88% with a clear path to 95%+ through systematic testing of:

1. Organization routes (43 lines)
2. Base CRUD advanced features (60 lines)
3. Organization CRUD exceptions (41 lines)
4. Permission dependencies (20 lines)
5. Misc utilities (40 lines)

Key Success Factors:

1. ✅ RESOLVED: pytest-cov tracking issue (+9% coverage)
2. Test organization module (highest remaining gap)
3. Exception path testing (low-hanging fruit)
4. Advanced CRUD feature testing (pagination, filtering, search)

Estimated Timeline to 95%:

• 15-20 hours of focused work across 5 phases
• Can be completed in 2-3 days with dedicated effort
• Most impactful: Phase 1 (organization routes) and Phase 2 (base CRUD)

---

References

Original Report (2025-11-01):

• Coverage: 79% (2,439 statements, 507 missing)
• Test count: 596 passing
• Issue: Coverage not tracking async routes

Updated Report (2025-11-02):

• Coverage: 88% (2,455 statements, 298 missing) ✅
• Test count: 598 passing
• Fix Applied: concurrency = thread,greenlet in .coveragerc
• Coverage improvement: +9 percentage points (+225 lines)
• Major improvements:
  • admin.py: 46% → 98% (+52 points)
  • auth.py: 79% → 95% (+16 points)
  • sessions.py: 49% → 84% (+35 points)
  • users.py: 60% → 93% (+33 points)