cardosofelipe/fast-next-template
Template
1
0
Fork 1
Code Issues 1 Pull Requests Packages Projects Releases Wiki Activity

Update coverage report with resolved tracking issue and 88% overall coverage

Browse Source 
Resolved `pytest-cov` tracking for async routes by adjusting `.coveragerc` to include `greenlet` concurrency. Coverage improved from 79% to 88%, with significant gains across key modules like `admin.py` (46% → 98%). Updated details on coverage gaps and priorities for reaching 95% target.
This commit is contained in:
Felipe Cardoso
2025-11-02 05:27:24 +01:00
parent 1aab73cb72
commit 406c3bcc82
1 changed files with 163 additions and 75 deletions
Download Patch File Download Diff File Expand all files Collapse all files

238
backend/docs/COVERAGE_REPORT.md
View File

@@ -1,56 +1,96 @@
# Test Coverage Analysis Report
**Date**: 2025-11-01
**Current Coverage**: 79% (1,932/2,439 lines)
**Date**: 2025-11-02 (Updated)
**Current Coverage**: 88% (2,157/2,455 lines)
**Previous Coverage**: 79% (1,932/2,439 lines)
**Target Coverage**: 95%
**Gap**: 270 lines needed to reach 90%, ~390 lines for 95%
**Gap**: ~175 lines needed to reach 95%
## Executive Summary
This report documents the current state of test coverage, identified issues with coverage tracking, and actionable paths to reach the 95% coverage target.
This report documents the **successful resolution** of the coverage tracking issue and the path to reach the 95% coverage target.
### Current Status
- **Total Tests**: 596 passing
- **Overall Coverage**: 79%
- **Lines Covered**: 1,932 / 2,439
- **Lines Missing**: 507
- **Total Tests**: 598 passing ✅
- **Overall Coverage**: 88% (up from 79%)
- **Lines Covered**: 2,157 / 2,455
- **Lines Missing**: 298 (down from 507)
- **Improvement**: +9 percentage points (+225 lines covered)
### Key Finding: Coverage Tracking Issue
### ✅ RESOLVED: Coverage Tracking Issue
**Critical Issue Identified**: Pytest-cov is not properly recording coverage for FastAPI route files when tests are executed, despite:
1. Tests passing successfully (596/596 ✓)
**Problem**: Pytest-cov was not properly recording coverage for FastAPI route files executed through httpx's `ASGITransport`, despite:
1. Tests passing successfully (598/598 ✓)
2. Manual verification showing code paths ARE being executed
3. Correct responses being returned from endpoints
**Root Cause**: Suspected interaction between pytest-cov, pytest-xdist (parallel execution), and the FastAPI async test client causing coverage data to not be collected for certain modules.
**Root Cause Identified**: Coverage.py was not configured to track async code execution through ASGI transport's greenlet-based concurrency model.
**Evidence**:
```bash
# Running with xdist shows "Module was never imported" warning
pytest --cov=app/api/routes/admin --cov-report=term-missing
# Warning: Module app/api/routes/admin was never imported
# Warning: No data was collected
**Solution**: Added `concurrency = thread,greenlet` to `.coveragerc`
```ini
[run]
source = app
concurrency = thread,greenlet # ← THIS WAS THE FIX!
omit = ...
```
## Detailed Coverage Breakdown
**Results After Fix**:
- **admin.py**: 46% → **98%** (+52 points!)
- **auth.py**: 79% → **95%** (+16 points)
- **sessions.py**: 49% → **84%** (+35 points)
- **users.py**: 60% → **93%** (+33 points)
- **Overall**: 79% → **88%** (+9 points)
### Files with Complete Coverage (100%) ✓
- `app/crud/session.py`
- `app/utils/security.py`
- `app/schemas/sessions.py`
- `app/utils/device.py` (97%)
- 12 other files with 100% coverage
## Detailed Coverage Breakdown (Post-Fix)
### Files Requiring Coverage Improvement
### Files with Excellent Coverage (95%+) ✅
- **app/crud/session.py**: 100%
- **app/utils/security.py**: 100%
- **app/schemas/sessions.py**: 100%
- **app/schemas/errors.py**: 100%
- **app/services/email_service.py**: 100%
- **app/services/session_cleanup.py**: 100%
- **app/api/main.py**: 100%
- **app/api/routes/admin.py**: **98%** (was 46%!)
- **app/core/config.py**: 98%
- **app/schemas/common.py**: 97%
- **app/utils/device.py**: 97%
- **app/auth.py**: 95%
- **app/core/exceptions.py**: 95%
#### 1. **app/api/routes/admin.py** - Priority: HIGH
- **Coverage**: 46% (118/259 lines)
- **Missing Lines**: 141
- **Impact**: Largest single coverage gap
### Files Requiring Coverage Improvement (to reach 95%)
#### 1. **app/api/routes/organizations.py** - Priority: CRITICAL ⚠️
- **Coverage**: 35% (23/66 lines)
- **Missing Lines**: 43
- **Impact**: Largest remaining gap, NO TESTS EXIST
**Missing Coverage Areas**:
```
Lines 109-116 : Pagination metadata creation (list users)
Lines 54-83 : List organizations endpoint (entire function)
Lines 103-128 : Get organization by ID (entire function)
Lines 150-172 : Add member to organization (entire function)
Lines 193-221 : Remove member from organization (entire function)
```
**Required Tests**: Create `tests/api/test_organizations.py` with ~12-15 tests
---
#### 2. **app/crud/base.py** - Priority: HIGH
- **Coverage**: 73% (164/224 lines)
- **Missing Lines**: 60
- **Impact**: Foundation class for all CRUD operations
**Missing Coverage Areas**:
```
Lines 77-78 : Exception handling in get()
Lines 119-120 : Exception handling in get_multi()
Lines 130-152 : Advanced filtering logic in get_multi()
Lines 254-296 : Pagination, sorting, filtering in get_multi_with_total()
Lines 342-343 : Exception handling in update()
Lines 383-384 : Exception handling in remove()
Lines 143-144 : User creation success logging
Lines 146-147 : User creation error handling (ValueError)
Lines 170-175 : Get user NotFoundError
@@ -297,7 +337,36 @@ Lines 170-183 : Password strength validation (length, uppercase, lowercase, di
---
## Path to 95% Coverage
---
## **UPDATED** Path to 95% Coverage (Post-Fix)
### Current State: 88% → Target: 95% (Need to cover ~175 more lines)
**Breakdown by Priority:**
| File | Current | Missing Lines | Priority | Estimated Tests Needed |
|------|---------|---------------|----------|------------------------|
| `organizations.py` (routes) | 35% | 43 | CRITICAL | 12-15 tests |
| `base.py` (crud) | 73% | 60 | HIGH | 15-20 tests |
| `organization.py` (crud) | 80% | 41 | MEDIUM | 12 tests |
| `permissions.py` (deps) | 53% | 20 | MEDIUM | 12-15 tests |
| `main.py` | 80% | 16 | LOW | 5-8 tests |
| `database.py` (core) | 78% | 14 | LOW | 5-8 tests |
| `validators.py` (schemas) | 62% | 10 | LOW | 8-10 tests |
**Quick Win Strategy** (Estimated 15-20 hours):
1. **Phase 1** (5h): Create `tests/api/test_organizations.py` → +43 lines (+1.8%)
2. **Phase 2** (6h): Test base CRUD advanced features → +60 lines (+2.4%)
3. **Phase 3** (4h): Test organization CRUD exceptions → +41 lines (+1.7%)
4. **Phase 4** (3h): Test permission dependencies → +20 lines (+0.8%)
5. **Phase 5** (2h): Misc coverage (validators, database utils) → +20 lines (+0.8%)
**Expected Result**: 88% + 7.5% = **95.5%** ✅
---
## Path to 95% Coverage (Historical - Pre-Fix)
### Recommended Prioritization
@@ -396,54 +465,54 @@ Mark initialization and setup code with `# pragma: no cover`.
---
## Critical Action Items
## Critical Action Items (UPDATED)
### Immediate (Do First)
1. ✅ **Investigate coverage tracking issue** - This is blocking accurate measurement
2. ✅ **Generate HTML coverage report** - Visual confirmation of what's actually covered
3. ✅ **Run coverage in single-process mode** - Eliminate xdist as variable
### ✅ Completed
1. ✅ **RESOLVED: Coverage tracking issue** - Added `concurrency = thread,greenlet` to `.coveragerc`
2. ✅ **Generated HTML coverage report** - Visualized actual vs missing coverage
3. ✅ **Ran coverage in single-process mode** - Confirmed xdist was not the issue
4. ✅ **Achieved 88% coverage** - Up from 79% (+9 percentage points)
### High Priority (Do Next)
4. ⬜ **Create organization routes tests** - Highest uncovered file (35%)
5. ⬜ **Complete organization CRUD exception tests** - Low-hanging fruit (80% → 95%+)
6. ⬜ **Test base CRUD advanced features** - Foundation for all CRUD operations
### High Priority (Path to 95%)
1. ⬜ **Create organization routes tests** - Highest uncovered file (35%, 43 lines missing)
- Estimated: 12-15 tests, 5 hours
- Impact: +1.8% coverage
2. ⬜ **Test base CRUD advanced features** - Foundation for all CRUD operations (73%, 60 lines)
- Estimated: 15-20 tests, 6 hours
- Impact: +2.4% coverage
3. ⬜ **Complete organization CRUD exception tests** - Exception handling (80%, 41 lines)
- Estimated: 12 tests, 4 hours
- Impact: +1.7% coverage
### Medium Priority
7. ⬜ **Test permission dependencies thoroughly** - Important for security
8. ⬜ **Complete validator tests** - Data integrity
4. ⬜ **Test permission dependencies thoroughly** - Security-critical (53%, 20 lines)
- Estimated: 12-15 tests, 3 hours
- Impact: +0.8% coverage
### Low Priority
9. ⬜ **Review init_db.py** - Consider excluding setup code
10. ⬜ **Test auth.py edge cases** - Already 93%
5. ⬜ **Miscellaneous coverage** - Validators, database utils, main.py (~40 lines total)
- Estimated: 15-20 tests, 2 hours
- Impact: +1.6% coverage
---
## Known Issues and Blockers
## Known Issues and Blockers (UPDATED)
### 1. Coverage Not Being Recorded for Routes
**Symptoms**:
- Tests pass: 596/596 ✓
- Endpoints return correct data (manually verified)
- Coverage shows 46% for admin.py despite 20+ tests
### ✅ RESOLVED: Coverage Not Being Recorded for Routes
**Attempted Solutions**:
- ✅ Added tests for all missing line ranges
- ✅ Verified tests execute and pass
- ✅ Manually confirmed endpoints work
- ⬜ Need to investigate pytest-cov configuration
**Problem**: Coverage.py was not tracking async code execution through httpx's ASGITransport
**Hypothesis**:
- FastAPI async test client may not be compatible with pytest-cov's default tracing
- xdist parallel execution interferes with coverage collection
- Dependency overrides may hide actual route execution from coverage
**Solution**: Added `concurrency = thread,greenlet` to `.coveragerc`
**Next Steps**:
1. Run with `-n 0` (single process)
2. Try `--cov-branch` for branch coverage
3. Use coverage HTML report to visualize
4. Consider using `coverage run -m pytest` directly
**Result**: Coverage jumped from 79% → 88%, with route files now properly tracked:
- admin.py: 46% → 98%
- auth.py: 79% → 95%
- sessions.py: 49% → 84%
- users.py: 60% → 93%
### 2. Dead Code in users.py
### Remaining Issue: Dead Code in users.py
**Issue**: Lines 150-154 and 270-275 check for `is_superuser` field in `UserUpdate`, but the schema doesn't include this field.
**Solution**: ✅ Marked with `# pragma: no cover`
@@ -492,25 +561,44 @@ Mark initialization and setup code with `# pragma: no cover`.
---
## Conclusion
## Conclusion (UPDATED)
Current coverage is **79%** with a path to **93%+** through systematic testing. The primary blocker is the coverage tracking issue with route tests - once resolved, coverage should jump significantly. With focused effort on organization routes, CRUD operations, and permission testing, the 95% goal is achievable within 20-30 hours of dedicated work.
✅ **Coverage tracking issue RESOLVED!** Coverage improved from **79% → 88%** by adding `concurrency = thread,greenlet` to `.coveragerc`.
Current coverage is **88%** with a clear path to **95%+** through systematic testing of:
1. Organization routes (43 lines)
2. Base CRUD advanced features (60 lines)
3. Organization CRUD exceptions (41 lines)
4. Permission dependencies (20 lines)
5. Misc utilities (40 lines)
**Key Success Factors**:
1. Resolve pytest-cov tracking issue (blocks 5-10% coverage)
2. Test organization module (highest gap)
1. ✅ **RESOLVED**: pytest-cov tracking issue (+9% coverage)
2. Test organization module (highest remaining gap)
3. Exception path testing (low-hanging fruit)
4. Advanced CRUD feature testing (pagination, filtering, search)
**Estimated Timeline to 95%**:
- With coverage fix: 2-3 days of focused work
- Without coverage fix: 4-5 days (includes investigation)
- **15-20 hours of focused work** across 5 phases
- Can be completed in **2-3 days** with dedicated effort
- Most impactful: Phase 1 (organization routes) and Phase 2 (base CRUD)
---
## References
- Coverage run output: `TOTAL 2439 507 79%`
**Original Report** (2025-11-01):
- Coverage: 79% (2,439 statements, 507 missing)
- Test count: 596 passing
- Tests added this session: 30+
- Coverage improvement: 58% → 63% (users.py)
- Issue: Coverage not tracking async routes
**Updated Report** (2025-11-02):
- Coverage: **88%** (2,455 statements, 298 missing) ✅
- Test count: **598 passing**
- **Fix Applied**: `concurrency = thread,greenlet` in `.coveragerc`
- Coverage improvement: **+9 percentage points (+225 lines)**
- Major improvements:
- admin.py: 46% → 98% (+52 points)
- auth.py: 79% → 95% (+16 points)
- sessions.py: 49% → 84% (+35 points)
- users.py: 60% → 93% (+33 points)