forked from cardosofelipe/fast-next-template
Delete outdated E2E documentation and performance optimization guides.
- Removed `E2E_COVERAGE_GUIDE.md` and `E2E_PERFORMANCE_OPTIMIZATION.md` from `frontend/docs` due to redundancy and irrelevance to recent workflows. - Cleared unused scripts (`convert-v8-to-istanbul.ts` and `merge-coverage.ts`) from `frontend/scripts`.
This commit is contained in:
Felipe Cardoso
@@ -1,26 +0,0 @@
|
||||
{
|
||||
"all": true,
|
||||
"include": ["src/**/*.{js,jsx,ts,tsx}"],
|
||||
"exclude": [
|
||||
"src/**/*.d.ts",
|
||||
"src/**/*.test.{js,jsx,ts,tsx}",
|
||||
"src/**/__tests__/**",
|
||||
"src/**/*.stories.{js,jsx,ts,tsx}",
|
||||
"src/lib/api/generated/**",
|
||||
"src/**/*.old.{js,jsx,ts,tsx}",
|
||||
"src/components/ui/**",
|
||||
"src/app/dev/**",
|
||||
"src/**/index.{js,jsx,ts,tsx}",
|
||||
"src/lib/utils/cn.ts",
|
||||
"src/middleware.ts"
|
||||
],
|
||||
"reporter": ["text", "text-summary", "html", "json", "lcov"],
|
||||
"report-dir": "./coverage-combined",
|
||||
"temp-dir": "./.nyc_output",
|
||||
"sourceMap": true,
|
||||
"instrument": true,
|
||||
"branches": 85,
|
||||
"functions": 85,
|
||||
"lines": 90,
|
||||
"statements": 90
|
||||
}
|
||||
@@ -1,651 +0,0 @@
|
||||
# E2E Coverage Integration Guide
|
||||
|
||||
This guide explains how to collect and merge E2E test coverage with unit test coverage to get a comprehensive view of your test coverage.
|
||||
|
||||
## 📋 Table of Contents
|
||||
|
||||
1. [Overview](#overview)
|
||||
2. [Quick Start](#quick-start)
|
||||
3. [Approach 1: V8 Coverage (Recommended)](#approach-1-v8-coverage-recommended)
|
||||
4. [Approach 2: Istanbul Instrumentation](#approach-2-istanbul-instrumentation)
|
||||
5. [Combined Coverage Workflow](#combined-coverage-workflow)
|
||||
6. [Integration Steps](#integration-steps)
|
||||
7. [Troubleshooting](#troubleshooting)
|
||||
8. [FAQ](#faq)
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
### Why Combined Coverage?
|
||||
|
||||
Your project uses a **dual testing strategy**:
|
||||
|
||||
- **Jest (Unit tests):** 97%+ coverage, excludes browser-specific code
|
||||
- **Playwright (E2E tests):** Tests excluded files (layouts, API hooks, error boundaries)
|
||||
|
||||
**Combined coverage** shows the full picture by merging both coverage sources.
|
||||
|
||||
### Current Exclusions from Jest
|
||||
|
||||
```javascript
|
||||
// From jest.config.js - These ARE tested by E2E:
|
||||
'!src/lib/api/hooks/**', // React Query hooks
|
||||
'!src/app/**/layout.tsx', // Next.js layouts
|
||||
'!src/app/**/error.tsx', // Error boundaries
|
||||
'!src/app/**/loading.tsx', // Loading states
|
||||
```
|
||||
|
||||
### Expected Results
|
||||
|
||||
```
|
||||
Unit test coverage: 97.19% (excluding above)
|
||||
E2E coverage: ~25-35% (user flows + excluded files)
|
||||
Combined coverage: 98-100% ✅
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Prerequisites
|
||||
|
||||
All infrastructure is already created! Just need dependencies:
|
||||
|
||||
```bash
|
||||
# Option 1: V8 Coverage (Chromium only, no instrumentation)
|
||||
npm install -D v8-to-istanbul istanbul-lib-coverage istanbul-lib-report istanbul-reports
|
||||
|
||||
# Option 2: Istanbul Instrumentation (all browsers)
|
||||
npm install -D @istanbuljs/nyc-config-typescript babel-plugin-istanbul nyc \
|
||||
istanbul-lib-coverage istanbul-lib-report istanbul-reports
|
||||
```
|
||||
|
||||
### Add Package Scripts
|
||||
|
||||
Add to `package.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"scripts": {
|
||||
"coverage:convert": "tsx scripts/convert-v8-to-istanbul.ts",
|
||||
"coverage:merge": "tsx scripts/merge-coverage.ts",
|
||||
"coverage:combined": "npm run test:coverage && E2E_COVERAGE=true npm run test:e2e && npm run coverage:convert && npm run coverage:merge",
|
||||
"coverage:view": "open coverage-combined/index.html"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Run Combined Coverage
|
||||
|
||||
```bash
|
||||
# Full workflow (unit + E2E + merge)
|
||||
npm run coverage:combined
|
||||
|
||||
# View HTML report
|
||||
npm run coverage:view
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Approach 1: V8 Coverage (Recommended)
|
||||
|
||||
### Pros & Cons
|
||||
|
||||
**Pros:**
|
||||
|
||||
- ✅ Native browser coverage (most accurate)
|
||||
- ✅ No build instrumentation needed (faster)
|
||||
- ✅ Works with source maps
|
||||
- ✅ Zero performance overhead
|
||||
|
||||
**Cons:**
|
||||
|
||||
- ❌ Chromium only (V8 engine specific)
|
||||
- ❌ Requires v8-to-istanbul conversion
|
||||
|
||||
### Setup Steps
|
||||
|
||||
#### 1. Install Dependencies
|
||||
|
||||
```bash
|
||||
npm install -D v8-to-istanbul istanbul-lib-coverage istanbul-lib-report istanbul-reports
|
||||
```
|
||||
|
||||
#### 2. Integrate into E2E Tests
|
||||
|
||||
Update your E2E test files to use coverage helpers:
|
||||
|
||||
```typescript
|
||||
// e2e/homepage.spec.ts
|
||||
import { test, expect } from '@playwright/test';
|
||||
import { withCoverage } from './helpers/coverage';
|
||||
|
||||
test.describe('Homepage Tests', () => {
|
||||
test.beforeEach(async ({ page }) => {
|
||||
// Start coverage collection
|
||||
await withCoverage.start(page);
|
||||
await page.goto('/');
|
||||
});
|
||||
|
||||
test.afterEach(async ({ page }, testInfo) => {
|
||||
// Stop and save coverage
|
||||
await withCoverage.stop(page, testInfo.title);
|
||||
});
|
||||
|
||||
test('displays header', async ({ page }) => {
|
||||
await expect(page.getByRole('heading')).toBeVisible();
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
#### 3. Run E2E Tests with Coverage
|
||||
|
||||
```bash
|
||||
E2E_COVERAGE=true npm run test:e2e
|
||||
```
|
||||
|
||||
This generates: `coverage-e2e/raw/*.json` (V8 format)
|
||||
|
||||
#### 4. Convert V8 to Istanbul
|
||||
|
||||
```bash
|
||||
npm run coverage:convert
|
||||
```
|
||||
|
||||
This converts to: `coverage-e2e/.nyc_output/e2e-coverage.json` (Istanbul format)
|
||||
|
||||
#### 5. Merge with Jest Coverage
|
||||
|
||||
```bash
|
||||
npm run coverage:merge
|
||||
```
|
||||
|
||||
This generates: `coverage-combined/index.html`
|
||||
|
||||
---
|
||||
|
||||
## Approach 2: Istanbul Instrumentation
|
||||
|
||||
### Pros & Cons
|
||||
|
||||
**Pros:**
|
||||
|
||||
- ✅ Works on all browsers (Firefox, Safari, etc.)
|
||||
- ✅ Industry standard tooling
|
||||
- ✅ No conversion needed
|
||||
|
||||
**Cons:**
|
||||
|
||||
- ❌ Requires code instrumentation (slower builds)
|
||||
- ❌ More complex setup
|
||||
- ❌ Slight test performance overhead
|
||||
|
||||
### Setup Steps
|
||||
|
||||
#### 1. Install Dependencies
|
||||
|
||||
```bash
|
||||
npm install -D @istanbuljs/nyc-config-typescript babel-plugin-istanbul \
|
||||
nyc istanbul-lib-coverage istanbul-lib-report istanbul-reports \
|
||||
@babel/core babel-loader
|
||||
```
|
||||
|
||||
#### 2. Configure Babel Instrumentation
|
||||
|
||||
Create `.babelrc.js`:
|
||||
|
||||
```javascript
|
||||
module.exports = {
|
||||
presets: ['next/babel'],
|
||||
env: {
|
||||
test: {
|
||||
plugins: [process.env.E2E_COVERAGE && 'istanbul'].filter(Boolean),
|
||||
},
|
||||
},
|
||||
};
|
||||
```
|
||||
|
||||
#### 3. Configure Next.js Webpack
|
||||
|
||||
Update `next.config.js`:
|
||||
|
||||
```javascript
|
||||
const nextConfig = {
|
||||
webpack: (config, { isServer }) => {
|
||||
// Add Istanbul instrumentation in E2E coverage mode
|
||||
if (process.env.E2E_COVERAGE && !isServer) {
|
||||
config.module.rules.push({
|
||||
test: /\.(js|jsx|ts|tsx)$/,
|
||||
exclude: /node_modules/,
|
||||
use: {
|
||||
loader: 'babel-loader',
|
||||
options: {
|
||||
presets: ['next/babel'],
|
||||
plugins: ['istanbul'],
|
||||
},
|
||||
},
|
||||
});
|
||||
}
|
||||
return config;
|
||||
},
|
||||
};
|
||||
|
||||
module.exports = nextConfig;
|
||||
```
|
||||
|
||||
#### 4. Integrate into E2E Tests
|
||||
|
||||
Use the Istanbul helper instead:
|
||||
|
||||
```typescript
|
||||
import { test, expect } from '@playwright/test';
|
||||
import { saveIstanbulCoverage } from './helpers/coverage';
|
||||
|
||||
test.describe('Homepage Tests', () => {
|
||||
test.afterEach(async ({ page }, testInfo) => {
|
||||
await saveIstanbulCoverage(page, testInfo.title);
|
||||
});
|
||||
|
||||
test('my test', async ({ page }) => {
|
||||
await page.goto('/');
|
||||
// Test code...
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
#### 5. Run Tests
|
||||
|
||||
```bash
|
||||
# Start dev server with instrumentation
|
||||
E2E_COVERAGE=true npm run dev
|
||||
|
||||
# In another terminal, run E2E tests
|
||||
E2E_COVERAGE=true npm run test:e2e
|
||||
```
|
||||
|
||||
#### 6. Merge Coverage
|
||||
|
||||
```bash
|
||||
npm run coverage:merge
|
||||
```
|
||||
|
||||
No conversion step needed! Istanbul coverage goes directly to `.nyc_output/`.
|
||||
|
||||
---
|
||||
|
||||
## Combined Coverage Workflow
|
||||
|
||||
### Full Workflow Diagram
|
||||
|
||||
```
|
||||
┌─────────────────────┐
|
||||
│ Jest Unit Tests │
|
||||
│ npm run test:cov │
|
||||
└──────────┬──────────┘
|
||||
│
|
||||
v
|
||||
coverage/coverage-final.json
|
||||
│
|
||||
├─────────────────────┐
|
||||
│ │
|
||||
v v
|
||||
┌─────────────────┐ ┌──────────────────┐
|
||||
│ E2E Tests │ │ E2E Tests │
|
||||
│ (V8 Coverage) │ │ (Istanbul) │
|
||||
└────────┬────────┘ └────────┬─────────┘
|
||||
│ │
|
||||
v v
|
||||
coverage-e2e/raw/*.json coverage-e2e/.nyc_output/*.json
|
||||
│ │
|
||||
v │
|
||||
scripts/convert-v8-to-istanbul.ts
|
||||
│ │
|
||||
v │
|
||||
coverage-e2e/.nyc_output/e2e-coverage.json
|
||||
│ │
|
||||
└──────────┬──────────┘
|
||||
v
|
||||
scripts/merge-coverage.ts
|
||||
│
|
||||
v
|
||||
coverage-combined/
|
||||
├── index.html
|
||||
├── lcov.info
|
||||
└── coverage-final.json
|
||||
```
|
||||
|
||||
### Commands Summary
|
||||
|
||||
```bash
|
||||
# 1. Run unit tests with coverage
|
||||
npm run test:coverage
|
||||
|
||||
# 2. Run E2E tests with coverage
|
||||
E2E_COVERAGE=true npm run test:e2e
|
||||
|
||||
# 3. Convert V8 to Istanbul (if using V8 approach)
|
||||
npm run coverage:convert
|
||||
|
||||
# 4. Merge all coverage
|
||||
npm run coverage:merge
|
||||
|
||||
# 5. View combined report
|
||||
npm run coverage:view
|
||||
|
||||
# OR: Do all at once
|
||||
npm run coverage:combined
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Integration Steps
|
||||
|
||||
### Phase 1: Pilot Integration (Single Test File)
|
||||
|
||||
Start with one E2E test file to verify the setup:
|
||||
|
||||
**File: `e2e/homepage.spec.ts`**
|
||||
|
||||
```typescript
|
||||
import { test, expect } from '@playwright/test';
|
||||
import { withCoverage } from './helpers/coverage';
|
||||
|
||||
test.describe('Homepage - Desktop Navigation', () => {
|
||||
test.beforeEach(async ({ page }) => {
|
||||
await withCoverage.start(page);
|
||||
await page.goto('/');
|
||||
await page.waitForLoadState('networkidle');
|
||||
});
|
||||
|
||||
test.afterEach(async ({ page }, testInfo) => {
|
||||
await withCoverage.stop(page, testInfo.title);
|
||||
});
|
||||
|
||||
test('should display header with logo and navigation', async ({ page }) => {
|
||||
await expect(page.getByRole('link', { name: /FastNext/i })).toBeVisible();
|
||||
await expect(page.getByRole('link', { name: 'Components' })).toBeVisible();
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**Test the pilot:**
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
npm install -D v8-to-istanbul istanbul-lib-coverage istanbul-lib-report istanbul-reports
|
||||
|
||||
# Run single test with coverage
|
||||
E2E_COVERAGE=true npx playwright test homepage.spec.ts
|
||||
|
||||
# Verify coverage files created
|
||||
ls coverage-e2e/raw/
|
||||
|
||||
# Convert and merge
|
||||
npm run coverage:convert
|
||||
npm run coverage:merge
|
||||
|
||||
# Check results
|
||||
npm run coverage:view
|
||||
```
|
||||
|
||||
### Phase 2: Rollout to All Tests
|
||||
|
||||
Once pilot works, update all 15 E2E spec files:
|
||||
|
||||
**Automated rollout script:**
|
||||
|
||||
```bash
|
||||
# Create a helper script: scripts/add-coverage-to-tests.sh
|
||||
#!/bin/bash
|
||||
|
||||
for file in e2e/*.spec.ts; do
|
||||
# Add import at top (if not already present)
|
||||
if ! grep -q "import.*coverage" "$file"; then
|
||||
sed -i "1i import { withCoverage } from './helpers/coverage';" "$file"
|
||||
fi
|
||||
|
||||
# Add beforeEach hook (manual review recommended)
|
||||
echo "Updated: $file"
|
||||
done
|
||||
```
|
||||
|
||||
**Or manually add to each file:**
|
||||
|
||||
1. Import coverage helper
|
||||
2. Add `beforeEach` with `withCoverage.start(page)`
|
||||
3. Add `afterEach` with `withCoverage.stop(page, testInfo.title)`
|
||||
|
||||
### Phase 3: CI/CD Integration
|
||||
|
||||
Add to your CI pipeline (e.g., `.github/workflows/test.yml`):
|
||||
|
||||
```yaml
|
||||
- name: Run E2E tests with coverage
|
||||
run: E2E_COVERAGE=true npm run test:e2e
|
||||
|
||||
- name: Convert E2E coverage
|
||||
run: npm run coverage:convert
|
||||
|
||||
- name: Merge coverage
|
||||
run: npm run coverage:merge
|
||||
|
||||
- name: Upload combined coverage
|
||||
uses: codecov/codecov-action@v3
|
||||
with:
|
||||
files: ./coverage-combined/lcov.info
|
||||
flags: combined
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Problem: No coverage files generated
|
||||
|
||||
**Symptoms:**
|
||||
|
||||
```bash
|
||||
npm run coverage:convert
|
||||
# ❌ No V8 coverage found at: coverage-e2e/raw
|
||||
```
|
||||
|
||||
**Solutions:**
|
||||
|
||||
1. Verify `E2E_COVERAGE=true` is set when running tests
|
||||
2. Check coverage helpers are imported: `import { withCoverage } from './helpers/coverage'`
|
||||
3. Verify `beforeEach` and `afterEach` hooks are added
|
||||
4. Check browser console for errors during test run
|
||||
|
||||
### Problem: V8 conversion fails
|
||||
|
||||
**Symptoms:**
|
||||
|
||||
```bash
|
||||
npm run coverage:convert
|
||||
# ❌ v8-to-istanbul not installed
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
|
||||
```bash
|
||||
npm install -D v8-to-istanbul
|
||||
```
|
||||
|
||||
### Problem: Coverage lower than expected
|
||||
|
||||
**Symptoms:**
|
||||
|
||||
```
|
||||
Combined: 85% (expected 99%)
|
||||
```
|
||||
|
||||
**Causes & Solutions:**
|
||||
|
||||
1. **E2E tests don't trigger all code paths**
|
||||
- Check which files are E2E-only: `npm run coverage:merge` shows breakdown
|
||||
- Add more E2E tests for uncovered scenarios
|
||||
|
||||
2. **Source maps not working**
|
||||
- Verify Next.js generates source maps: check `next.config.js`
|
||||
- Istanbul needs source maps to map coverage back to source
|
||||
|
||||
3. **Wrong files included**
|
||||
- Check `.nycrc.json` includes correct patterns
|
||||
- Verify excluded files match between Jest and NYC configs
|
||||
|
||||
### Problem: Istanbul coverage is empty
|
||||
|
||||
**Symptoms:**
|
||||
|
||||
```typescript
|
||||
await saveIstanbulCoverage(page, testName);
|
||||
// ⚠️ No Istanbul coverage found
|
||||
```
|
||||
|
||||
**Solutions:**
|
||||
|
||||
1. Verify `babel-plugin-istanbul` is configured
|
||||
2. Check `window.__coverage__` exists:
|
||||
```typescript
|
||||
const hasCoverage = await page.evaluate(() => !!(window as any).__coverage__);
|
||||
console.log('Istanbul available:', hasCoverage);
|
||||
```
|
||||
3. Ensure dev server started with `E2E_COVERAGE=true npm run dev`
|
||||
|
||||
### Problem: Merge script fails
|
||||
|
||||
**Symptoms:**
|
||||
|
||||
```bash
|
||||
npm run coverage:merge
|
||||
# ❌ Error: Cannot find module 'istanbul-lib-coverage'
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
|
||||
```bash
|
||||
npm install -D istanbul-lib-coverage istanbul-lib-report istanbul-reports
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## FAQ
|
||||
|
||||
### Q: Should I use V8 or Istanbul coverage?
|
||||
|
||||
**A: V8 coverage (Approach 1)** if:
|
||||
|
||||
- ✅ You only test in Chromium
|
||||
- ✅ You want zero instrumentation overhead
|
||||
- ✅ You want the most accurate coverage
|
||||
|
||||
**Istanbul (Approach 2)** if:
|
||||
|
||||
- ✅ You need cross-browser coverage
|
||||
- ✅ You already use Istanbul tooling
|
||||
- ✅ You need complex coverage transformations
|
||||
|
||||
### Q: Do I need to remove Jest exclusions?
|
||||
|
||||
**A: No!** Keep them. The `.nycrc.json` config handles combined coverage independently.
|
||||
|
||||
### Q: Will this slow down my tests?
|
||||
|
||||
**V8 Approach:** Minimal overhead (~5% slower)
|
||||
**Istanbul Approach:** Moderate overhead (~15-20% slower due to instrumentation)
|
||||
|
||||
### Q: Can I run coverage only for specific tests?
|
||||
|
||||
**Yes:**
|
||||
|
||||
```bash
|
||||
# Single file
|
||||
E2E_COVERAGE=true npx playwright test homepage.spec.ts
|
||||
|
||||
# Specific describe block
|
||||
E2E_COVERAGE=true npx playwright test --grep "Mobile Menu"
|
||||
```
|
||||
|
||||
### Q: How do I exclude files from E2E coverage?
|
||||
|
||||
Edit `.nycrc.json` and add to `exclude` array:
|
||||
|
||||
```json
|
||||
{
|
||||
"exclude": ["src/app/dev/**", "src/lib/utils/debug.ts"]
|
||||
}
|
||||
```
|
||||
|
||||
### Q: Can I see which lines are covered by E2E vs Unit tests?
|
||||
|
||||
Not directly in the HTML report, but you can:
|
||||
|
||||
1. Generate separate reports:
|
||||
|
||||
```bash
|
||||
npx nyc report --reporter=html --report-dir=coverage-unit --temp-dir=coverage/.nyc_output
|
||||
npx nyc report --reporter=html --report-dir=coverage-e2e-only --temp-dir=coverage-e2e/.nyc_output
|
||||
```
|
||||
|
||||
2. Compare the two reports to see differences
|
||||
|
||||
### Q: What's the performance impact on CI?
|
||||
|
||||
Typical impact:
|
||||
|
||||
- V8 coverage: +2-3 minutes (conversion time)
|
||||
- Istanbul coverage: +5-7 minutes (build instrumentation)
|
||||
- Merge step: ~10 seconds
|
||||
|
||||
Total CI time increase: **3-8 minutes**
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
### After Phase 1 (Infrastructure - DONE ✅)
|
||||
|
||||
You've completed:
|
||||
|
||||
- ✅ `.nycrc.json` configuration
|
||||
- ✅ Merge script (`scripts/merge-coverage.ts`)
|
||||
- ✅ Conversion script (`scripts/convert-v8-to-istanbul.ts`)
|
||||
- ✅ Coverage helpers (`e2e/helpers/coverage.ts`)
|
||||
- ✅ This documentation
|
||||
|
||||
### Phase 2: Activation (When Ready)
|
||||
|
||||
1. **Install dependencies:**
|
||||
|
||||
```bash
|
||||
npm install -D v8-to-istanbul istanbul-lib-coverage istanbul-lib-report istanbul-reports
|
||||
```
|
||||
|
||||
2. **Add package.json scripts** (see Quick Start)
|
||||
|
||||
3. **Test with one E2E file** (homepage.spec.ts recommended)
|
||||
|
||||
4. **Rollout to all E2E tests**
|
||||
|
||||
5. **Add to CI/CD pipeline**
|
||||
|
||||
### Expected Timeline
|
||||
|
||||
- **Phase 1:** ✅ Done (non-disruptive infrastructure)
|
||||
- **Phase 2:** ~1-2 hours (pilot + dependency installation)
|
||||
- **Rollout:** ~30 minutes (add hooks to 15 test files)
|
||||
- **CI integration:** ~20 minutes
|
||||
|
||||
---
|
||||
|
||||
## Additional Resources
|
||||
|
||||
- [Istanbul Coverage](https://istanbul.js.org/)
|
||||
- [NYC Configuration](https://github.com/istanbuljs/nyc#configuration-files)
|
||||
- [Playwright Coverage](https://playwright.dev/docs/api/class-coverage)
|
||||
- [V8 to Istanbul](https://github.com/istanbuljs/v8-to-istanbul)
|
||||
|
||||
---
|
||||
|
||||
**Questions or issues?** Check troubleshooting section or review the example in `e2e/helpers/coverage.ts`.
|
||||
@@ -1,313 +0,0 @@
|
||||
# E2E Test Performance Optimization Plan
|
||||
|
||||
**Current State**: 230 tests, ~2100 seconds total execution time
|
||||
**Target**: Reduce to <900 seconds (60% improvement)
|
||||
|
||||
## Bottleneck Analysis
|
||||
|
||||
### 1. Authentication Overhead (HIGHEST IMPACT)
|
||||
|
||||
**Problem**: Each test logs in fresh via UI
|
||||
|
||||
- **Impact**: 5-7s per test × 133 admin tests = ~700s wasted
|
||||
- **Root Cause**: Using `loginViaUI(page)` in every `beforeEach`
|
||||
|
||||
**Example of current slow pattern:**
|
||||
|
||||
```typescript
|
||||
test.beforeEach(async ({ page }) => {
|
||||
await setupSuperuserMocks(page);
|
||||
await loginViaUI(page); // ← 5-7s UI login EVERY test
|
||||
await page.goto('/admin');
|
||||
});
|
||||
```
|
||||
|
||||
**Solution: Playwright Storage State** (SAVE ~600-700s)
|
||||
|
||||
```typescript
|
||||
// auth.setup.ts - Run ONCE per worker
|
||||
import { test as setup } from '@playwright/test';
|
||||
|
||||
setup('authenticate as admin', async ({ page }) => {
|
||||
await setupSuperuserMocks(page);
|
||||
await loginViaUI(page);
|
||||
await page.context().storageState({ path: 'e2e/.auth/admin.json' });
|
||||
});
|
||||
|
||||
setup('authenticate as regular user', async ({ page }) => {
|
||||
await setupAuthenticatedMocks(page);
|
||||
await loginViaUI(page);
|
||||
await page.context().storageState({ path: 'e2e/.auth/user.json' });
|
||||
});
|
||||
```
|
||||
|
||||
```typescript
|
||||
// playwright.config.ts
|
||||
export default defineConfig({
|
||||
projects: [
|
||||
{ name: 'setup', testMatch: /.*\.setup\.ts/ },
|
||||
{
|
||||
name: 'admin tests',
|
||||
use: { storageState: 'e2e/.auth/admin.json' },
|
||||
dependencies: ['setup'],
|
||||
testMatch: /admin-.*\.spec\.ts/,
|
||||
},
|
||||
{
|
||||
name: 'user tests',
|
||||
use: { storageState: 'e2e/.auth/user.json' },
|
||||
dependencies: ['setup'],
|
||||
testMatch: /settings-.*\.spec\.ts/,
|
||||
},
|
||||
],
|
||||
});
|
||||
```
|
||||
|
||||
```typescript
|
||||
// admin-users.spec.ts - NO MORE loginViaUI!
|
||||
test.beforeEach(async ({ page }) => {
|
||||
// Auth already loaded from storageState
|
||||
await page.goto('/admin/users'); // ← Direct navigation, ~1-2s
|
||||
});
|
||||
```
|
||||
|
||||
**Expected Improvement**: 5-6s → 0.5-1s per test = **~600s saved** (133 tests × 5s)
|
||||
|
||||
---
|
||||
|
||||
### 2. Redundant Navigation Tests (MEDIUM IMPACT)
|
||||
|
||||
**Problem**: Separate tests for "navigate to X" and "display X page"
|
||||
|
||||
- **Impact**: 3-5s per redundant test × ~15 tests = ~60s wasted
|
||||
|
||||
**Current slow pattern:**
|
||||
|
||||
```typescript
|
||||
test('should navigate to users page', async ({ page }) => {
|
||||
await page.goto('/admin/users'); // 3s
|
||||
await expect(page).toHaveURL('/admin/users');
|
||||
await expect(page.locator('h1')).toContainText('User Management');
|
||||
});
|
||||
|
||||
test('should display user management page', async ({ page }) => {
|
||||
await page.goto('/admin/users'); // 3s DUPLICATE
|
||||
await expect(page.locator('h1')).toContainText('User Management');
|
||||
await expect(page.getByText(/manage users/i)).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
**Optimized pattern:**
|
||||
|
||||
```typescript
|
||||
test('should navigate to users page and display content', async ({ page }) => {
|
||||
await page.goto('/admin/users'); // 3s ONCE
|
||||
|
||||
// Navigation assertions
|
||||
await expect(page).toHaveURL('/admin/users');
|
||||
|
||||
// Content assertions
|
||||
await expect(page.locator('h1')).toContainText('User Management');
|
||||
await expect(page.getByText(/manage users/i)).toBeVisible();
|
||||
await expect(page.getByRole('button', { name: 'Create User' })).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
**Expected Improvement**: **~45-60s saved** (15 tests eliminated)
|
||||
|
||||
---
|
||||
|
||||
### 3. Flaky Test Fix (CRITICAL)
|
||||
|
||||
**Problem**: Test #218 failed once, passed on retry
|
||||
|
||||
```
|
||||
Test: settings-password.spec.ts:24:7 › Password Change › should display password change form
|
||||
Failed: 12.8s → Retry passed: 8.3s
|
||||
```
|
||||
|
||||
**Root Cause Options**:
|
||||
|
||||
1. Race condition in form rendering
|
||||
2. Slow network request not properly awaited
|
||||
3. Animation/transition timing issue
|
||||
|
||||
**Investigation needed:**
|
||||
|
||||
```typescript
|
||||
// Current test (lines 24-35)
|
||||
test('should display password change form', async ({ page }) => {
|
||||
await page.goto('/settings/password');
|
||||
|
||||
// ← Likely missing waitForLoadState or explicit wait
|
||||
await expect(page.getByLabel(/current password/i)).toBeVisible();
|
||||
await expect(page.getByLabel(/new password/i)).toBeVisible();
|
||||
await expect(page.getByLabel(/confirm password/i)).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
**Temporary Solution: Skip until fixed**
|
||||
|
||||
```typescript
|
||||
test.skip('should display password change form', async ({ page }) => {
|
||||
// TODO: Fix race condition (issue #XXX)
|
||||
await page.goto('/settings/password');
|
||||
await page.waitForLoadState('networkidle'); // ← Add this
|
||||
await expect(page.getByLabel(/current password/i)).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
**Expected Improvement**: Eliminate retry overhead + improve reliability
|
||||
|
||||
---
|
||||
|
||||
### 4. Optimize Wait Timeouts (LOW IMPACT)
|
||||
|
||||
**Problem**: Default timeout is 10s for all assertions
|
||||
|
||||
- **Impact**: Tests wait unnecessarily when elements load faster
|
||||
|
||||
**Current global timeout:**
|
||||
|
||||
```typescript
|
||||
// playwright.config.ts
|
||||
export default defineConfig({
|
||||
timeout: 30000, // Per test
|
||||
expect: { timeout: 10000 }, // Per assertion
|
||||
});
|
||||
```
|
||||
|
||||
**Optimized for fast-loading pages:**
|
||||
|
||||
```typescript
|
||||
export default defineConfig({
|
||||
timeout: 20000, // Reduce from 30s
|
||||
expect: { timeout: 5000 }, // Reduce from 10s (most elements load <2s)
|
||||
});
|
||||
```
|
||||
|
||||
**Expected Improvement**: **~100-150s saved** (faster failures, less waiting)
|
||||
|
||||
---
|
||||
|
||||
## Implementation Priority
|
||||
|
||||
### Phase 1: Quick Wins (1-2 hours work)
|
||||
|
||||
1. ✅ **Skip flaky test #218** temporarily
|
||||
2. ✅ **Reduce timeout defaults** (5s for expects, 20s for tests)
|
||||
3. ✅ **Combine 5 most obvious redundant navigation tests**
|
||||
|
||||
**Expected savings**: ~100-150s (5-7% improvement)
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Auth State Caching (2-4 hours work)
|
||||
|
||||
1. ✅ Create `e2e/auth.setup.ts` with storage state setup
|
||||
2. ✅ Update `playwright.config.ts` with projects + dependencies
|
||||
3. ✅ Remove `loginViaUI` from all admin test `beforeEach` hooks
|
||||
4. ✅ Update auth helper to support both mock + storageState modes
|
||||
|
||||
**Expected savings**: ~600-700s (30-35% improvement)
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Deep Optimization (4-8 hours work)
|
||||
|
||||
1. ✅ Investigate and fix flaky test root cause
|
||||
2. ✅ Audit all navigation tests for redundancy
|
||||
3. ✅ Combine related assertions (e.g., all stat cards in one test)
|
||||
4. ✅ Profile slowest 10 tests individually
|
||||
|
||||
**Expected savings**: ~150-200s (7-10% improvement)
|
||||
|
||||
---
|
||||
|
||||
## Total Expected Improvement
|
||||
|
||||
| Phase | Time Investment | Time Saved | % Improvement |
|
||||
| --------- | --------------- | ---------- | ------------- |
|
||||
| Phase 1 | 1-2 hours | ~150s | 7% |
|
||||
| Phase 2 | 2-4 hours | ~700s | 35% |
|
||||
| Phase 3 | 4-8 hours | ~200s | 10% |
|
||||
| **Total** | **7-14 hours** | **~1050s** | **50-60%** |
|
||||
|
||||
**Final target**: 2100s → 1050s = **~17-18 minutes** (currently ~35 minutes)
|
||||
|
||||
---
|
||||
|
||||
## Risks and Considerations
|
||||
|
||||
### Storage State Caching Risks:
|
||||
|
||||
1. **Test isolation**: Shared auth state could cause cross-test pollution
|
||||
- **Mitigation**: Use separate storage files per role, clear cookies between tests
|
||||
2. **Stale auth tokens**: Mock tokens might expire
|
||||
- **Mitigation**: Use long-lived test tokens (24h expiry)
|
||||
3. **Debugging difficulty**: Harder to debug auth issues
|
||||
- **Mitigation**: Keep `loginViaUI` tests for auth flow verification
|
||||
|
||||
### Recommended Safeguards:
|
||||
|
||||
```typescript
|
||||
// Clear non-auth state between tests
|
||||
test.beforeEach(async ({ page }) => {
|
||||
await page.goto('/admin');
|
||||
await page.evaluate(() => {
|
||||
// Clear localStorage except auth tokens
|
||||
const tokens = {
|
||||
access_token: localStorage.getItem('access_token'),
|
||||
refresh_token: localStorage.getItem('refresh_token'),
|
||||
};
|
||||
localStorage.clear();
|
||||
if (tokens.access_token) localStorage.setItem('access_token', tokens.access_token);
|
||||
if (tokens.refresh_token) localStorage.setItem('refresh_token', tokens.refresh_token);
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
**Immediate Actions (Do Now):**
|
||||
|
||||
1. Skip flaky test #218 with TODO comment
|
||||
2. Reduce timeout defaults in playwright.config.ts
|
||||
3. Create this optimization plan issue/ticket
|
||||
|
||||
**Short-term (This Week):**
|
||||
|
||||
1. Implement auth storage state (Phase 2)
|
||||
2. Combine obvious redundant tests (Phase 1)
|
||||
|
||||
**Medium-term (Next Sprint):**
|
||||
|
||||
1. Investigate flaky test root cause
|
||||
2. Audit all tests for redundancy
|
||||
3. Measure and report improvements
|
||||
|
||||
---
|
||||
|
||||
## Metrics to Track
|
||||
|
||||
Before optimization:
|
||||
|
||||
- Total time: ~2100s (35 minutes)
|
||||
- Avg test time: 9.1s
|
||||
- Slowest test: 20.1s (settings navigation)
|
||||
- Flaky tests: 1
|
||||
|
||||
After Phase 1+2 target:
|
||||
|
||||
- Total time: <1200s (20 minutes) ✅
|
||||
- Avg test time: <5.5s ✅
|
||||
- Slowest test: <12s ✅
|
||||
- Flaky tests: 0 ✅
|
||||
|
||||
After Phase 3 target:
|
||||
|
||||
- Total time: <1050s (17 minutes) 🎯
|
||||
- Avg test time: <4.8s 🎯
|
||||
- Slowest test: <10s 🎯
|
||||
- Flaky tests: 0 🎯
|
||||
@@ -21,9 +21,6 @@ test.describe('Password Change', () => {
|
||||
await page.getByLabel(/current password/i).waitFor({ state: 'visible' });
|
||||
});
|
||||
|
||||
// TODO: Fix flaky test - failed once at 12.8s, passed on retry at 8.3s
|
||||
// Likely race condition in form rendering or async state update
|
||||
// See: E2E_PERFORMANCE_OPTIMIZATION.md - Phase 3
|
||||
test.skip('should display password change form', async ({ page }) => {
|
||||
// Check page title
|
||||
await expect(page.getByRole('heading', { name: 'Password' })).toBeVisible();
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -1,211 +0,0 @@
|
||||
#!/usr/bin/env tsx
|
||||
/**
|
||||
* V8 to Istanbul Coverage Converter
|
||||
*
|
||||
* Converts Playwright's V8 coverage format to Istanbul format
|
||||
* so it can be merged with Jest coverage data.
|
||||
*
|
||||
* Usage:
|
||||
* npm run coverage:convert
|
||||
* # or directly:
|
||||
* tsx scripts/convert-v8-to-istanbul.ts
|
||||
*
|
||||
* Input:
|
||||
* - V8 coverage files in: ./coverage-e2e/raw/*.json
|
||||
* - Generated by Playwright's page.coverage API
|
||||
*
|
||||
* Output:
|
||||
* - Istanbul coverage in: ./coverage-e2e/.nyc_output/e2e-coverage.json
|
||||
* - Ready to merge with Jest coverage
|
||||
*
|
||||
* Prerequisites:
|
||||
* - npm install -D v8-to-istanbul
|
||||
* - E2E tests must collect coverage using page.coverage API
|
||||
*/
|
||||
|
||||
import fs from 'fs/promises';
|
||||
import path from 'path';
|
||||
import { fileURLToPath } from 'url';
|
||||
|
||||
// V8 coverage entry format
|
||||
interface V8CoverageEntry {
|
||||
url: string;
|
||||
scriptId: string;
|
||||
source?: string;
|
||||
functions: Array<{
|
||||
functionName: string;
|
||||
ranges: Array<{
|
||||
startOffset: number;
|
||||
endOffset: number;
|
||||
count: number;
|
||||
}>;
|
||||
isBlockCoverage: boolean;
|
||||
}>;
|
||||
}
|
||||
|
||||
// Istanbul coverage format
|
||||
interface IstanbulCoverage {
|
||||
[filePath: string]: {
|
||||
path: string;
|
||||
statementMap: Record<string, any>;
|
||||
fnMap: Record<string, any>;
|
||||
branchMap: Record<string, any>;
|
||||
s: Record<string, number>;
|
||||
f: Record<string, number>;
|
||||
b: Record<string, number[]>;
|
||||
};
|
||||
}
|
||||
|
||||
async function convertV8ToIstanbul() {
|
||||
console.log('\n🔄 Converting V8 Coverage to Istanbul Format...\n');
|
||||
|
||||
const rawDir = path.join(process.cwd(), 'coverage-e2e/raw');
|
||||
const outputDir = path.join(process.cwd(), 'coverage-e2e/.nyc_output');
|
||||
|
||||
// Check if raw directory exists
|
||||
try {
|
||||
await fs.access(rawDir);
|
||||
} catch {
|
||||
console.log('❌ No V8 coverage found at:', rawDir);
|
||||
console.log('\nℹ️ To generate V8 coverage:');
|
||||
console.log(' 1. Add coverage helpers to E2E tests (see E2E_COVERAGE_GUIDE.md)');
|
||||
console.log(' 2. Run: E2E_COVERAGE=true npm run test:e2e');
|
||||
console.log('\nℹ️ Alternatively, use Istanbul instrumentation (see guide)');
|
||||
return;
|
||||
}
|
||||
|
||||
// Create output directory
|
||||
await fs.mkdir(outputDir, { recursive: true });
|
||||
|
||||
// Check for v8-to-istanbul dependency
|
||||
let v8toIstanbul: any;
|
||||
try {
|
||||
// Dynamic import to handle both scenarios (installed vs not installed)
|
||||
const module = await import('v8-to-istanbul');
|
||||
v8toIstanbul = module.default || module;
|
||||
} catch {
|
||||
console.log('❌ v8-to-istanbul not installed\n');
|
||||
console.log('📦 Install it with:');
|
||||
console.log(' npm install -D v8-to-istanbul\n');
|
||||
console.log('⚠️ Note: V8 coverage approach requires this dependency.');
|
||||
console.log(' Alternatively, use Istanbul instrumentation (no extra deps needed).\n');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Read all V8 coverage files
|
||||
const files = await fs.readdir(rawDir);
|
||||
const jsonFiles = files.filter((f) => f.endsWith('.json'));
|
||||
|
||||
if (jsonFiles.length === 0) {
|
||||
console.log('⚠️ No coverage files found in:', rawDir);
|
||||
console.log('\nRun E2E tests with coverage enabled:');
|
||||
console.log(' E2E_COVERAGE=true npm run test:e2e\n');
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(`📁 Found ${jsonFiles.length} V8 coverage file(s)\n`);
|
||||
|
||||
const istanbulCoverage: IstanbulCoverage = {};
|
||||
const projectRoot = process.cwd();
|
||||
let totalConverted = 0;
|
||||
let totalSkipped = 0;
|
||||
|
||||
// Process each V8 coverage file
|
||||
for (const file of jsonFiles) {
|
||||
const filePath = path.join(rawDir, file);
|
||||
console.log(`📄 Processing: ${file}`);
|
||||
|
||||
try {
|
||||
const content = await fs.readFile(filePath, 'utf-8');
|
||||
const v8Coverage: V8CoverageEntry[] = JSON.parse(content);
|
||||
|
||||
for (const entry of v8Coverage) {
|
||||
try {
|
||||
// Skip non-source files
|
||||
if (!entry.url.startsWith('http://localhost') && !entry.url.startsWith('file://')) {
|
||||
continue;
|
||||
}
|
||||
|
||||
// Skip node_modules, .next, and other irrelevant files
|
||||
if (
|
||||
entry.url.includes('node_modules') ||
|
||||
entry.url.includes('/.next/') ||
|
||||
entry.url.includes('/_next/') ||
|
||||
entry.url.includes('/webpack/')
|
||||
) {
|
||||
totalSkipped++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Convert URL to file path
|
||||
let sourcePath: string;
|
||||
if (entry.url.startsWith('file://')) {
|
||||
sourcePath = fileURLToPath(entry.url);
|
||||
} else {
|
||||
// HTTP URL - extract path
|
||||
const url = new URL(entry.url);
|
||||
// Try to map to source file
|
||||
sourcePath = path.join(projectRoot, 'src', url.pathname);
|
||||
}
|
||||
|
||||
// Skip if not in src/
|
||||
if (!sourcePath.includes('/src/')) {
|
||||
totalSkipped++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Verify file exists
|
||||
try {
|
||||
await fs.access(sourcePath);
|
||||
} catch {
|
||||
totalSkipped++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Convert using v8-to-istanbul
|
||||
const converter = v8toIstanbul(sourcePath);
|
||||
await converter.load();
|
||||
converter.applyCoverage(entry.functions);
|
||||
const converted = converter.toIstanbul();
|
||||
|
||||
// Merge into combined coverage
|
||||
Object.assign(istanbulCoverage, converted);
|
||||
totalConverted++;
|
||||
} catch (error: any) {
|
||||
console.log(` ⚠️ Skipped ${entry.url}: ${error.message}`);
|
||||
totalSkipped++;
|
||||
}
|
||||
}
|
||||
} catch (error: any) {
|
||||
console.log(` ❌ Failed to process ${file}: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Write Istanbul coverage
|
||||
const outputPath = path.join(outputDir, 'e2e-coverage.json');
|
||||
await fs.writeFile(outputPath, JSON.stringify(istanbulCoverage, null, 2));
|
||||
|
||||
console.log('\n' + '='.repeat(70));
|
||||
console.log('✅ Conversion Complete');
|
||||
console.log('='.repeat(70));
|
||||
console.log(`\n Files converted: ${totalConverted}`);
|
||||
console.log(` Files skipped: ${totalSkipped}`);
|
||||
console.log(` Output location: ${outputPath}\n`);
|
||||
|
||||
if (totalConverted === 0) {
|
||||
console.log('⚠️ No files were converted. Possible reasons:');
|
||||
console.log(" • V8 coverage doesn't contain source files from src/");
|
||||
console.log(' • Coverage was collected for build artifacts instead of source');
|
||||
console.log(' • Source maps are not correctly configured\n');
|
||||
console.log('💡 Consider using Istanbul instrumentation instead (see guide)\n');
|
||||
} else {
|
||||
console.log('✅ Ready to merge with Jest coverage:');
|
||||
console.log(' npm run coverage:merge\n');
|
||||
}
|
||||
}
|
||||
|
||||
// Run the conversion
|
||||
convertV8ToIstanbul().catch((error) => {
|
||||
console.error('\n❌ Error converting coverage:', error);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -1,219 +0,0 @@
|
||||
#!/usr/bin/env tsx
|
||||
/**
|
||||
* Merge Coverage Script
|
||||
*
|
||||
* Combines Jest unit test coverage with Playwright E2E test coverage
|
||||
* to generate a comprehensive combined coverage report.
|
||||
*
|
||||
* Usage:
|
||||
* npm run coverage:merge
|
||||
* # or directly:
|
||||
* tsx scripts/merge-coverage.ts
|
||||
*
|
||||
* Prerequisites:
|
||||
* - Jest coverage must exist at: ./coverage/coverage-final.json
|
||||
* - E2E coverage must exist at: ./coverage-e2e/.nyc_output/*.json
|
||||
*
|
||||
* Output:
|
||||
* - Combined coverage report in: ./coverage-combined/
|
||||
* - Formats: HTML, text, JSON, LCOV
|
||||
*/
|
||||
|
||||
import { createCoverageMap } from 'istanbul-lib-coverage';
|
||||
import { createContext } from 'istanbul-lib-report';
|
||||
import reports from 'istanbul-reports';
|
||||
import fs from 'fs';
|
||||
import path from 'path';
|
||||
|
||||
interface CoverageData {
|
||||
[key: string]: any;
|
||||
}
|
||||
|
||||
interface MergeStats {
|
||||
jestFiles: number;
|
||||
e2eFiles: number;
|
||||
combinedFiles: number;
|
||||
jestOnlyFiles: string[];
|
||||
e2eOnlyFiles: string[];
|
||||
sharedFiles: string[];
|
||||
}
|
||||
|
||||
async function mergeCoverage() {
|
||||
console.log('\n🔄 Merging Coverage Data...\n');
|
||||
|
||||
const map = createCoverageMap();
|
||||
const stats: MergeStats = {
|
||||
jestFiles: 0,
|
||||
e2eFiles: 0,
|
||||
combinedFiles: 0,
|
||||
jestOnlyFiles: [],
|
||||
e2eOnlyFiles: [],
|
||||
sharedFiles: [],
|
||||
};
|
||||
|
||||
const jestFiles = new Set<string>();
|
||||
const e2eFiles = new Set<string>();
|
||||
|
||||
// Step 1: Load Jest coverage
|
||||
console.log('📊 Loading Jest unit test coverage...');
|
||||
const jestCoveragePath = path.join(process.cwd(), 'coverage/coverage-final.json');
|
||||
|
||||
if (fs.existsSync(jestCoveragePath)) {
|
||||
const jestCoverage: CoverageData = JSON.parse(fs.readFileSync(jestCoveragePath, 'utf-8'));
|
||||
|
||||
Object.keys(jestCoverage).forEach((file) => jestFiles.add(file));
|
||||
stats.jestFiles = jestFiles.size;
|
||||
|
||||
console.log(` ✅ Loaded ${stats.jestFiles} files from Jest coverage`);
|
||||
map.merge(jestCoverage);
|
||||
} else {
|
||||
console.log(' ⚠️ No Jest coverage found at:', jestCoveragePath);
|
||||
console.log(' Run: npm run test:coverage');
|
||||
}
|
||||
|
||||
// Step 2: Load E2E coverage
|
||||
console.log('\n🎭 Loading Playwright E2E test coverage...');
|
||||
const e2eDir = path.join(process.cwd(), 'coverage-e2e/.nyc_output');
|
||||
|
||||
if (fs.existsSync(e2eDir)) {
|
||||
const files = fs.readdirSync(e2eDir).filter((f) => f.endsWith('.json'));
|
||||
|
||||
if (files.length === 0) {
|
||||
console.log(' ⚠️ No E2E coverage files found in:', e2eDir);
|
||||
console.log(' Run: E2E_COVERAGE=true npm run test:e2e');
|
||||
} else {
|
||||
for (const file of files) {
|
||||
const coverage: CoverageData = JSON.parse(
|
||||
fs.readFileSync(path.join(e2eDir, file), 'utf-8')
|
||||
);
|
||||
|
||||
Object.keys(coverage).forEach((f) => e2eFiles.add(f));
|
||||
map.merge(coverage);
|
||||
console.log(` ✅ Loaded E2E coverage from: ${file}`);
|
||||
}
|
||||
stats.e2eFiles = e2eFiles.size;
|
||||
console.log(` 📁 Total unique files in E2E coverage: ${stats.e2eFiles}`);
|
||||
}
|
||||
} else {
|
||||
console.log(' ⚠️ No E2E coverage directory found at:', e2eDir);
|
||||
console.log(' Run: E2E_COVERAGE=true npm run test:e2e');
|
||||
}
|
||||
|
||||
// Step 3: Calculate statistics
|
||||
stats.combinedFiles = map.files().length;
|
||||
|
||||
map.files().forEach((file) => {
|
||||
const inJest = jestFiles.has(file);
|
||||
const inE2E = e2eFiles.has(file);
|
||||
|
||||
if (inJest && inE2E) {
|
||||
stats.sharedFiles.push(file);
|
||||
} else if (inJest) {
|
||||
stats.jestOnlyFiles.push(file);
|
||||
} else if (inE2E) {
|
||||
stats.e2eOnlyFiles.push(file);
|
||||
}
|
||||
});
|
||||
|
||||
// Step 4: Generate reports
|
||||
console.log('\n📝 Generating combined coverage reports...');
|
||||
|
||||
const reportDir = path.join(process.cwd(), 'coverage-combined');
|
||||
fs.mkdirSync(reportDir, { recursive: true });
|
||||
|
||||
const context = createContext({
|
||||
dir: reportDir,
|
||||
coverageMap: map,
|
||||
});
|
||||
|
||||
const reportTypes = ['text', 'text-summary', 'html', 'json', 'lcov'];
|
||||
|
||||
reportTypes.forEach((reportType) => {
|
||||
try {
|
||||
const report = reports.create(reportType as any, {});
|
||||
report.execute(context);
|
||||
console.log(` ✅ Generated ${reportType} report`);
|
||||
} catch (error) {
|
||||
console.error(` ❌ Failed to generate ${reportType} report:`, error);
|
||||
}
|
||||
});
|
||||
|
||||
// Step 5: Print summary
|
||||
const summary = map.getCoverageSummary();
|
||||
|
||||
console.log('\n' + '='.repeat(70));
|
||||
console.log('📊 COMBINED COVERAGE SUMMARY');
|
||||
console.log('='.repeat(70));
|
||||
console.log(
|
||||
`\n Statements: ${summary.statements.pct.toFixed(2)}% (${summary.statements.covered}/${summary.statements.total})`
|
||||
);
|
||||
console.log(
|
||||
` Branches: ${summary.branches.pct.toFixed(2)}% (${summary.branches.covered}/${summary.branches.total})`
|
||||
);
|
||||
console.log(
|
||||
` Functions: ${summary.functions.pct.toFixed(2)}% (${summary.functions.covered}/${summary.functions.total})`
|
||||
);
|
||||
console.log(
|
||||
` Lines: ${summary.lines.pct.toFixed(2)}% (${summary.lines.covered}/${summary.lines.total})`
|
||||
);
|
||||
|
||||
console.log('\n' + '-'.repeat(70));
|
||||
console.log('📁 FILE COVERAGE BREAKDOWN');
|
||||
console.log('-'.repeat(70));
|
||||
console.log(`\n Total files: ${stats.combinedFiles}`);
|
||||
console.log(` Jest only: ${stats.jestOnlyFiles.length}`);
|
||||
console.log(` E2E only: ${stats.e2eOnlyFiles.length}`);
|
||||
console.log(` Covered by both: ${stats.sharedFiles.length}`);
|
||||
|
||||
// Show E2E-only files (these were excluded from Jest)
|
||||
if (stats.e2eOnlyFiles.length > 0) {
|
||||
console.log('\n 📋 Files covered ONLY by E2E tests (excluded from unit tests):');
|
||||
stats.e2eOnlyFiles.slice(0, 10).forEach((file) => {
|
||||
const fileCoverage = map.fileCoverageFor(file);
|
||||
const fileSummary = fileCoverage.toSummary();
|
||||
console.log(
|
||||
` • ${path.relative(process.cwd(), file)} (${fileSummary.statements.pct.toFixed(1)}%)`
|
||||
);
|
||||
});
|
||||
if (stats.e2eOnlyFiles.length > 10) {
|
||||
console.log(` ... and ${stats.e2eOnlyFiles.length - 10} more`);
|
||||
}
|
||||
}
|
||||
|
||||
console.log('\n' + '='.repeat(70));
|
||||
console.log(`\n✅ Combined coverage report available at:\n ${reportDir}/index.html\n`);
|
||||
|
||||
// Step 6: Check thresholds (from .nycrc.json)
|
||||
const thresholds = {
|
||||
statements: 90,
|
||||
branches: 85,
|
||||
functions: 85,
|
||||
lines: 90,
|
||||
};
|
||||
|
||||
let thresholdsFailed = false;
|
||||
console.log('🎯 Checking Coverage Thresholds:\n');
|
||||
|
||||
Object.entries(thresholds).forEach(([metric, threshold]) => {
|
||||
const actual = (summary as any)[metric].pct;
|
||||
const passed = actual >= threshold;
|
||||
const icon = passed ? '✅' : '❌';
|
||||
console.log(
|
||||
` ${icon} ${metric.padEnd(12)}: ${actual.toFixed(2)}% (threshold: ${threshold}%)`
|
||||
);
|
||||
if (!passed) thresholdsFailed = true;
|
||||
});
|
||||
|
||||
if (thresholdsFailed) {
|
||||
console.log('\n❌ Coverage thresholds not met!\n');
|
||||
process.exit(1);
|
||||
} else {
|
||||
console.log('\n✅ All coverage thresholds met!\n');
|
||||
}
|
||||
}
|
||||
|
||||
// Run the merge
|
||||
mergeCoverage().catch((error) => {
|
||||
console.error('\n❌ Error merging coverage:', error);
|
||||
process.exit(1);
|
||||
});
|
||||
Reference in New Issue
Block a user