Skip to main content
Empirical runs Playwright tests that treat your application as a black box. This makes it harder to measure code coverage for these tests. We can combine some work on the app code with test side changes to fix this. Specifically, setting up code coverage requires three steps:
  1. App bundler changes — instrument your app with Istanbul (requires app source code, done by app team)
  2. Test repo changes — enable coverage collection in your test fixtures (can be done by Empirical agents)
  3. Build final report — download coverage artifacts and generate an HTML report (requires app source code, done by app team)

​
App repo changes

Add the vite-plugin-istanbul plugin to your Vite config to instrument your source code with Istanbul coverage hooks. If you are running tests against a deployed URL (not localhost), also add forceBuildInstrument: true to ensure instrumentation is included in production builds. 
diff --git a/vite.config.ts b/vite.config.ts
index cfc922f..f2e8295 100644
--- a/vite.config.ts
+++ b/vite.config.ts
@@ -2,6 +2,7 @@ import { defineConfig } from "vite";
 import react from "@vitejs/plugin-react";
 import tailwindcss from "@tailwindcss/vite";
 import { tanstackRouter } from "@tanstack/router-plugin/vite";
+import istanbul from "vite-plugin-istanbul";
 
 function resolvePath(relativePath: string): string {
@@ -23,6 +24,12 @@ export default defineConfig({
     }),
     react(),
     tailwindcss(),
+    istanbul({
+      include: "src/*",
+      exclude: ["node_modules", "test/"],
+      extension: [".js", ".ts", ".tsx"],
+      requireEnv: false,
+      forceBuildInstrument: true,
+    }),
   ],
   resolve: {
     alias: {

​
Tests repo changes

@empiricalrun/playwright-utils 0.43.0+ includes coverage collection utilities. In your test fixture file, enable coverage by passing { collectCoverage: true } to baseTestFixture. 
import { test as base, expect as baseExpect } from "@playwright/test";
import { baseTestFixture, extendExpect } from "@empiricalrun/playwright-utils/test";

export const test = baseTestFixture(base, { collectCoverage: true });
export const expect = extendExpect(baseExpect);
This automatically collects window.__coverage__ data from every page before it closes and attaches the Istanbul coverage JSON files to the test report. These files are uploaded like other artifacts (e.g. videos).

​
Build final report

To build the final coverage report with nyc, we will first download the coverage JSON files from Empirical’s storage buckets, and then build the report. First, fetch the test run details using the Get Test Run API, then download the Playwright JSON report from the summary_url property. 
# Fetch the test run to get the report URL
REPORT_URL=$(curl -s "https://dash.empirical.run/api/test-runs/<TEST_RUN_ID>" \
  -H "Authorization: Bearer $EMPIRICALRUN_KEY" \
  | jq -r '.data.test_run.testRun.summary_url')

# Download the report
curl -s "$REPORT_URL" -o report.json
Then, build a mapping of test case IDs to their coverage filenames: 
jq '
  [.suites[] | recurse(.suites[]?) | .specs[]? |
    {id: .id} + {
      coverage: [.tests[]? | .results[]? | .attachments[]?
        | select(.contentType == "application/json" and (.name | test("coverage")))
        | (.path | split("/") | last)
      ]
    }
  ] | map(select(.coverage | length > 0))
' report.json > coverage-map.json
Download all coverage files into .nyc_output: 
mkdir -p .nyc_output
jq -r '
  [.suites[] | recurse(.suites[]?) | .specs[]? | .tests[]? | .results[]? | .attachments[]?
   | select(.contentType == "application/json" and (.name | test("coverage"))) | .path
  ] | .[]
' report.json | while read -r url; do
  curl -s "$url" -o ".nyc_output/$(basename "$url")"
done
Finally, generate the HTML report using nyc from your app source directory: 
cd /path/to/your-app-code

npx nyc report --reporter=html \
               --temp-dir=/path/to/final-output-dir/.nyc_output \
               --report-dir=/path/to/final-output-dir