After executing the test, you can access the results through a reporter.

Empirical provides 2 built-in reporter types:

  • terminal : Displays test summary in the console.
  • webapp : Launches a web server to view test summary in a browser.

​
Terminal reporter

A terminal reporter presents a test summary in the console. This summary is available immediately after running test samples using the npx empiricalrun command.

The summary includes:

  • Table containing statistical summaries for each configured run
  • Total number of dataset samples
  • Duration of the test run
  • Errors during the test run, if any

​
Statistical summary

The summary table includes statistics for following metrics:

  • output : Percentage of successful outputs from the run provider. A score lower than 100% means that there were instances where the run provider failed to respond.
  • scorer : Average score of the configured scorer represented in percentage.
    • This metric is shown for each configured scorer separately by name.

​
Web app reporter

The web app reporter launches a local web server to view detailed summary of the runs in the browser.

To start the web app reporter, run the following command after running the test:

npx empiricalrun ui

Running the above command will open a view on a browser similar to the following

The web app reporter view contains:

  • inputs : A list of all dataset test samples
  • runs : Outputs for each run configured
  • statistics : Statistical summary for each run

The web app also allows you to:

  • Modify run and execute it
  • Delete a run

The web app interactivity saves you the hassle of repeatedly navigating back to empiricalrc.json to configure, run and compare results. Using web app, one can seamlessly modify, execute and delete runs in real-time, offering a much higher iteration speed and productivity.

​
Modifying a run

A run configuration can be modified and executed in 2 steps:

1

Click "Show config" button next to the run

A run configuration box will open, allowing you to update the prompt and other parameters of the config. You can update them as per your need.

2

Hit "Run"

Click the “Run” button to execute the run with the updated parameters.

Post execution, a new run will be added to the table, allowing you to compare the results side-by-side.

​
Deleting a run

To delete a run, click on the (-) button next to the run you want to remove. This will permanently remove the run from the web app.

Run in GitHub ActionsTelemetry