JavaScript scorer

This requires a JS/TS configuration file

You can configure a custom JS/TS scorer by specifying a function in the scorers section of the configuration. Both sync and async functions are supported.

The function has the following signature:

• Arguments
  • Object with
    • output: object with key value to get the output value (string) and key metadata to get metadata (object); see output object
    • inputs: object of key-value pairs from the dataset sample
• Returns
  • Score object: object with score (number between 0 to 1), message (optional, string) and name (optional, string)

​

Multiple scores

It is possible for the method to return an array of score objects. Use name to distinguish between them.

function score({ output, inputs }) {
  // ...
  return [
    { score: 1, name: "syntax-score" },
    { score: 0, name: "semantic-score", message: "failure reason"}
  ]
}

​

Example

The Spider using TS uses this scorer.