Skip to main content
POST
/
v2
/
autoevals
Start an evaluation job
curl --request POST \
  --url https://api.scale.com/v2/autoevals \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "libraryName": "<string>",
  "evaluationType": {
    "chatCompletion": {
      "solutions": 1,
      "shots": 0,
      "chainOfThought": false
    }
  },
  "modelVersionConfig": "<string>",
  "modelParams": {}
}'
{
  "jobId": "<string>",
  "status": "Pending",
  "libraryName": "<string>",
  "libraryVersion": "<string>",
  "evaluationOptions": {},
  "modelVersionConfig": "<string>",
  "modelParams": {},
  "createdAt": "2023-11-07T05:31:56Z",
  "completedAt": "2023-11-07T05:31:56Z"
}

Authorizations

​
Authorization
string
header
required

Your API Key is the Bearer token. See the Authentication section to learn how to access your key.

Body

application/json
​
libraryName
string
required

Name of the evaluation library.

​
modelVersionConfig
string
required

ID of the Model Version Config to be evaluated.

​
evaluationType
object
​
modelParams
object

Model params overrides.

Response

Auto-evaluation job successfully started.

​
jobId
string

Unique identifier for the evaluation job.

​
status
enum<string>

Current job status.

Available options:
Pending,
InProgress,
Errored,
Completed
​
libraryName
string

Name of the evaluation library.

​
libraryVersion
string

Version of the evaluation library.

​
evaluationOptions
object

Options used for evaluation.

​
modelVersionConfig
string

ID of the Model Version Config

​
modelParams
object

Model params overrides.

​
createdAt
string<date-time>

Timestamp when the job was created.

​
completedAt
string<date-time> | null

Timestamp when the job was completed.

⌘I