Get Started
API Reference
Project Archetypes
- Introduction
- Archetypes
Endpoints
- v2
- Tasks
- Deliveries
- Annotations
- v1
Get Tasks in a Delivery
curl --request GET \
--url https://api.scale.com/v2/delivery \
--header 'Authorization: Bearer <token>'{
"delivery": {
"id": "delivery_abc123",
"name": "My Scale Delivery",
"task_count": 1000,
"delivered_at": "2023-11-07T05:31:56Z",
"project": "project_abc123"
},
"tasks": [
{
"task_id": "task_abc123",
"project": "project_abc123",
"batch": "batch_abc123",
"status": "pending",
"created_at": "2023-11-07T05:31:56Z",
"completed_at": "2023-11-07T05:31:56Z",
"metadata": {},
"threads": [
{
"id": "thread_abc123",
"turns": [
{
"id": "turn_abc123",
"messages": [
{
"role": "system",
"content": {
"text": "<string>",
"reference_texts": [
{
"content": "<string>",
"category": "<string>",
"url": "<string>"
}
],
"attachments": [
{
"content": "aSDinaTvuI8gbWludGxpZnk=",
"mime_type": "text/plain",
"scale_url": "<string>",
"url": "<string>"
}
]
},
"source_id": "source_abc123",
"model_parameters": {
"model": "my-model-123",
"temperature": 0.9,
"max_completion_tokens": 1000
},
"annotations": [
{
"id": "an_abc123efg456",
"key": "formatting",
"type": "<string>",
"title": "Response Formatting",
"description": "Does the response contain issues with formatting?",
"labels": [
"Major Issues",
"Minor Issues",
"No Issues"
],
"metadata": {
"criteria": "overall_quality"
},
"value": 3,
"possible_values": [
[
1,
2,
3
]
]
}
]
}
],
"annotations": [
{
"id": "an_abc123efg456",
"key": "formatting",
"type": "<string>",
"title": "Response Formatting",
"description": "Does the response contain issues with formatting?",
"labels": [
"Major Issues",
"Minor Issues",
"No Issues"
],
"metadata": {
"criteria": "overall_quality"
},
"value": 3,
"possible_values": [
[
1,
2,
3
]
]
}
]
}
],
"annotations": [
{
"id": "an_abc123efg456",
"key": "formatting",
"type": "<string>",
"title": "Response Formatting",
"description": "Does the response contain issues with formatting?",
"labels": [
"Major Issues",
"Minor Issues",
"No Issues"
],
"metadata": {
"criteria": "overall_quality"
},
"value": 3,
"possible_values": [
[
1,
2,
3
]
]
}
]
}
],
"errors": [
{
"type": "UNSUPPORTED_LANGUAGE",
"message": "The specified language or locale is not supported"
}
]
}
],
"next_token": "<string>"
}Every request sent to Scale’s API requires authentication. In short, your API Key is the Bearer token. See the Authentication section for more details.
The API returns a maximum of 100 tasks per request (or less if you specify a limit).
If your request returns more tasks than your specified limit, API response will also contain a next_token until you reach to the last page.
You can set the next_token in your next request to continue downloading tasks from the next page.
Required query parameters
You are expected to provide one of the following to find a delivery:
delivery_idDelivery IDs are globally unique and can be used to retrieve a specific delivery.delivery_namewith a project identifier (project_idorproject_name)
Example Code
# Downloads all tasks from a delivery
import json
import requests
API_KEY = 'live_...'
PROJECT_NAME = 'My Test Project'
DELIVERY_NAME = 'My Test Delivery'
def get_tasks_by_delivery(project_name: str, delivery_name: str):
tasks = []
params = {
"project_name": project_name,
"delivery_name": delivery_name,
}
should_fetch = True
while should_fetch:
response = requests.request(
"GET",
url="https://api.scale.com/v2/delivery",
params=params,
headers={
"Accept": "application/json",
"Authorization": f"Bearer {API_KEY}",
},
)
json_resp = response.json()
next_token = json_resp.get('next_token')
if next_token:
params['next_token'] = next_token
else:
should_fetch = False
tasks.extend(json_resp['tasks'])
return tasks
if __name__ == "__main__":
tasks = get_tasks_by_delivery(PROJECT_NAME, DELIVERY_NAME)
with open(f'{DELIVERY_NAME}.jsonl', 'w+') as f:
for task in tasks:
json.dump(task, f)
f.write('\n')
Authorizations
Your API Key is the Bearer token. See the Authentication section to learn how to access your key.
Query Parameters
Scale's unique identifier for the delivery.
The name of the delivery.
Scale's unique identifier for the project.
The name of the project.
Limit the number of entities returned.
1 < x < 100A token used to retrieve the next page of results if there are more. You can find the next_token in your last request.
List of fields to expand in the response.
Entities that can be expanded from an ID to an object.
project, batch, delivery List of properties to include in the task response.
Property that can be included in the annotations
attachment_details, model_parameters Response
Array of task objects
Represents a single task
Unique identifier for the task.
Current status of the task.
pending, completed, canceled, error UTC timestamp when the task was created.
UTC timestamp when the task was completed.
Task metadata defined during task creation.
Threads associated with the task. Tasks that do not have a status of completed will have an empty threads array.
Represents a thread of messages in a task.
Unique identifier for the thread.
Turns within the thread.
A unique identifier for the turn.
A list of messages associated with this turn.
The role of the sender in the conversation (e.g., user, assistant).
system, user, assistant, function The content of the message, including text and any attachments.
The ID of the source system or user that sent the message.
Annotations specific to this message.
Represents a generic annotation.
Annotations applied to the entire turn.
Represents a generic annotation.
Unique identifier for the annotation.
Key for the annotation.
The type of the value and the possible_values, if they exist.
Title of the annotation.
Further details about the question.
String representation of the possible options.
A string representation of the annotation.
Integer type annotation value.
The possible values for this annotation.
Integer type annotation value.
Annotations for the entire thread.
Represents a generic annotation.
Unique identifier for the annotation.
Key for the annotation.
The type of the value and the possible_values, if they exist.
Title of the annotation.
Further details about the question.
String representation of the possible options.
A string representation of the annotation.
Unique identifier for an annotation.
Integer type annotation value.
The possible values for this annotation.
Integer type annotation value.
Errors associated with the task. Available when the task status is error
Details of the error on the task. Available when the task status is error
Delivery ID or Delivery associated with the task.
Unique identifier for the delivery.
The name of the delivery.
The number of tasks in the delivery.
UTC timestamp when the delivery was created.
A token used to retrieve the next page of results if there are more. You can find the next_token in your last request
curl --request GET \
--url https://api.scale.com/v2/delivery \
--header 'Authorization: Bearer <token>'{
"delivery": {
"id": "delivery_abc123",
"name": "My Scale Delivery",
"task_count": 1000,
"delivered_at": "2023-11-07T05:31:56Z",
"project": "project_abc123"
},
"tasks": [
{
"task_id": "task_abc123",
"project": "project_abc123",
"batch": "batch_abc123",
"status": "pending",
"created_at": "2023-11-07T05:31:56Z",
"completed_at": "2023-11-07T05:31:56Z",
"metadata": {},
"threads": [
{
"id": "thread_abc123",
"turns": [
{
"id": "turn_abc123",
"messages": [
{
"role": "system",
"content": {
"text": "<string>",
"reference_texts": [
{
"content": "<string>",
"category": "<string>",
"url": "<string>"
}
],
"attachments": [
{
"content": "aSDinaTvuI8gbWludGxpZnk=",
"mime_type": "text/plain",
"scale_url": "<string>",
"url": "<string>"
}
]
},
"source_id": "source_abc123",
"model_parameters": {
"model": "my-model-123",
"temperature": 0.9,
"max_completion_tokens": 1000
},
"annotations": [
{
"id": "an_abc123efg456",
"key": "formatting",
"type": "<string>",
"title": "Response Formatting",
"description": "Does the response contain issues with formatting?",
"labels": [
"Major Issues",
"Minor Issues",
"No Issues"
],
"metadata": {
"criteria": "overall_quality"
},
"value": 3,
"possible_values": [
[
1,
2,
3
]
]
}
]
}
],
"annotations": [
{
"id": "an_abc123efg456",
"key": "formatting",
"type": "<string>",
"title": "Response Formatting",
"description": "Does the response contain issues with formatting?",
"labels": [
"Major Issues",
"Minor Issues",
"No Issues"
],
"metadata": {
"criteria": "overall_quality"
},
"value": 3,
"possible_values": [
[
1,
2,
3
]
]
}
]
}
],
"annotations": [
{
"id": "an_abc123efg456",
"key": "formatting",
"type": "<string>",
"title": "Response Formatting",
"description": "Does the response contain issues with formatting?",
"labels": [
"Major Issues",
"Minor Issues",
"No Issues"
],
"metadata": {
"criteria": "overall_quality"
},
"value": 3,
"possible_values": [
[
1,
2,
3
]
]
}
]
}
],
"errors": [
{
"type": "UNSUPPORTED_LANGUAGE",
"message": "The specified language or locale is not supported"
}
]
}
],
"next_token": "<string>"
}