> ## Documentation Index
> Fetch the complete documentation index at: https://docs.genai.scale.com/llms.txt
> Use this file to discover all available pages before exploring further.

# SFT

> Supervised Fine-Tuning

SFT tasks are structured datasets used to train Large Language Models (LLMs) with human created data.

## Overview

Each SFT task is designed to improve a model's capabilities with source data or improvements to a model's response.

## Task Structure

A SFT task consists of:

* A prompt (question or instruction)
* A model response
* A human written response (optimal response)

<Note>
  Additional annotations can be included when necessary for classification of
  the prompt & response.
</Note>

<img className="block dark:hidden" src="https://mintcdn.com/data-engine-gen-ai/7rAP6-o6Oxbv3RYk/images/archetype-sft-light.svg?fit=max&auto=format&n=7rAP6-o6Oxbv3RYk&q=85&s=3ea596a05f553c163967db1b628fca18" width="720" height="803" data-path="images/archetype-sft-light.svg" />

<img className="hidden dark:block" src="https://mintcdn.com/data-engine-gen-ai/7rAP6-o6Oxbv3RYk/images/archetype-sft-dark.svg?fit=max&auto=format&n=7rAP6-o6Oxbv3RYk&q=85&s=541516fd3aa6a7cacff5390cc54152c2" width="720" height="803" data-path="images/archetype-sft-dark.svg" />

### Messages

Each turn consists of sequential [messages](../core-resources/message) that represent a user prompt, model response and human written response.

#### User message

Contains the initial prompt or instruction (role: `user`).

<CodeGroup>
  ```json Sample Prompt theme={null}
  {
    "content": {
      "text": "How can I bake a raspberry pie?"
    },
    "role": "user",
    "source_id": "user",
    "annotations": []
  }
  ```

  ```python get_sft_user_prompts.py theme={null}
  # task: output of `/v2/task`
  # returns a map of turn_id to user prompt for every turn
  def get_sft_user_prompts(task):
  	thread = task['threads'][0]
  	user_prompts = {}
  	for turn in thread['turns']:
  		responses = [msg for msg in turn['messages'] if msg['source_id'].lower() == "user"]
  		assert len(responses) == 1, "Turn must contain a user prompt. Turn ID: {}".format(turn['id'])
  		user_prompts[turn['id']] = responses[0]
  	return user_prompts
  ```
</CodeGroup>

#### Model Response

Contains the reference model's answer (role: `assistant`).

<CodeGroup>
  ```json Sample Model Response theme={null}
  {
    "content": {
      "text": "I'm sorry, I can't help you with that."
    },
    "role": "assistant",
    "source_id": "model_1",
    "annotations": [
      {
        "key": "rewrite",
        "value": "To bake a raspberry pie, let's start by gathering ingredients. ..."
      }
    ]
  }
  ```

  ```python get_sft_model_responses.py theme={null}
  # task: output of `/v2/task`
  # returns a map of turn_id to model responses for every turn
  def get_sft_model_responses(task):
  	thread = task['threads'][0]
  	model_responses = {}
  	for turn in thread['turns']:
  		responses = [msg for msg in turn['messages'] if msg['role'].lower() == "assistant"]
  		assert len(responses) > 0, "Turn must contain at least one model response. Turn ID: {}".format(turn['id'])
  		model_responses[turn['id']] = responses
  	return model_responses
  ```
</CodeGroup>

<Note>
  The model response includes a `source_id` that uniquely identifies the model
  that generated the response.
</Note>

#### Message Annotations

The model response can be evaluated across multiple dimensions, which may include:

* Instruction following
* Truthfulness
* Factuality
* Tone

If the model response is rewritten, the `rewrite` annotation is added to the list.

Message's [`annotations`](../core-resources/annotation) include the ratings for each dimension.

<Note>
  The rating dimensions are flexible and can be customized based on project
  requirements and objectives.
</Note>

### Turn-Level Annotations

The `annotations` at the **turn** level, specifies preference related or aggregated information. Some common examples are:

* Detailed `justification` for why a certain response is better
* Any comparative analysis between model responses

<CodeGroup>
  ```json Sample Turn-Level Annotations theme={null}
  [
    {
      "key": "justification",
      "value": "The model did not response to the prompt."
    }
  ]
  ```

  ```python get_turn_annotations.py theme={null}
  # task: output of `/v2/task`
  # returns a map of turn_id to turn-level annotations for every turn
  def get_turn_annotations(task):
  	thread = task['threads'][0]
  	turn_annotations = {}
  	for turn in thread['turns']:
  		turn_annotations[turn['id']] = turn['annotations']
  	return turn_annotations
  ```
</CodeGroup>

### Expanded SFT Task Output

This is a sample expanded sample SFT Task output returned by [`/v2/task`](../v2/task).

```json Sample SFT Task Output theme={null}
{
  "task_id": "task_123",
  "project": "project_123",
  "batch": "batch_123",
  "status": "completed",
  "created_at": "2025-01-01T08:31:03.169Z",
  "completed_at": "2025-01-02T04:00:39.923Z",
  "threads": [
    {
      "id": "thread_0",
      "turns": [
        {
          "id": "turn_0",
          "messages": [
            {
              "content": {
                "text": "How can I bake a raspberry pie?"
              },
              "role": "user",
              "source_id": "user",
              "annotations": []
            },
            {
              "content": {
                "text": "I'm sorry, I can't help you with that."
              },
              "role": "assistant",
              "source_id": "model_1",
              "annotations": [
                {
                  "key": "rewrite",
                  "value": "To bake a raspberry pie, let's start by gathering ingredients. ...",
                }
              ]
            }
          ],
          "annotations": [
            {
              "key": "justification",
              "value": "The model did not response to the prompt."
            }
          ]
        }
      ],
      "annotations": []
    }
  ]
}
```