SFT Trainer
Start with supervised fine-tuning
Learn how input and target examples are organized, then sketch a small task-specific dataset.
Start with: SFT Trainer
RedLectern topic
Fine-tuning
Task and domain adaptation · supervised tuning · efficient adapters · distillation.
For: Technical builders
After this page you can: Prepare and evaluate a small fine-tuning experiment.
Page reviewed
Part of: Artificial Intelligence › AI training and adaptation
About Fine-tuning
Fine-tuning adapts a pretrained model to a target task or domain using additional training data. In supervised fine-tuning, examples pair inputs with desired outputs; the model learns from those targets. This is useful when you need a repeatable output format or behavior that is demonstrated in examples, and can be measured on held-out cases.
Start by defining the task and what counts as a good answer, then prepare representative examples and keep evaluation cases separate from training. Choose a training approach that fits the experiment: update the full model, or train a smaller set of added or modified parameters with a parameter-efficient method. LoRA and QLoRA are examples of adapter-based approaches; distillation instead trains a student from signals produced by a teacher. Compare the adapted model with the untouched base on the same held-out cases, including cases outside the target task. The goal is evidence about whether this data and method helped your intended task, not a claim that fine-tuning is always the right tool.
Start here
Parameter-efficient fine-tuning
Explore parameter-efficient adaptation
See how LoRA updates selected weights and how QLoRA combines LoRA with a quantized base.
Start with: Parameter-efficient fine-tuning
Learn the foundations
1.Choose an adaptation methodSupervised fine-tuning uses input and output examples to train a model toward target responses.
Supervised fine-tuning uses input and output examples to train a model toward target responses. Before training, decide whether the task needs new behavior demonstrated by examples or greater familiarity with a domain’s language and patterns; write down what success looks like for both. Keep the examples representative of the inputs the model will actually receive.
Full fine-tuning updates the model’s parameters. Parameter-efficient methods train a limited set of parameters or added components; LoRA uses low-rank update matrices while leaving the original weights frozen. QLoRA pairs LoRA training with a quantized base model. An adapter is an added trainable component. Distillation is a separate route: a student model learns from signals produced by a teacher. These approaches have different training setups, so make the method part of the experiment plan.
- Parameter-efficient fine-tuning
Describes adapters trained on top of a pretrained model and their checkpoints.
- Quantization
Explains how QLoRA combines quantization with LoRA training.
- Distilling the Knowledge in a Neural Network
A research paper on compressing an ensemble's knowledge into a single model.
Supervised fine-tuning
Adapting a model by training it on examples that pair inputs with desired outputs.
Parameter-efficient fine-tuning
Adapting a model by training a limited set of parameters or added components.
LoRA
An adaptation method that learns low-rank updates to selected model weights.
QLoRA
Training LoRA adapters while keeping the base model quantized.
Adapter
A small trainable component added to adapt an existing model.
Distillation
Training a student model to imitate signals from a teacher model.
Try it
You have examples of customer questions paired with approved answers. In one sentence, name a suitable adaptation method and explain what the pairs provide.
Supervised fine-tuning is suitable because each customer question is paired with a desired answer that can serve as a training target.
2.Prepare a small experimentWrite a narrow task statement before assembling examples: specify the input, the expected output, and how you will judge it.
Write a narrow task statement before assembling examples: specify the input, the expected output, and how you will judge it. For supervised fine-tuning, each example should make the intended input-to-output behavior clear. A dataset can use text examples, prompt-completion pairs, or conversational messages; choose one format that matches the task and inspect the processed examples before training.
Set aside evaluation cases before training so they are not part of the examples used to update the model. Include typical cases and a few cases that reveal likely failure modes. Record the base model, data version, training method and settings, then change one important choice at a time. This gives you a more interpretable comparison than changing the data and training setup together.
- SFT Trainer
Use its examples to inspect how conversational and prompt-completion samples are represented.
Try it
Draft one training example for a task that converts a support question into a concise answer. Then write one held-out test question that checks the same behavior.
Training example: input “How do I reset my password?”; target “Use the password reset link on the sign-in page.” Held-out example: a different account-access question with its own approved answer.
3.Evaluate the resultCompare the adapted model with the untouched base using the same held-out inputs and the same scoring procedure.
Compare the adapted model with the untouched base using the same held-out inputs and the same scoring procedure. For each task, choose measures that match the intended output: a classification task can use label-based metrics, while generated answers may need a mix of structured checks and human review. Keep a few examples for close inspection so a single aggregate score does not hide specific errors.
Check whether the tuned model improved the target behavior and whether it regressed on cases outside that target. Record the evaluation data, metric, decoding setup and any manual judgments alongside the result. A model, dataset and metric form a useful evaluation unit, but conclusions still depend on the test set and procedure. Treat one small experiment as evidence about that setup rather than a universal ranking.
- Using the evaluator
Shows task-specific evaluation inputs and explains that reported timing depends on hardware and pipeline processing.
Try it
A tuned model improves on your held-out task cases but answers unrelated general questions less reliably. What two results should you report?
Report the target-task result and the result on unrelated general questions separately, using the same evaluation sets and procedure for the base and tuned models.
Latest
No recent updates are available.
Books
Hands-On Large Language Models
Jay Alammar and Maarten Grootendorst · 2024
A practical route through fine-tuning existing language models for specific applications. Its examples span representation and generation models; the review notes that model and library details can age quickly.
Build a Large Language Model (From Scratch)
Sebastian Raschka · 2024
A hands-on way to understand the model and training loop behind fine-tuning. The edition teaches GPT-style models and task-specific tuning; its examples use an educational-scale model.