What is Fine-tuning?

How it works

A base LLM has learned general language from pre-training. Fine-tuning continues training on examples of the behaviour you want, most often pairs of inputs and ideal outputs (supervised fine-tuning). The model's weights shift so that it produces outputs like your examples.

Full fine-tuning updates every parameter, which needs a lot of GPU memory. Parameter-efficient fine-tuning (PEFT) is now the common approach. LoRA, introduced in 2021, freezes the original weights and trains small low-rank "adapter" matrices instead; the paper reported cutting trainable parameters by up to 10,000 times for GPT-3 175B. QLoRA goes further by loading the base model in 4-bit quantized form, which lets much larger models be tuned on a single GPU.

Fine-tuning can also use preference data rather than ideal answers, as in RLHF and related methods.

Why it matters

• Consistency. It is good for a fixed output format, a house tone, a classification scheme or a narrow skill the base model handles poorly.
• Smaller, cheaper models. A fine-tuned small language model can match a big general model on one task at a fraction of the inference cost.
• It is a poor way to add facts. Knowledge that changes is better supplied with RAG. Fine-tuning on facts can still leave the model guessing and is costly to refresh.
• Risks. Narrow training data can degrade general abilities or weaken safety behaviour, so evaluate before and after.

Example

A company wants support-ticket triage into 14 categories with a fixed JSON output. Prompting a large model works but is slow and expensive. They fine-tune an open-weight 8B model with LoRA on 5,000 labelled tickets, test it against a held-out set, and serve it on their own GPU.