Fine-tuning
Also: Adaptation · Task-specific training
Continuing to train an existing model on a narrow dataset so it performs a specific task better than prompting alone can achieve.
The base model supplies general capability; the additional training supplies the format, vocabulary and judgement of one domain. Modern practice usually adapts a small number of added parameters rather than the whole model, which keeps the cost low enough that a narrow task can justify it.
Why it came back
It was widely written off when context windows grew and retrieval improved — if the relevant material fits in the prompt, why train. The answer turned out to be narrow, repetitive tasks where the same instructions are re-sent on every call: fine-tuning moves that cost from inference time to training time once.
When it is the wrong tool
- The knowledge changes. A fine-tuned model is a snapshot; retrieval reads current material.
- You need to cite sources. Training bakes information into weights and destroys its provenance.
- The problem is a prompt problem. A large share of what gets fine-tuned would be solved by a clearer instruction, at no cost.
The decision is not fine-tune against retrieve. Retrieval supplies facts; fine-tuning supplies behaviour, and most production systems need both.