Fine tuning

What fine tuning means

In fine tuning an already trained language model is trained further with your own examples. The model therefore does not start empty but already knows language and now receives a direction. A distinction is made between training from scratch, which needs enormous amounts of data and computing power, and fine tuning, which gets by with a few hundred to a few thousand examples.

In practice it usually runs through a provider's interface: upload example files, start a run, receive your own model. The result is a model that only the client uses.

What it learns and what it does not

Fine tuning is good at teaching form and behaviour. It learns tone, sentence structure, the typical shape of a product description, an output format and the way a company speaks. It also learns patterns in the task: which kind of question is answered how.

It is less reliable for facts. A model fine tuned on last month's prices still returns those prices when they are outdated, and it can mix them with other values. For knowledge that can change, that is unsuited. The rule is therefore: fine tuning for form, retrieval from a source for facts.

When it is the right choice

• Consistent formats: when hundreds of outputs have to be built exactly the same way, the model learns that better than a long instruction.
• Short instructions: a fine tuned model needs less context in the prompt because the format already sits in the model. That lowers cost per request.
• Specific tone: a very particular style is shown better through examples than described.
• Fixed classification: when enquiries have to be sorted into categories, a small fine tuned model can be cheaper and faster than a large one.

What it brings along

The advantage is bought. A model of your own has to be maintained: if the style changes, a new run is needed. Errors in the example data are carried over, which is why choosing the examples is the real work and not the technical step. Data protection comes on top: uploading example data hands over content from the company. That has to be settled contractually beforehand. And finally traceability: a fine tuned model with an error cannot be fixed by changing one line in a document. With retrieval that is exactly enough.

Frequent questions

• Does fine tuning replace RAG? No, usually it complements it. The two solve different tasks.
• How many examples are needed? For a format often a few hundred, for behaviour more. Fewer examples lead to overfitting.
• Can I change the model afterwards? Only through a new run. That is the key difference from retrieval out of a source.