How you take a general model and make it yours — specialized for your task, your tone, your domain — without needing a data center. And why the clever shortcut called LoRA changed everything.
A base model like Llama or Gemma is a generalist — it knows a bit about everything but nothing about your specifics: your product's support style, your legal templates, your company's tone of voice. Fine-tuning is how you specialize a general model by training it a little more on examples of what you want.
The result is a model that reliably answers in the right format, uses your terminology, and behaves consistently — without you having to explain it all in every prompt.
Before fine-tuning, know it's one of three tools — and often the last one you should reach for. They stack from cheapest to most involved:
Prompting = just wording your request well (and giving examples in the prompt). RAG (from the embeddings guide) = feeding the model the right documents so it has the knowledge. Fine-tuning = actually changing the model's weights so it learns a skill or style. We'll compare when to use which in section 7.
Remember the weights — the billions of numbers that are the model's knowledge. Training set them in the first place. Fine-tuning continues that training, but gently, on a small set of your own examples (say, hundreds or thousands of ideal question-and-answer pairs).
The model nudges its weights just enough to absorb the pattern in your examples, while keeping everything it already knew.
Full fine-tuning means updating all of the model's weights. For a multi-billion-parameter model that's brutal:
This put fine-tuning out of reach for most people — until a shortcut arrived.
LoRA (Low-Rank Adaptation) is a wonderfully simple insight: don't touch the giant model at all. Freeze every original weight. Instead, add a tiny set of new weights alongside it, and train only those.
Those add-on weights (called an adapter) are typically well under 1% of the model's size. You get most of the benefit of full fine-tuning for a fraction of the memory, storage, and cost.
A model's big weight blocks are grids of numbers (matrices). Fully retraining one means changing every cell. LoRA's trick: instead of learning a big grid of changes, it learns two much skinnier grids that multiply together to approximate that change.
A "1000 × 1000" change would be a million numbers. But two skinny grids of "1000 × 8" and "8 × 1000" are only about 16,000 numbers — yet multiplied together they cover the same shape. That "8" is the rank: small rank = tiny adapter, slightly less expressive; larger rank = bigger adapter, more capacity.
Two lovely side-effects: an adapter is a small file (a few megabytes), and because the base model is untouched, you can keep many adapters and swap them in and out — one for legal tone, one for coding, one for customer support — all sharing the same base model.
Remember quantization from the first guide (storing weights in 4 bits to shrink them)? QLoRA combines the two ideas: quantize the frozen base model down to 4 bits and train a LoRA adapter on top.
Now even the frozen base barely uses memory, so you can fine-tune surprisingly large models on a single ordinary GPU. Quantization shrinks the base; LoRA shrinks the training. Together they made custom models genuinely accessible.
A quick decision guide — these are complements, not rivals, and you'll often combine them:
| Approach | Best for | Watch out |
|---|---|---|
| Prompting | Quick tasks; trying things out; when a clear instruction or a few examples is enough. | Long prompts get costly; behavior can be inconsistent. |
| RAG | Giving the model knowledge — private, large, or frequently-changing facts and documents. | Doesn't change the model's style or skills; needs a retrieval setup. |
| Fine-tuning / LoRA | Teaching a skill, format, or tone you need reliably and repeatedly. | Needs good example data; not the way to add fast-changing facts (use RAG for those). |
LoRA is a natural fit for edge devices. Because adapters are tiny and the base model is shared, a phone can keep one base model in memory and swap small adapters to change its personality or specialty — cheaply, instantly, and offline.
Pick an answer and it'll explain why. Score updates as you go.
1. What does fine-tuning do to a model?
Specializes it. Fine-tuning nudges the weights to learn your task, style, or format.
2. Why is full fine-tuning expensive?
All the weights. Each fine-tuned copy is a full-size model, and training needs lots of memory.
3. What is LoRA's core trick?
Freeze + tiny adapter. The big model is untouched; only a small set of new weights is trained.
4. Roughly how big is a LoRA adapter compared to the model?
Under 1%. That's why adapters are small files you can store and swap easily.
5. What does QLoRA add to LoRA?
Quantized base + LoRA. Combining the two shrink-tricks makes fine-tuning big models accessible.
6. You need the model to answer using constantly-changing internal docs. Best tool?
RAG. For fast-changing facts, retrieval is better; fine-tuning is for skills and style.
7. Why do LoRA adapters suit edge devices so well?
Tiny & swappable. Keep one base in memory and swap small adapters for different specialties — offline.
→ Next up: How Training Actually Works — where all these weights come from in the first place.