RAG vs fine-tuning: how to choose

Use RAG when…

• The answer depends on facts that change or that are private.
• You need to show where an answer came from.
• Different users may see different documents.

Use fine-tuning when…

• You need a consistent style, format or tone.
• The task is narrow and repeated, and prompts alone are not reliable enough.
• You want a smaller model to do one job well.

Maintenance

RAG is updated by updating the index. Fine-tuning is updated by training again, so facts that change quickly are a poor fit. Knowledge baked into weights is also hard to audit or remove.

A practical order

1. Try a clear prompt with examples.
2. Add retrieval if the model lacks facts.
3. Consider fine-tuning only if behaviour is still wrong and you have good examples.
4. Measure each step on your own test set before moving on.

IICTP is a new independent credential and is not ISO/IEC 17024 accredited.