Fine-Tuning

Fine-tuning is the process of further training a pre-trained model on a smaller, domain-specific dataset so its behavior shifts toward a particular task or style, such as parsing resumes or drafting job descriptions in a consistent format. Unlike Retrieval-Augmented Generation (RAG), which fetches fresh content for each question, fine-tuning changes the model itself and happens ahead of time, before the model is ever used.

The two techniques solve different problems. Fine-tuning suits stable behaviors, like output format, tone, and performance on a well-defined task. RAG suits facts that change and need to stay current and private, like policies, balances, and org structure. A common misconception is that fine-tuning teaches the model your policies. It does not do that reliably, and it cannot keep up with change. Grounding is what handles that. Niyuk fine-tunes open-source models on anonymized HR datasets using parameter-efficient techniques, and uses RAG for tenant-specific facts.