Follow the Fine-Tuning Open Models learning path on AI Builders Network - a free, structured roadmap with curated resources and step-by-step progress tracking.
This path is for ML and AI engineers who have exhausted prompting and RAG and have a genuine fine-tuning case: style consistency, domain vocabulary, or cost reduction via smaller specialized models. It is the most hardware-adjacent path in the catalog: you will rent GPUs and run real training.
By the end you'll be able to: Apply a defensible fine-tune vs prompt vs RAG decision framework; Select base models from the open-weight landscape on evidence; Run LoRA training with PEFT, Axolotl, or Unsloth on rented GPUs; Build high-quality training datasets, including synthetic data; Prove wins over the base model and serve with vLLM or Ollama.
Fine-tune for behavior, retrieve for knowledge. Fine-tuning excels at consistent style and format, domain-specific vocabulary, and distilling a large model's skill into a smaller, cheaper one. It is the wrong tool for injecting fresh facts: RAG does that better, stays current, and provides citations.
Less than most expect: LoRA fine-tuning a 7-8B model on a few thousand examples runs $10-50 of rented GPU time, and a 70B-class model typically lands in the low hundreds. Dataset construction is the real cost: expect to spend far more time curating examples than running training.
LoRA (Low-Rank Adaptation) freezes the base model and trains small adapter matrices instead: typically under 1% of parameters. It cuts memory and cost by an order of magnitude with near-parity quality for most tasks, fits on single GPUs, and produces small swappable adapter files instead of full model copies.