PEFT is a family of techniques, including LoRA, QLoRA, DoRA, and adapter tuning, that adapts large pre-trained models to new tasks by training only a small frac
Parameter-efficient fine-tuning (PEFT) adapts large models by training tiny added components, LoRA's low-rank matrices, adapters, or learned prompt vectors, while the base stays frozen. You get task adaptation at a sliver of full fine-tuning's compute and storage, with artifacts small enough to swap per task or tenant.
PEFT made customization affordable: single-GPU adaptation of strong open models, hundreds of task adapters multiplexed over one deployment, and no catastrophic forgetting of base skills. It's the default fine-tuning mode in 2026: full-parameter training is reserved for frontier-scale needs.
For most applied tasks, yes: within noise on typical benchmarks at a fraction of cost. Full fine-tuning retains an edge for deep behavioral overhauls or maximal capability extraction, which few applications need.
LoRA (or QLoRA on constrained hardware) is the pragmatic default with the richest tooling. Alternatives, adapters, prefix/prompt tuning, matter in research or special serving setups; start with LoRA unless you know why not.
Adapters are megabytes, so inference stacks hot-swap or batch many adapters over one base model: enabling per-task or per-tenant specialization without per-model GPU fleets.