LoRA (Low-Rank Adaptation) is a Parameter-Efficient Fine-Tuning (PEFT) technique that adapts powerful AI models to specific tasks by injecting small, trainable
LoRA (Low-Rank Adaptation) freezes the pretrained model's weights and trains small low-rank matrices inserted into key layers: typically well under 1% of total parameters. Those adapters capture the task-specific adjustment; at inference they're merged or applied alongside the base weights. The result is fine-tuning that fits on modest GPUs, produces megabyte-scale artifacts instead of full model copies, and allows many adapters to share one base model.
LoRA democratized model customization: teams adapt strong open-weight models on a single GPU and swap task adapters cheaply instead of hosting separate full models. It's the backbone of 2026's enterprise fine-tuning economy and the reason 'a tuned model per workflow' is financially viable.
Full fine-tuning updates every weight and needs heavy multi-GPU infrastructure; LoRA trains tiny adapter matrices on a frozen base, reaching comparable task quality for most applications at a fraction of the cost, and produces small, swappable artifacts.
LoRA applied on top of a quantized (e.g. 4-bit) base model, shrinking memory needs further so even large models can be adapted on consumer-grade GPUs with minimal quality loss.
Yes: adapters can be merged, weighted, or routed per request, enabling one deployment to serve many specialized behaviors. Inference stacks in 2026 commonly multiplex hundreds of adapters over a shared base model.