Parameter-Efficient Fine-Tuning (PEFT)
Explore how to apply parameter-efficient fine-tuning techniques such as LoRA and QLoRA to improve language model output consistency and format adherence. Understand the training mechanisms, adapter placement, rank selection, and debugging steps to optimize performance while managing VRAM constraints and inference latency.
A base open-weight model can answer in the right topic and still miss a strict format. For example, a prompt might require valid JSON with keys decision and rationale, but repeated runs produce extra prose, missing keys, or a different ordering that breaks a parser.
If we attach a small trained add-on and run the same prompt with the same context length and no retrieval changes, the output class shifts. The model still sometimes makes factual mistakes, but it more consistently adheres to the target style and schema. With that contrast in view, the mechanism question is concrete. What changed during training if most of the base weights remain untouched?
LoRA trains a low-rank update
The key idea behind LoRA is that we do not train a full update to a large weight matrix. Instead, training learns two small matrices whose product acts like a constrained update added during the forward pass.
Here W is a frozen base weight matrix in some linear layer, A and B are trainable matrices, and