Introduction


Fine-tuning adapts a pre-trained language model to a specific task or domain. While prompt engineering and RAG handle many use cases out of the box, fine-tuning is essential when you need consistent formatting, domain-specific knowledge, or behavior that base models cannot achieve through prompting alone. This guide covers the full spectrum of fine-tuning approaches.


When to Fine-Tune


Before investing in fine-tuning, consider whether simpler approaches suffice:


  • **Prompt engineering**: Good for simple formatting changes and basic instructions
  • **RAG**: Ideal for knowledge-intensive tasks with verifiable sources
  • **Fine-tuning**: Necessary for specialized output formats, tone adaptation, and consistent behavior patterns

    • Fine-tuning becomes cost-effective when you need to run many similar queries and can amortize the training cost over thousands or millions of inference calls.


    Fine-Tuning Approaches


    Full Fine-Tuning


    Full fine-tuning updates all model parameters on a target dataset. This approach achieves the highest quality but requires substantial compute — full fine-tuning of a 7B parameter model requires approximately 56 GB of GPU memory per batch.


    **When to use full fine-tuning:**

  • You have access to high-memory GPUs (A100 80GB or H100)
  • Your dataset is large and diverse (10,000+ examples)
  • The domain shift from pre-training data is significant
  • Maximum quality is critical

    • LoRA (Low-Rank Adaptation)


    LoRA injects trainable rank-decomposition matrices into the model's attention layers, reducing the number of trainable parameters by 10,000x. A 7B model can be fine-tuned with LoRA on a single consumer GPU with 24 GB memory.


    
from peft import LoraConfig, get_peft_model



lora_config = LoraConfig(

    r=16,           # Rank of the update matrices

    lora_alpha=32,  # Scaling factor

    target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],

    lora_dropout=0.05,

    bias="none",

    task_type="CAUSAL_LM"

)



model = get_peft_model(base_model, lora_config)

print(f"Trainable params: {model.num_parameters(only_trainable=True):,}")

# Output for Llama 2 7B: ~4,194,304 params (0.06% of total)

    **Key hyperparameters:**

  • **r (rank)**: Higher values (16-64) for complex tasks, lower (4-8) for simpler formatting. Rank 16 works well for most use cases
  • **alpha**: Typically double the rank value (alpha = 2 * r)
  • **Target modules**: Include all attention projection matrices for best results

    • QLoRA (Quantized LoRA)


    QLoRA combines 4-bit quantization with LoRA, enabling fine-tuning of 65B models on a single 48GB GPU. The model weights are quantized to 4-bit while LoRA adapters remain in full precision.


    
from transformers import BitsAndBytesConfig



bnb_config = BitsAndBytesConfig(

    load_in_4bit=True,

    bnb_4bit_use_double_quant=True,

    bnb_4bit_quant_type="nf4",

    bnb_4bit_compute_dtype=torch.bfloat16

)



model = AutoModelForCausalLM.from_pretrained(

    "meta-llama/Llama-2-7b-hf",

    quantization_config=bnb_config,

    device_map="auto"

)

    QLoRA achieves approximately 99% of full fine-tuning performance while reducing memory requirements by 4x.


    Dataset Preparation


    Dataset quality matters more than quantity. A well-curated 1,000-example dataset outperforms a noisy 10,000-example one.


    **Guidelines for instruction tuning datasets:**


  • **Diverse prompts**: Cover all edge cases your system will encounter

    • 2. **Correct responses**: Each response must be factually accurate and follow the desired format

    3. **Consistent formatting**: Use the same chat template throughout

    4. **Balanced distribution**: Avoid over-representing common patterns

    5. **Validation split**: Hold out 5-10% for evaluation


    **Format example:**


    
{

  "instruction": "Summarize the following meeting notes in 2-3 bullet points.",

  "input": "Team discussed Q1 results. Revenue grew 15%. Engineering shipped 3 features. Marketing launched new campaign.",

  "output": "- Q1 revenue grew 15%\n- Engineering shipped 3 new features\n- Marketing launched a new campaign"

}

    Training Process


    Modern fine-tuning uses the SFT (Supervised Fine-Tuning) trainer:


    
from trl import SFTTrainer



trainer = SFTTrainer(

    model=model,

    train_dataset=train_dataset,

    eval_dataset=eval_dataset,

    dataset_text_field="text",

    max_seq_length=2048,

    args=TrainingArguments(

        per_device_train_batch_size=4,

        gradient_accumulation_steps=4,

        learning_rate=2e-4,

        num_train_epochs=3,

        logging_steps=10,

        save_strategy="epoch",

    )

)



trainer.train()

    Evaluation


    Evaluate fine-tuned models on:

  • **Task accuracy**: Does the model produce correct outputs?
  • **Format compliance**: Does it follow the required structure?
  • **Hallucination rate**: Does it invent facts?
  • **Regression**: Has performance degraded on unrelated tasks?

    • Use an automated evaluation harness comparing the fine-tuned model against the base model on a held-out test set.


    Conclusion


    Fine-tuning remains the most powerful tool for adapting LLMs to specific domains and tasks. Start with QLoRA for cost-effective experimentation, scale to full fine-tuning only when quality demands it. Focus on dataset quality over quantity, and always measure performance against a clear baseline.