ITERATIVE_FINETUNING.md

# Iterative Fine-tuning Process for ZamAI-Mistral-7B-Pashto

This document outlines a systematic approach to iteratively improve your Pashto language model through multiple fine-tuning cycles. Each iteration builds on insights from the previous one to create a progressively better model.

## The Iterative Workflow

```
┌─────────────────┐
│                 │
│  Initial Setup  │
│                 │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│                 │
│ Prepare Dataset ├────┐
│                 │    │
└────────┬────────┘    │
         │             │
         ▼             │
┌─────────────────┐    │
│                 │    │
│   Fine-tune     │    │
│                 │    │
└────────┬────────┘    │
         │             │
         ▼             │
┌─────────────────┐    │
│                 │    │
│    Evaluate     │    │
│                 │    │
└────────┬────────┘    │
         │             │
         ▼             │
┌─────────────────┐    │
│  Analyze and    │    │
│ Identify Issues ├────┘
└─────────────────┘
```

## Iteration 1: Initial Fine-tuning

**Goal**: Establish a baseline model and identify key performance issues.

1. **Dataset Preparation**:
   - Run `prepare_for_autotrain.py` with default parameters
   - Use the instruction-response format for first iteration

2. **Fine-tuning**:
   - Use `autotrain_finetune.py` with:
     - Learning rate: 2e-4
     - Epochs: 3
     - LoRA r: 16
     - LoRA alpha: 32

3. **Evaluation**:
   - Run `evaluate_and_iterate.py` on 20+ diverse samples
   - Document performance in key areas (completion quality, language accuracy)

4. **Analysis**:
   - Identify the most obvious issues (language mixing, short responses, etc.)
   - Determine which dataset aspects need improvement

## Iteration 2: Targeted Improvements

**Goal**: Address the major issues identified in the first iteration.

1. **Dataset Updates**:
   - Add more examples in underrepresented categories
   - Clean existing examples with issues
   - Consider experimenting with different formats (text vs instruction-response)

2. **Fine-tuning Adjustments**:
   - Adjust learning rate based on first run (increase if underfitting, decrease if overfitting)
   - Potentially increase epochs to 4-5 if needed

3. **Evaluation**:
   - Compare new model against the baseline
   - Identify if the targeted issues show improvement
   - Document new issues that emerge

## Iteration 3: Parameter Optimization

**Goal**: Fine-tune the training parameters for optimal performance.

1. **Parameter Experiments**:
   - Try different LoRA configurations:
     - Higher rank (24 or 32) for more capacity
     - Different target modules if certain outputs show weakness
   - Experiment with batch size and optimization parameters

2. **Focused Evaluation**:
   - Test specifically on challenging examples
   - Evaluate language consistency in longer responses
   - Measure performance improvements against previous iterations

## Iteration 4: Dataset Expansion

**Goal**: Expand the model's capabilities to handle a wider range of content.

1. **Dataset Expansion**:
   - Add examples in different domains based on evaluation gaps
   - Create specialized test sets for different use cases
   - Consider augmenting data with variations of successful examples

2. **Advanced Fine-tuning**:
   - Consider mixed-precision training for efficiency
   - Experiment with different weights for different example types

## Measuring Progress

For each iteration, track these key metrics:

1. **Output Quality**: 
   - Coherence and relevance of generated text
   - Grammar and spelling accuracy

2. **Language Consistency**: 
   - Consistent use of Pashto throughout responses
   - Handling of code-switching if relevant to your use case

3. **Specific Task Performance**:
   - Performance on targeted tasks (translation, question answering, etc.)
   - Ability to follow complex instructions

4. **Technical Metrics**:
   - Training loss curves
   - Generation speed
   - Memory usage

## When to Stop Iterating

Consider your fine-tuning process complete when:

1. The model meets your quality thresholds for your target use cases
2. Successive iterations show diminishing returns in improvement
3. The most important failure cases have been addressed

Remember that perfection is not the goal - a useful, reliable model for your specific needs is!

## Additional Resources

- Use our `check_dataset_format.py` to validate your dataset before each iteration
- Maintain a versioning system for your models to track progress
- Document training settings and results for each iteration