Update README.md

README.md

@@ -45,18 +45,18 @@ This model has been fine-tuned using LoRA (Low-Rank Adaptation) technique on a c
 - **Learning Rate**: 2e-4
 - **Batch Size**: 6 per device
 - **Gradient Accumulation**: 1 step
-- **Warmup Steps**: 10
+- **Warmup Steps**: 5
 - **Weight Decay**: 0.01
 - **LR Scheduler**: linear
 - **Optimizer**: paged_adamw_8bit
 - **Precision**: bfloat16
 
 ### LoRA Configuration
-- **LoRA Rank**: 16
+- **LoRA Rank**: 32
 - **LoRA Alpha**: 32
-- **Target Modules**: All linear layers
-- **Dropout**: 0.1
-- **Max Sequence Length**: 2048
+- **Target Modules**: ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]
+- **Dropout**: 0.05
+- **Max Sequence Length**: 4096
 
 ### Dataset
 - **Size**: 25,650 examples