|
|
--- |
|
|
license: llama3.1 |
|
|
datasets: |
|
|
- newmindai/EuroHPC-Legal |
|
|
language: |
|
|
- tr |
|
|
- en |
|
|
base_model: |
|
|
- meta-llama/Llama-3.1-8B-Instruct |
|
|
tags: |
|
|
- EuroHPC |
|
|
- Karolina |
|
|
- Axolotl |
|
|
- Unsloth |
|
|
--- |
|
|
|
|
|
<div style="display: flex; justify-content: center; flex-wrap: wrap; gap: 15px; align-items: flex-start;"> |
|
|
|
|
|
<div style="padding: 10px; border-radius: 8px; background-color: #000; |
|
|
color-scheme: light dark; margin-top: 40px;"> |
|
|
<img |
|
|
src="https://cdn-uploads.huggingface.co/production/uploads/683d4880e639f8d647355997/iA0QsWk-k-fQ7CCt65znl.png" |
|
|
style="width: 100%; max-width: 250px; height: 40px; |
|
|
filter: invert(0) brightness(1) contrast(1);" /> |
|
|
</div> |
|
|
|
|
|
<img src="https://cdn-uploads.huggingface.co/production/uploads/683d4880e639f8d647355997/mqbOdFfrC7KjDZbQlLFFj.png" |
|
|
style="width: 100%; max-width: 300px; height: auto;" /> |
|
|
|
|
|
<img |
|
|
src="https://cdn-uploads.huggingface.co/production/uploads/683d4880e639f8d647355997/VGnh14pYg-SXSaEt640qz.png" |
|
|
style="width: 100%; max-width: 150px; height: 100px; margin-top: 35px;" /> |
|
|
|
|
|
</div> |
|
|
|
|
|
## Model Card |
|
|
|
|
|
This document describes a parameter-efficient fine-tuning setup using LoRA on the EuroHPC Karolina system. Axolotl provides flexible orchestration and Unsloth supplies optimized kernels for high-throughput training on the newmindai/EuroHPC-Legal dataset. This model is fine-tuned using Parameter-Efficient Fine-Tuning (PEFT) with LoRA (Low-Rank Adaptation) on the EuroHPC dataset, specifically the competition subset. The fine-tuning leverages the Axolotl framework for orchestration and Unsloth library for optimized training kernels. |
|
|
|
|
|
### Hyperparameters |
|
|
|
|
|
* **LoRA Rank**: 16 |
|
|
* **LoRA Alpha**: 32 |
|
|
* **LoRA Dropout**: 0.05 |
|
|
* **Learning Rate**: 3×10⁻⁵ with cosine scheduling |
|
|
* **Training Epochs**: 3 per domain |
|
|
* **Batch Size**: Optimized for A100 memory capacity |
|
|
|
|
|
### Architecture |
|
|
|
|
|
* **Base Model**: Llama-3.1-8B-Instruct (Meta) |
|
|
* **Fine-tuning Method**: LoRA (Low-Rank Adaptation) |
|
|
* **Parameter Efficiency**: Only trainable LoRA parameters, frozen base model |
|
|
* **Model Size**: 8B parameters (base) + LoRA adapters |
|
|
|
|
|
## Hardware and Software |
|
|
|
|
|
* **Orchestration**: [Axolotl framework](https://axolotl.ai/) |
|
|
* **Acceleration**: [Unsloth library](https://unsloth.ai/) |
|
|
* **Backend**: PyTorch with CUDA support |
|
|
* **System**: EuroHPC Karolina supercomputer |
|
|
* **GPUs**: NVIDIA A100 (8 × 40 GB per node, 320 GB HBM2 total) |
|
|
* **Utilization**: 85–90% GPU and memory efficiency |
|
|
* **Total Compute**: ~600 GPU hours |
|
|
|
|
|
## Data |
|
|
|
|
|
### Input Format |
|
|
|
|
|
The dataset follows the **Alpaca format** with three key fields: |
|
|
|
|
|
```json |
|
|
{ |
|
|
"instruction": "Task description or question", |
|
|
"input": "Additional context or input data", |
|
|
"output": "Expected response or answer" |
|
|
} |
|
|
``` |
|
|
|
|
|
**Dataset**: `newmindai/EuroHPC-Legal` (competition subset) |
|
|
|
|
|
## How to Use |
|
|
|
|
|
```python |
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM |
|
|
from peft import PeftModel |
|
|
|
|
|
# Load base model and tokenizer |
|
|
base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct") |
|
|
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B-Instruct") |
|
|
|
|
|
# Load LoRA adapter |
|
|
model = PeftModel.from_pretrained(base_model, "newmindai/Llama-3.1-8B-Instruct-competition-alpaca") |
|
|
|
|
|
# Format input according to Alpaca format |
|
|
def format_prompt(instruction, input_text=""): |
|
|
if input_text: |
|
|
return f"### Instruction:\n{instruction}\n\n### Input:\n{input_text}\n\n### Response:\n" |
|
|
else: |
|
|
return f"### Instruction:\n{instruction}\n\n### Response:\n" |
|
|
|
|
|
# Example usage |
|
|
prompt = format_prompt("Explain the benefits of regular exercise") |
|
|
inputs = tokenizer(prompt, return_tensors="pt") |
|
|
outputs = model.generate(**inputs, max_new_tokens=256) |
|
|
response = tokenizer.decode(outputs[0], skip_special_tokens=True) |
|
|
``` |
|
|
|
|
|
## Acknowledgments |
|
|
|
|
|
This research was supported by the EuroHPC Joint Undertaking (EuroHPC JU) under the Benchmark Access |
|
|
grant agreement No EHPC-BEN-2024B11-003. The authors gratefully acknowledge the computational resources |
|
|
provided by the IT4Innovations National Supercomputing Center (Czech Republic) on the Karolina supercomputer, |
|
|
made available through the EuroHPC JU. |
|
|
|
|
|
## Citation |
|
|
|
|
|
```bibtex |
|
|
@article{newmind2025, |
|
|
title={Tailoring AI for Turkish Law: Domain-Specific Fine-Tuning of Small Language Models for Legal Expertise}, |
|
|
author={New Mind AI Team}, |
|
|
journal={Procedia Computer Science}, |
|
|
year={2025}, |
|
|
volume={239}, |
|
|
doi={10.1016/j.procs.2025.08.239}, |
|
|
note={Available online 23 September 2025, Version of Record 23 September 2025} |
|
|
} |
|
|
``` |
