Added Model Card

README.md

@@ -1,3 +1,217 @@
----
-license: mit
----
+---
+license: mit
+datasets:
+- newmindai/RAGTruth-TR
+language:
+- tr
+- en
+metrics:
+- precision
+- recall
+- f1
+- roc_auc
+base_model:
+- newmindai/TurkEmbed4STS
+pipeline_tag: token-classification
+---
+
+# TurkEmbed4STS-HallucinationDetection
+
+## Model Description
+
+**TurkEmbed4STS-HallucinationDetection** is a Turkish hallucination detection model based on the GTE-multilingual architecture, optimized for semantic textual similarity and adapted for hallucination detection. This model is part of the Turk-LettuceDetect suite, specifically designed for detecting hallucinations in Turkish Retrieval-Augmented Generation (RAG) applications.
+
+## Model Details
+
+- **Model Type:** Token-level binary classifier for hallucination detection
+- **Base Architecture:** GTE-multilingual-base (TurkEmbed4STS)
+- **Language:** Turkish (tr)
+- **Training Dataset:** Machine-translated RAGTruth dataset (17,790 training instances)
+- **Context Length:** Up to 8,192 tokens
+- **Model Size:** ~135M parameters
+
+## Intended Use
+
+### Primary Use Cases
+- Hallucination detection in Turkish RAG systems
+- Token-level classification of supported vs. hallucinated content
+- Stable performance across diverse Turkish text generation tasks
+- Applications requiring consistent precision-recall balance
+
+### Supported Tasks
+- Question Answering (QA) hallucination detection
+- Data-to-text generation verification  
+- Text summarization fact-checking
+
+## Performance
+
+### Overall Performance (F1-Score)
+- **Whole Dataset:** 0.7666
+- **Question Answering:** 0.7420
+- **Data-to-text Generation:** 0.7797
+- **Summarization:** 0.6123
+
+### Key Strengths
+- Most consistent performance across all task types
+- Stable behavior avoiding extreme precision-recall imbalances
+- Good semantic understanding from Turkish fine-tuning
+
+## Training Details
+
+### Training Data
+- **Dataset:** Machine-translated RAGTruth benchmark
+- **Size:** 17,790 training instances, 2,700 test instances
+- **Tasks:** Question answering (MS MARCO), data-to-text (Yelp), summarization (CNN/Daily Mail)
+- **Translation Model:** Google Gemma-3-27b-it
+
+### Training Configuration
+- **Epochs:** 6
+- **Learning Rate:** 1e-5
+- **Batch Size:** 4
+- **Hardware:** NVIDIA A100 40GB GPU
+- **Training Time:** ~2 hours
+- **Optimization:** Cross-entropy loss with token masking
+
+### Pre-training Background
+- Built on GTE-multilingual-base architecture
+- Fine-tuned for NLI and STS tasks
+- Optimized for Turkish language understanding
+- Fine-tuned specifically for hallucination detection
+
+## Technical Specifications
+
+### Architecture Features
+- **Base Model:** GTE-multilingual encoder
+- **Specialization:** Turkish semantic textual similarity
+- **Maximum Sequence Length:** 8,192 tokens
+- **Classification Head:** Binary token-level classifier
+- **Embedding Dimension:** Based on GTE-multilingual architecture
+
+### Input Format
+```
+Input: [CONTEXT] [QUESTION] [GENERATED_ANSWER]
+Output: Token-level binary labels (0=supported, 1=hallucinated)
+```
+
+## Limitations and Biases
+
+### Known Limitations
+- Lower performance on summarization tasks compared to structured tasks
+- Performance dependent on translation quality of training data
+- Smaller model size may limit complex reasoning capabilities
+- Optimized for Turkish but built on multilingual foundation
+
+### Potential Biases
+- Translation artifacts from machine-translated training data
+- Bias toward semantic similarity patterns from STS pre-training
+- May favor shorter, more structured text over longer abstracts
+
+## Usage
+
+### Installation
+```bash
+pip install lettucedetect
+```
+
+### Basic Usage
+```python
+from lettucedetect.models.inference import HallucinationDetector
+
+# Initialize the Turkish-specific hallucination detector
+detector = HallucinationDetector(
+    method="transformer", 
+    model_path="newmindai/TurkEmbed4STS-HD"
+)
+
+# Turkish context, question, and answer
+context = "İstanbul Türkiye'nin en büyük şehridir. Şehir 15 milyonluk nüfusla Avrupa'nın en kalabalık şehridir."
+question = "İstanbul'un nüfusu nedir? İstanbul Avrupa'nın en kalabalık şehri midir?"
+answer = "İstanbul'un nüfusu yaklaşık 16 milyondur ve Avrupa'nın en kalabalık şehridir."
+
+# Get span-level predictions (start/end indices, confidence scores)
+predictions = detector.predict(
+    context=context, 
+    question=question, 
+    answer=answer, 
+    output_format="spans"
+)
+
+print("Tespit Edilen Hallusinasyonlar:", predictions)
+# Örnek çıktı: 
+# [{'start': 34, 'end': 57, 'confidence': 0.92, 'text': 'yaklaşık 16 milyondur'}]
+```
+
+
+## Evaluation
+
+### Benchmark Results
+Evaluated on machine-translated Turkish RAGTruth test set, showing the most consistent behavior across all three task types with stable precision-recall balance.
+
+**Example-level Results**
+
+<img 
+  src="https://cdn-uploads.huggingface.co/production/uploads/683d4880e639f8d647355997/RejTWu3JNjH8t0teV1Txf.png" 
+  width="1000" 
+  style="object-fit: contain; margin: auto; display: block;"
+/>
+**Token-level Results**
+
+<img 
+  src="https://cdn-uploads.huggingface.co/production/uploads/683d4880e639f8d647355997/ECyrfN5Jv8fZSM0svxLXq.png" 
+  width="500" 
+  style="object-fit: contain; margin: auto; display: block;"
+/>
+
+### Comparative Analysis
+- Most stable performance across diverse tasks
+- Consistent precision-recall balance (unlike models with extreme values)
+- Suitable for applications prioritizing reliability over peak performance
+
+## Citation
+
+```bibtex
+@inproceedings{turklettucedetect2025,
+  title={Turk-LettuceDetect: A Hallucination Detection Models for Turkish RAG Applications},
+  author={Authors Hidden for Review},
+  booktitle={9th International Artificial Intelligence and Data Processing Symposium (IDAP'25)},
+  year={2025},
+  address={Malatya, Turkey}
+}
+```
+
+## Related Work
+
+This model builds upon the TurkEmbed4STS model:
+```bibtex
+@article{turkembed4sts,
+  title={TurkEmbed4Retrieval: Turkish Embedding Model for Retrieval Task},
+  author={Ezerceli, Ö. and Gümüşçekicci, G. and Erkoç, T. and Özenc, B.},
+  journal={preprint},
+  year={2024}
+}
+```
+
+```bibtex
+@misc{Kovacs:2025,
+      title={LettuceDetect: A Hallucination Detection Framework for RAG Applications}, 
+      author={Ádám Kovács and Gábor Recski},
+      year={2025},
+      eprint={2502.17125},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2502.17125}, 
+}
+```
+
+
+## License
+
+This model is released under an open-source license to support research and development in Turkish NLP applications.
+
+## Contact
+
+For questions about this model or other Turkish hallucination detection models, please refer to the original paper or contact the authors.
+
+---
+
+**Note:** This model is optimized for stability and consistency across different Turkish RAG tasks, making it ideal for production environments where reliable performance is more important than peak metrics.