🇮🇳 Bharat-Tiny-LLM

India's first native edge AI for Hinglish & Hindi — running fully offline on ₹8,000 phones.
A 1.5B-parameter, LoRA-fine-tuned model that speaks fluent Hinglish (Romanized Hindi) and Devanagari Hindi, entirely on-device.

---

✨ Why Bharat-Tiny-LLM?

Most Indian-language models do one script. Bharat-Tiny-LLM does both — and does it on hardware that costs less than a pair of shoes.

• 🌐 Truly bilingual — Hinglish and Devanagari Hindi from a single 1.5B model.
• 📱 Edge-native — an 880 MB 4-bit MLX build runs offline on ₹8,000 Android phones & Apple Silicon. No API, no cloud, no internet.
• 🪶 Small but real — 1.5B params, ~57 tok/s on a Mac Mini M4.
• 💸 $0 training cost — fine-tuned on a Mac Mini M4, zero cloud compute.
• 🆓 Open weights — Apache-2.0, self-hostable, no vendor lock-in.

Built by eulogik — an India-first AI lab shipping practical, edge-deployable models.

---

📊 Model details

Property	Value
Base model	Qwen2.5-1.5B (multilingual, 29 languages)
Method	LoRA — 16 layers, rank 16, alpha 32, scale 2.0, dropout 0.05
Trainable params	10.55M (0.68% of 1.5B)
Training data	436K cleaned Hinglish/Devanagari conversations (gold + NebulaByte + smangrul)
Training iters	110,000 (Mac Mini M4, batch 4, seq 768)
Best val loss	0.937
Quantization	MLX 4-bit affine, group size 64 (~880 MB)
Inference speed	~57 tok/s on Mac Mini M4
License	Apache-2.0

---

🚀 Quick start

Python (recommended)

pip install bharat-tiny-llm[mlx]   # Apple Silicon
pip install bharat-tiny-llm[torch] # CPU / CUDA

from bharat_tiny_llm import chat

print(chat([{"role": "user", "content": "Chai peete hain?"}]))

The package ships with a canonical generation config so output is clean out of the box:

temperature=0.3, top_p=0.85, repetition_penalty=1.25, no_repeat_ngram_size=3

MLX CLI

pip install mlx-lm
mlx_lm chat --model eulogik/Bharat-Tiny-LLM

Transformers (fp16 fused variant)

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("eulogik/Bharat-Tiny-LLM-fused")
tokenizer = AutoTokenizer.from_pretrained("eulogik/Bharat-Tiny-LLM-fused")

⚠️ Generation config matters. The base Qwen2.5-1.5B emits garbled out-of-script tokens at high temperature. Always use temperature ≈ 0.3 + repetition_penalty ≥ 1.25

• no_repeat_ngram_size = 3. The bharat-tiny-llm package and the demo space do this for you.

llama.cpp (GGUF — Android / Pi / CPU)

# download Q4_K_M GGUF
pip install llama-cpp-python

from llama_cpp import Llama
llm = Llama(model_path="bharat-tiny-llm-q4_k_m.gguf", n_ctx=1024)
print(llm.create_chat_completion(
    messages=[{"role": "user", "content": "Chai peete hain?"}],
    temperature=0.3, top_p=0.85, max_tokens=256, repeat_penalty=1.25,
)["choices"][0]["message"]["content"])

Get the GGUF from eulogik/Bharat-Tiny-LLM-GGUF.

---

💬 Example outputs

Generated with the 4-bit MLX build at the canonical settings above.

Q: Chai peete hain? A: Haan bilkul! Chai toh humari life hai. Subah uthte hi pehli cheez jo dimag mein aati hai woh chai hi hoti hai...

Q: Kal interview hai, nervous ho raha hoon A: Okay chill karo yaar! Interview mein sabse pehle confidence dikhana hota hai. Deep breath lo, apna intro clear rakho...

Q: Biryani kaise banate hain? A: Biryani ek dum simple process hai, jismein kuchh ingredients hote hain aur cooking time bhi kam hota hai. Sabse important cheez hai ingredients...

Q: Ghar baithe paise kaise kamayein? A: Haan, bilkul sahi sawaal hai! Ghar baithe paise kaise kamayein? Yeh toh bahut common sawaal hai aaj kal. Tension nahi lene ka, main tumhe seedha-seedha points mein batata hoon...

Q: भारत के कुछ प्रसिद्ध त्योहार बताओ। A: भारत के प्रसिद्ध त्योहार हैं: दीवाली, होली, रक्षा बंधन, ईद, नवरात्रि और बहुत से और। हर त्योहार का अपना अलग महत्व है...

---

🗂️ Model variants

Repo	Format	Size	Use
eulogik/Bharat-Tiny-LLM	MLX 4-bit	~880 MB	Edge / Apple Silicon (this repo)
eulogik/Bharat-Tiny-LLM-GGUF	GGUF Q4_K_M	~1.06 GB	Cross-platform (llama.cpp, Android, Pi, CPU)
eulogik/Bharat-Tiny-LLM-fused	PyTorch fp16	~3.3 GB	Server / fine-tuning base

---

🧩 Limitations

• 1.5B params: great for short conversational replies, weaker on long factual essays or complex reasoning.
• Trained primarily on Hinglish + Hindi; other Indian languages are inherited from the Qwen2.5 base, not fine-tuned.
• Quantization (4-bit) trades a little fluency for a 4× size reduction.
• May occasionally repeat or drift on very long generations — keep max_new_tokens modest and use the n-gram guard.

🛣️ Roadmap

• More Indic languages (Tamil, Telugu, Bengali, Marathi)
• DPO / preference tuning for a quality leap
• BharatTiny-Bench — an Indic edge-model evaluation suite
• GGUF build for broader Android / Raspberry Pi deployment

---

🔗 Links

• 🤗 Model card: https://huggingface.co/eulogik/Bharat-Tiny-LLM
• 🤗 GGUF (llama.cpp): https://huggingface.co/eulogik/Bharat-Tiny-LLM-GGUF
• 🤗 fp16 fused: https://huggingface.co/eulogik/Bharat-Tiny-LLM-fused
• 🚀 Demo space: https://huggingface.co/spaces/eulogik/Bharat-Tiny-LLM
• 💻 Source code: https://github.com/eulogik/Bharat-Tiny-LLM
• 📦 PyPI: https://pypi.org/project/bharat-tiny-llm/
• 🏢 Built by eulogik

📜 License

Apache-2.0. Base weights Qwen2.5-1.5B are Apache-2.0; the LoRA adapter is Apache-2.0.