nineninesix/kani-tts-400m-ky

KaniTTS Kyrgyz

A high-speed, high-fidelity Text-to-Speech model optimized for real-time conversational AI applications.

Overview

KaniTTS uses a two-stage pipeline combining a large language model with an efficient audio codec for exceptional speed and audio quality. The architecture generates compressed token representations through a backbone LLM, then rapidly synthesizes waveforms via neural audio codec, achieving extremely low latency.

Key Specifications:

• Model Size: 400M parameters
• Sample Rate: 22kHz
• Language: Kyrgyz
• License: Apache 2.0

Performance

On NovitaAI RTX 5090 using vLLM:

• RTF: ~0.2 (5 times faster than realtime)
• Memory: 16GB GPU VRAM used
• Source Code: https://github.com/nineninesix-ai/kanitts-vllm

GPU Benchmark Results

GPU Model	VRAM	Cost ($/hr)	RTF
RTX 5090	32GB	$0.423	0.190
RTX 4080	16GB	$0.220	0.200
RTX 5060 Ti	16GB	$0.138	0.529
RTX 4060 Ti	16GB	$0.122	0.537
RTX 3060	12GB	$0.093	0.600

Lower RTF is better (< 1.0 means faster than real-time). Benchmarks conducted on Vast AI.

Datasets

• https://huggingface.co/datasets/laion/Emolia

Voices:

• syimyk
• elina

Audio Examples

Text	Audio
Салам, менин атым Каныкей! Кантип жардам бере алам силерге?
Ысык-Көлдүн жээгинде күн батканда, асман көгүлтүр алтынга айланат — көз жоосун алган сулуулук!
Тоолорду карачы, канчалык бийик болсо да, кыргыздын руху андан да бийик!
Бишкек бүгүн кандай сонун, ээ? – Ооба, шамал да жумшак, асман да ачык!
Ой, бозо менен самсанын жыты чыгып кетти, ачка болуп кеттим го!

Emotion Control Tags

During fine-tuning, the dataset included special emotional control tokens such as

<laughs>
<giggles>
<sighs>

These tags help the model adjust tone, rhythm, and expressive style during speech synthesis.

Use Cases

• Conversational AI: Real-time speech for chatbots and virtual assistants
• Edge/Server Deployment: Resource-efficient inference on affordable hardware
• Accessibility: Screen readers and language learning applications
• Research: Fine-tuning for specific voices, accents, or emotions

Limitations

• Performance degrades with inputs exceeding 15 seconds (need to use sliding window chunking)
• Limited expressivity without fine-tuning for specific emotions
• May inherit biases from training data in prosody or pronunciation
• Optimized primarily for English; other languages may require additional training

Optimization Tips

• Multilingual Performance: Continually pretrain on target language datasets and fine-tune NanoCodec
• Batch Processing: Use batches of 8-16 for high-throughput scenarios
• Hardware: Optimized for NVIDIA Blackwell architecture GPUs

Resources

Models:

• Pretrained Model: nineninesix/kani-tts-400m-0.3-pt
• Space: nineninesix/KaniTTS-KyrGyz

Examples:

• OpenAI compatible API Example: https://github.com/nineninesix-ai/kanitts-vllm
• Finetuning code pipeline: https://github.com/nineninesix-ai/KaniTTS-Finetune-pipeline
• Dataset preparation pipeline: https://github.com/nineninesix-ai/nano-codec-dataset-pipeline
• Example Dataset: https://huggingface.co/datasets/nineninesix/expresso-conversational-en-nano-codec-dataset
• GitHub Repository: https://github.com/nineninesix-ai/kani-tts
• ComfyUI node: https://github.com/wildminder/ComfyUI-KaniTTS by WildAi
• NextJS basic app: https://github.com/nineninesix-ai/open-audio. It uses the OpenAI npm package to connect to the OpenAI-compatible server API provided by kanitts-vllm.

Links:

• Website: https://www.nineninesix.ai
• Contact Form: https://airtable.com/appX2G2TpoRk4M5Bf/pagO2xbIOjiwulPcP/form

Acknowledgments

Built on top of LiquidAI LFM2 350M as the backbone and Nvidia NanoCodec for audio processing.

Responsible Use

Prohibited activities include:

• Illegal content or harmful, threatening, defamatory, or obscene material
• Hate speech, harassment, or incitement of violence
• Generating false or misleading information
• Impersonating individuals without consent
• Malicious activities such as spamming, phishing, or fraud By using this model, you agree to comply with these restrictions and all applicable laws.

Contact

Have a question, feedback, or need support? Please fill out our contact form and we'll get back to you as soon as possible.

Citation

@inproceedings{emilialarge,
  author={He, Haorui and Shang, Zengqiang and Wang, Chaoren and Li, Xuyuan and Gu, Yicheng and Hua, Hua and Liu, Liwei and Yang, Chen and Li, Jiaqi and Shi, Peiyang and Wang, Yuancheng and Chen, Kai and Zhang, Pengyuan and Wu, Zhizheng},
  title={Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generation},
  booktitle={arXiv:2501.15907},
  year={2025}
}

@article{emonet_voice_2025,
  author={Schuhmann, Christoph and Kaczmarczyk, Robert and Rabby, Gollam and Friedrich, Felix and Kraus, Maurice and Nadi, Kourosh and Nguyen, Huu and Kersting, Kristian and Auer, Sören},
  title={EmoNet-Voice: A Fine-Grained, Expert-Verified Benchmark for Speech Emotion Detection},
  journal={arXiv preprint arXiv:2506.09827},
  year={2025}
}