Hugging Face's logo

ggunio
/
B2NL-IntelligentTokenizer-v6.2.1

PyTorch
Safetensors
b2nl_tokenizer
tokenizer
byte-to-nl
compression
embeddings
semantic-boundaries
Eval Results
Model card Files
xet
Community
B2NL-IntelligentTokenizer-v6.2.1

Commit History

Update README.md
4df097b
verified

ggunio commited on

Update README.md
d56648b
verified

ggunio commited on

Update README.md
26443d3
verified

ggunio commited on

Update: Professional model card with accurate performance metrics and proper English documentation
57f34d3
verified

ggunio commited on

์ •์งํ•œ ์„ฑ๋Šฅ ๊ณต๊ฐœ: UTF-8 ๊ฒฝ๊ณ„ ๋ฒ„๊ทธ, ํ•œ๊ธ€ ๋ฌธ์ œ, 48๋ฐ”์ดํŠธ ์ œํ•œ ๋ช…์‹œ
ea406a8
verified

ggunio commited on

์ •ํ™•ํ•œ ์ •๋ณด๋กœ ์—…๋ฐ์ดํŠธ: FLORES-200๋งŒ ์‚ฌ์šฉ, RTX 4070, LinkedIn ์ˆ˜์ •, ํ˜„์žฌ ํ•œ๊ณ„ ๋ช…์‹œ
4355434
verified

ggunio commited on

Fix: Correct embedding dimension to 1280
d2f40f4
verified

ggunio commited on

Complete documentation: semantic boundaries, padding explanation, LinkedIn link, full technical details
3c31be2
verified

ggunio commited on

Fix: Remove Korean text error and exaggerations, add proper citations
6a2d9f3
verified

ggunio commited on

Fix: Remove incorrect dataset reference, add actual training data info
63a97a3
verified

ggunio commited on

Initial upload: B2NL-IntelligentTokenizer v6.2.1 (Autoregressive Mode)
ffbd655
verified

ggunio commited on

initial commit
7b8dd8f
verified

ggunio commited on