Incorporating Domain Knowledge into Materials Tokenization
While language models are increasingly utilized in materials science, typical
models rely on frequency-centric tokenization methods originally developed for
natural language processing. However, these methods frequently produce
excessive fragmentation and semantic loss, failing to maintain the structural
and semantic integrity of material concepts. To address this issue, we propose
MATTER, a novel tokenization approach that integrates material knowledge into
tokenization. Based on MatDetector trained on our materials knowledge base and
a re-ranking method prioritizing material concepts in token merging, MATTER
maintains the structural integrity of identified material concepts and prevents
fragmentation during tokenization, ensuring their semantic meaning remains
intact. The experimental results demonstrate that MATTER outperforms existing
tokenization methods, achieving an average performance gain of 4% and 2%
in the generation and classification tasks, respectively. These results
underscore the importance of domain knowledge for tokenization strategies in
scientific text processing. Our code is available at
https://github.com/yerimoh/MATTER
685107725e07650ecce891d6
Yerim Oh
claimed_verified
2025-06-17T07:20:35.446Z
65e746079cf349af294e1f10
/avatars/0a3f13e03b1f9249595b387001203908.svg
yerim Oh
user
yerim0210
685107725e07650ecce891d7
Jun-Hyung Park
685107725e07650ecce891d8
Junho Kim
685107725e07650ecce891d9
SungHo Kim
685107725e07650ecce891da
SangKeun Lee
Incorporating Domain Knowledge into Materials Tokenization
While language models are increasingly utilized in materials science, typical
models rely on frequency-centric tokenization methods originally developed for
natural language processing. However, these methods frequently produce
excessive fragmentation and semantic loss, failing to maintain the structural
and semantic integrity of material concepts. To address this issue, we propose
MATTER, a novel tokenization approach that integrates material knowledge into
tokenization. Based on MatDetector trained on our materials knowledge base and
a re-ranking method prioritizing material concepts in token merging, MATTER
maintains the structural integrity of identified material concepts and prevents
fragmentation during tokenization, ensuring their semantic meaning remains
intact. The experimental results demonstrate that MATTER outperforms existing
tokenization methods, achieving an average performance gain of 4% and 2%
in the generation and classification tasks, respectively. These results
underscore the importance of domain knowledge for tokenization strategies in
scientific text processing. Our code is available at
https://github.com/yerimoh/MATTER
MATTER, a novel tokenization approach incorporating material knowledge, improves performance in scientific text processing tasks by maintaining structural and semantic material integrity.
MATTER MatDetector tokenization material knowledge token merging semantic integrity generation tasks classification tasks