DarkBERT: A Language Model for the Dark Side of the Internet
Recent research has suggested that there are clear differences in the
language used in the Dark Web compared to that of the Surface Web. As studies
on the Dark Web commonly require textual analysis of the domain, language
models specific to the Dark Web may provide valuable insights to researchers.
In this work, we introduce DarkBERT, a language model pretrained on Dark Web
data. We describe the steps taken to filter and compile the text data used to
train DarkBERT to combat the extreme lexical and structural diversity of the
Dark Web that may be detrimental to building a proper representation of the
domain. We evaluate DarkBERT and its vanilla counterpart along with other
widely used language models to validate the benefits that a Dark Web domain
specific model offers in various use cases. Our evaluations show that DarkBERT
outperforms current language models and may serve as a valuable resource for
future research on the Dark Web.
6462f02dd9b4902188122137
Youngjin Jin
claimed_verified
2023-05-17T08:04:30.119Z
6200c0ea93ccb1407fe4e57a
/avatars/af937bea147867c18de387c1e50259e1.svg
Youngjin Jin
user
Youngjin
6462f02dd9b4902188122138
Eugene Jang
claimed_verified
2024-01-25T08:05:51.394Z
64a3ad72ed01f3dea26bfde6
https://cdn-avatars.huggingface.co/v1/production/uploads/64a3ad72ed01f3dea26bfde6/Z7TmrRjUUukKsZ_jRWT2-.png
Eugene Jang
user
genesith
6462f02dd9b4902188122139
Jian Cui
6462f02dd9b490218812213a
Jin-Woo Chung
6462f02dd9b490218812213b
Yongjae Lee
6462f02dd9b490218812213c
Seungwon Shin
DarkBERT: A Language Model for the Dark Side of the Internet
Recent research has suggested that there are clear differences in the
language used in the Dark Web compared to that of the Surface Web. As studies
on the Dark Web commonly require textual analysis of the domain, language
models specific to the Dark Web may provide valuable insights to researchers.
In this work, we introduce DarkBERT, a language model pretrained on Dark Web
data. We describe the steps taken to filter and compile the text data used to
train DarkBERT to combat the extreme lexical and structural diversity of the
Dark Web that may be detrimental to building a proper representation of the
domain. We evaluate DarkBERT and its vanilla counterpart along with other
widely used language models to validate the benefits that a Dark Web domain
specific model offers in various use cases. Our evaluations show that DarkBERT
outperforms current language models and may serve as a valuable resource for
future research on the Dark Web.
DarkBERT, a language model trained on Dark Web data, outperforms existing models in various use cases due to its specialized training on the Dark Web's unique linguistic characteristics.
language model Dark Web DarkBERT textual analysis lexical diversity structural diversity vanilla counterpart