LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset
Studying how people interact with large language models (LLMs) in real-world
scenarios is increasingly important due to their widespread use in various
applications. In this paper, we introduce LMSYS-Chat-1M, a large-scale dataset
containing one million real-world conversations with 25 state-of-the-art LLMs.
This dataset is collected from 210K unique IP addresses in the wild on our
Vicuna demo and Chatbot Arena website. We offer an overview of the dataset's
content, including its curation process, basic statistics, and topic
distribution, highlighting its diversity, originality, and scale. We
demonstrate its versatility through four use cases: developing content
moderation models that perform similarly to GPT-4, building a safety benchmark,
training instruction-following models that perform similarly to Vicuna, and
creating challenging benchmark questions. We believe that this dataset will
serve as a valuable resource for understanding and advancing LLM capabilities.
The dataset is publicly available at
https://huggingface.co/datasets/lmsys/lmsys-chat-1m.
650ce0c54d77a0952255a311
Lianmin Zheng
admin_assigned
2023-09-22T15:08:26.491Z
62d35f3ceaf3858ce253ab7a
https://cdn-avatars.huggingface.co/v1/production/uploads/62d35f3ceaf3858ce253ab7a/3yiiMriltXJBi242FZY-W.jpeg
Lianmin
user
lmzheng
650ce0c54d77a0952255a312
Wei-Lin Chiang
admin_assigned
2023-09-22T10:10:19.030Z
63071772fd79b417f1b92291
/avatars/ec68d20dcedf4553bb31d9f6e0ded813.svg
Wei-Lin Chiang
user
weichiang
650ce0c54d77a0952255a313
Ying Sheng
admin_assigned
2023-09-22T15:03:21.201Z
6455b969654d8bccae50736c
https://cdn-avatars.huggingface.co/v1/production/uploads/6455b969654d8bccae50736c/WsK0_sbQSVeosi-fh-O_U.jpeg
Ying Sheng
user
ying1123
650ce0c54d77a0952255a314
Tianle Li
claimed_verified
2023-09-22T15:03:30.024Z
64ab7f188a1a9187c219e597
https://cdn-avatars.huggingface.co/v1/production/uploads/64ab7f188a1a9187c219e597/cieU4bdmPIn9dTNIVf3n7.jpeg
Tianle Li
user
Timmli
650ce0c54d77a0952255a315
Siyuan Zhuang
650ce0c54d77a0952255a316
Zhanghao Wu
admin_assigned
2023-09-22T15:03:54.858Z
6429b84e8852afdf89beeae5
/avatars/7cfc1a484077a8d511c46392de8dee07.svg
Zhanghao Wu
user
michaelvll
650ce0c54d77a0952255a317
Yonghao Zhuang
admin_assigned
2023-09-22T15:04:02.791Z
64ab987073790912c7ad79e6
/avatars/bfa00418c46da0fd1e522d6561944ec9.svg
Yonghao Zhuang
user
ZYHowell
650ce0c54d77a0952255a318
Zhuohan Li
admin_assigned
2023-09-22T15:04:10.259Z
649dd1fae16abbd8fd7a3bdf
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/2t1p0Ku7qmNcSapRdzxv9.jpeg
Zhuohan  Li
user
zhuohan123
650ce0c54d77a0952255a319
Zi Lin
650ce0c54d77a0952255a31a
Eric. P Xing
admin_assigned
2023-09-22T15:05:00.730Z
64ff67722ad36636be6c4542
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/sLIrNelAWPVOy4e3oo5LB.jpeg
Eric Xing
user
EricX003
650ce0c54d77a0952255a31b
Joseph E. Gonzalez
admin_assigned
2023-09-22T15:06:07.969Z
645d2e8401f4eaab2a0878ce
/avatars/1273c5fb607b4b622a746a42692fa632.svg
Joseph E. Gonzalez
user
ProfJoeyG
650ce0c54d77a0952255a31c
Ion Stoica
650ce0c54d77a0952255a31d
Hao Zhang
admin_assigned
2023-09-22T15:09:17.704Z
62d363143eebd640a4fa41fa
https://cdn-avatars.huggingface.co/v1/production/uploads/62d363143eebd640a4fa41fa/pvPwXlJ5OOb-UIfmffv4E.jpeg
Hao Zhang
user
zhisbug
LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset
Studying how people interact with large language models (LLMs) in real-world
scenarios is increasingly important due to their widespread use in various
applications. In this paper, we introduce LMSYS-Chat-1M, a large-scale dataset
containing one million real-world conversations with 25 state-of-the-art LLMs.
This dataset is collected from 210K unique IP addresses in the wild on our
Vicuna demo and Chatbot Arena website. We offer an overview of the dataset's
content, including its curation process, basic statistics, and topic
distribution, highlighting its diversity, originality, and scale. We
demonstrate its versatility through four use cases: developing content
moderation models that perform similarly to GPT-4, building a safety benchmark,
training instruction-following models that perform similarly to Vicuna, and
creating challenging benchmark questions. We believe that this dataset will
serve as a valuable resource for understanding and advancing LLM capabilities.
The dataset is publicly available at
https://huggingface.co/datasets/lmsys/lmsys-chat-1m.
A large-scale dataset of one million real-world conversations with state-of-the-art LLMs provides a variety of applications including content moderation, safety benchmarking, and instruction-following model training.
large language models LLMs Vicuna demo Chatbot Arena content moderation models safety benchmark instruction-following models