Introducing v0.5 of the AI Safety Benchmark from MLCommons
This paper introduces v0.5 of the AI Safety Benchmark, which has been created
by the MLCommons AI Safety Working Group. The AI Safety Benchmark has been
designed to assess the safety risks of AI systems that use chat-tuned language
models. We introduce a principled approach to specifying and constructing the
benchmark, which for v0.5 covers only a single use case (an adult chatting to a
general-purpose assistant in English), and a limited set of personas (i.e.,
typical users, malicious users, and vulnerable users). We created a new
taxonomy of 13 hazard categories, of which 7 have tests in the v0.5 benchmark.
We plan to release version 1.0 of the AI Safety Benchmark by the end of 2024.
The v1.0 benchmark will provide meaningful insights into the safety of AI
systems. However, the v0.5 benchmark should not be used to assess the safety of
AI systems. We have sought to fully document the limitations, flaws, and
challenges of v0.5. This release of v0.5 of the AI Safety Benchmark includes
(1) a principled approach to specifying and constructing the benchmark, which
comprises use cases, types of systems under test (SUTs), language and context,
personas, tests, and test items; (2) a taxonomy of 13 hazard categories with
definitions and subcategories; (3) tests for seven of the hazard categories,
each comprising a unique set of test items, i.e., prompts. There are 43,090
test items in total, which we created with templates; (4) a grading system for
AI systems against the benchmark; (5) an openly available platform, and
downloadable tool, called ModelBench that can be used to evaluate the safety of
AI systems on the benchmark; (6) an example evaluation report which benchmarks
the performance of over a dozen openly available chat-tuned language models;
(7) a test specification for the benchmark.
6621ced43ec06d6503eecd9e
Bertie Vidgen
6621ced43ec06d6503eecd9f
Adarsh Agrawal
6621ced43ec06d6503eecda0
Ahmed M. Ahmed
6621ced43ec06d6503eecda1
Victor Akinwande
admin_assigned
2024-04-19T08:54:25.019Z
643b07dcb54b346556532841
/avatars/7806b22b86a22306d5944e75a822787f.svg
Victor Akinwande
user
vakinwan
6621ced43ec06d6503eecda2
Namir Al-Nuaimi
6621ced43ec06d6503eecda3
Najla Alfaraj
6621ced43ec06d6503eecda4
Elie Alhajjar
6621ced43ec06d6503eecda5
Lora Aroyo
admin_assigned
2024-04-19T08:54:42.242Z
659d7258dde639191fa89a34
/avatars/ccbe525e7444aba27b7f4e1d91553c65.svg
Lora Aroyo
user
laroyo
6621ced43ec06d6503eecda6
Trupti Bavalatti
6621ced43ec06d6503eecda7
Borhane Blili-Hamelin
admin_assigned
2024-04-19T08:54:52.910Z
64149f87634b1a9b24cd237b
https://cdn-avatars.huggingface.co/v1/production/uploads/1679073077831-noauth.jpeg
Borhane Blili-Hamelin
user
borhane
6621ced43ec06d6503eecda8
Kurt Bollacker
admin_assigned
2024-04-19T08:54:59.655Z
6570a4139c503c4a6af43279
/avatars/315c4bf8c5acd44364c8f899c40d6a02.svg
Kurt Bollacker
user
bollacker
6621ced43ec06d6503eecda9
Rishi Bomassani
6621ced43ec06d6503eecdaa
Marisa Ferrara Boston
claimed_verified
2025-03-12T20:46:31.914Z
65303a00d849382d5559a67e
/avatars/a599ca55c7aeee37f4298272cfcb064d.svg
Marisa
user
maReins
6621ced43ec06d6503eecdab
Siméon Campos
admin_assigned
2024-04-19T08:55:20.028Z
61740ed8ab394c394536a082
/avatars/bed4740530478b643440b8725b001340.svg
Simeon Campos
user
WayZNLP
6621ced43ec06d6503eecdac
Kal Chakra
admin_assigned
2024-04-19T08:55:26.523Z
64e0666a8e2084e1d7c77c19
/avatars/3e0a9c71f0f0ed9315ccaef46625fbfc.svg
Kal Chakra
user
kalchakra
6621ced43ec06d6503eecdad
Canyu Chen
admin_assigned
2024-04-19T08:55:33.404Z
6483af58571c2dcfa98cae82
https://cdn-avatars.huggingface.co/v1/production/uploads/6483af58571c2dcfa98cae82/d8Vnh_EG1KxeEoNAAQXHZ.jpeg
Canyu Chen
user
canyuchen
6621ced43ec06d6503eecdae
Cody Coleman
6621ced43ec06d6503eecdaf
Zacharie Delpierre Coudert
6621ced43ec06d6503eecdb0
Leon Derczynski
admin_assigned
2024-04-19T08:55:54.536Z
61caf9d13918092781f7b745
https://cdn-avatars.huggingface.co/v1/production/uploads/1640692164586-noauth.jpeg
Leon Derczynski
user
leondz
6621ced43ec06d6503eecdb1
Debojyoti Dutta
admin_assigned
2024-04-19T08:56:06.895Z
6191539da26ced2814661ec8
/avatars/ac5fd0906704549d69a72834866dc8fc.svg
debojyoti dutta
user
ddutta
6621ced43ec06d6503eecdb2
Ian Eisenberg
admin_assigned
2024-04-19T08:56:12.889Z
62a954409f60be260fbfa5b9
/avatars/22f914656ba9c840b32812bca7104a17.svg
Ian Eisenberg
user
IanE
6621ced43ec06d6503eecdb3
James Ezick
6621ced43ec06d6503eecdb4
Heather Frase
admin_assigned
2024-04-19T08:56:22.184Z
647dd166e4d52fe0e01cb11b
/avatars/934ac7ecacd607585585bbcbaa991143.svg
Heather Frase
user
Hnfrase
6621ced43ec06d6503eecdb5
Brian Fuller
6621ced43ec06d6503eecdb6
Ram Gandikota
6621ced43ec06d6503eecdb7
Agasthya Gangavarapu
admin_assigned
2024-04-19T08:56:53.552Z
64ed26cf476cdc850285d596
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/GvS4Z1kAuocgy385AnL6n.jpeg
Agasthya Gangavarapu
user
itzAugust
6621ced43ec06d6503eecdb8
Ananya Gangavarapu
admin_assigned
2024-04-19T08:57:01.200Z
64e3e777d5e75575cd79fe59
/avatars/091ed4cfdf8442b06800580872684e1e.svg
Ananya Gangavarapu
user
anigasan
6621ced43ec06d6503eecdb9
James Gealy
6621ced43ec06d6503eecdba
Rajat Ghosh
6621ced43ec06d6503eecdbb
James Goel
6621ced43ec06d6503eecdbc
Usman Gohar
6621ced43ec06d6503eecdbd
Sujata Goswami
6621ced43ec06d6503eecdbe
Scott A. Hale
6621ced43ec06d6503eecdbf
Wiebke Hutiri
admin_assigned
2024-04-19T09:06:10.711Z
640aff86b871d7117d612d1e
/avatars/740e7328e379e84d48b70a4230f9b8aa.svg
Wiebke Hutiri
user
wiebke-codes
6621ced43ec06d6503eecdc0
Joseph Marvin Imperial
claimed_verified
2024-05-21T07:24:36.416Z
620a1e42e190de4b8f27ba1f
https://cdn-avatars.huggingface.co/v1/production/uploads/620a1e42e190de4b8f27ba1f/jhRHn5o5A_HLO-iYonZAl.png
Joseph Imperial
user
josephimperial
6621ced43ec06d6503eecdc1
Surgan Jandial
6621ced43ec06d6503eecdc2
Nick Judd
6621ced43ec06d6503eecdc3
Felix Juefei-Xu
claimed_verified
2024-12-16T16:55:51.175Z
6417cf37dce1e4c0229f17b1
https://cdn-avatars.huggingface.co/v1/production/uploads/6417cf37dce1e4c0229f17b1/7h-ZCB5f4wif7TsnF-B1M.jpeg
Felix Xu
user
katanaxu
6621ced43ec06d6503eecdc4
Foutse Khomh
admin_assigned
2024-04-19T09:05:47.554Z
65d60476876b7d939870c5af
/avatars/57a35d093d25ac084ebffcaf9e3599ee.svg
Foutse Khomh
user
foutsekh
6621ced43ec06d6503eecdc5
Bhavya Kailkhura
admin_assigned
2024-04-19T09:05:54.753Z
65cb79db6427380bc21261e2
/avatars/a003eb5d0955417329c1a4170ae65879.svg
Bhavya Kailkhura
user
bhavyakailkhura
6621ced43ec06d6503eecdc6
Hannah Rose Kirk
admin_assigned
2024-04-19T09:06:01.953Z
6255baa66b1f65958caf026c
https://cdn-avatars.huggingface.co/v1/production/uploads/1649842998853-6255baa66b1f65958caf026c.jpeg
Hannah Rose Kirk
user
HannahRoseKirk
6621ced43ec06d6503eecdc7
Kevin Klyman
admin_assigned
2024-04-19T09:05:11.282Z
64817160ec65b8b77d83e910
/avatars/0dc892cccb8c71a1ad1c1a1801ee9772.svg
Kevin Klyman
user
kklyman
6621ced43ec06d6503eecdc8
Chris Knotz
admin_assigned
2024-04-19T09:05:04.418Z
65f8aeebc1823e23cf92aa97
/avatars/4a3a81d466bd194945b1b59dcdfbdad9.svg
Chris Knotz
user
ChrisKnotz
6621ced43ec06d6503eecdc9
Michael Kuchnik
admin_assigned
2024-04-19T09:04:57.028Z
63bc3dad1374e3ef9131d75c
/avatars/d58f700ac56b6bf11e18aaf5f8d2691c.svg
Michael Kuchnik
user
mkuchnik
6621ced43ec06d6503eecdca
Shachi H. Kumar
6621ced43ec06d6503eecdcb
Chris Lengerich
6621ced43ec06d6503eecdcc
Bo Li
6621ced43ec06d6503eecdcd
Zeyi Liao
claimed_verified
2025-06-02T07:59:20.810Z
632ed3e534c562cb1d328fa7
/avatars/e29e47e734740787ea40c729eec146b5.svg
zeyi liao
user
lzy337
6621ced43ec06d6503eecdce
Eileen Peters Long
6621ced43ec06d6503eecdcf
Victor Lu
6621ced43ec06d6503eecdd0
Yifan Mai
admin_assigned
2024-04-19T09:05:19.831Z
645beb41f9d4ec91fdd935d9
/avatars/a4bddecae3096088613b915b42354b81.svg
Yifan Mai
user
yifanmai
6621ced43ec06d6503eecdd1
Priyanka Mary Mammen
admin_assigned
2024-04-19T09:05:27.381Z
66036ac2fb66ef4a5cf82fe5
/avatars/d9e9bd3b5e651453b3bc76fbfdf8cf97.svg
Priyanka Mary Mammen
user
pmammen
6621ced43ec06d6503eecdd2
Kelvin Manyeki
6621ced43ec06d6503eecdd3
Sean McGregor
6621ced43ec06d6503eecdd4
Virendra Mehta
claimed_verified
2024-09-23T13:07:16.939Z
604eb19e3050a33ebb17ef58
/avatars/23650002ba3befee83060fe978a251c8.svg
Virendra Mehta
user
Veeru
6621ced43ec06d6503eecdd5
Shafee Mohammed
admin_assigned
2024-04-19T09:03:50.738Z
62bf8e6be140faaa8532a128
/avatars/c0a442a952f372d33b9f517cfad36500.svg
shafee mohammed
user
shafcodes
6621ced43ec06d6503eecdd6
Emanuel Moss
6621ced43ec06d6503eecdd7
Lama Nachman
6621ced43ec06d6503eecdd8
Dinesh Jinenhally Naganna
6621ced43ec06d6503eecdd9
Amin Nikanjam
admin_assigned
2024-04-19T09:02:46.503Z
65d754e96f1ee3a269bfa30d
/avatars/4d579a4a23ef8618b929a50ca4c172fb.svg
Amin Nikanjam
user
aminnikanjam
6621ced43ec06d6503eecdda
Besmira Nushi
admin_assigned
2024-04-19T09:02:53.348Z
638ee2ed848625f9fad11034
https://cdn-avatars.huggingface.co/v1/production/uploads/1670309301345-638ee2ed848625f9fad11034.jpeg
Besmira Nushi
user
nushib
6621ced43ec06d6503eecddb
Luis Oala
admin_assigned
2024-04-19T09:03:00.478Z
61170637468eb5fb52c297cd
https://cdn-avatars.huggingface.co/v1/production/uploads/1640126895078-61170637468eb5fb52c297cd.jpeg
Luis Oala
user
luisoala
6621ced43ec06d6503eecddc
Iftach Orr
6621ced43ec06d6503eecddd
Alicia Parrish
admin_assigned
2024-04-19T09:03:13.916Z
659d71eaf0ae1a57ae140831
/avatars/b9eb8c70b4fe5dbd0f1555602ae41b65.svg
Alicia Parrish
user
avparrish
6621ced43ec06d6503eecdde
Cigdem Patlak
admin_assigned
2024-04-19T09:03:31.186Z
65ba06bbb4ff0b04a992190d
/avatars/98ddfef042298671705289d15247bfbd.svg
Cigdem Patlak
user
Cigdemp
6621ced43ec06d6503eecddf
William Pietri
admin_assigned
2024-04-19T09:03:23.709Z
5f19c119925b9863e28ad645
https://cdn-avatars.huggingface.co/v1/production/uploads/1595523321955-noauth.png
William Pietri
user
wpietri
6621ced43ec06d6503eecde0
Forough Poursabzi-Sangdeh
6621ced43ec06d6503eecde1
Eleonora Presani
admin_assigned
2024-04-19T09:02:09.187Z
64b19e3d93679cd00911cdd1
/avatars/bab5a4543ce4f657415f7f36ca4fc6fb.svg
Eleonora Presani
user
epresani
6621ced43ec06d6503eecde2
Fabrizio Puletti
6621ced43ec06d6503eecde3
Paul Röttger
admin_assigned
2024-04-19T09:02:00.949Z
602ce925374a0dbe5856eca1
https://cdn-avatars.huggingface.co/v1/production/uploads/602ce925374a0dbe5856eca1/lBvTn6hYOCF0ggD-rk_mf.jpeg
Paul Röttger
user
Paul
6621ced43ec06d6503eecde4
Saurav Sahay
6621ced43ec06d6503eecde5
Tim Santos
6621ced43ec06d6503eecde6
Nino Scherrer
admin_assigned
2024-04-19T09:01:43.629Z
643dc14d37ca9134b589bfe0
/avatars/998977d708f205dd5df430705e55ebc9.svg
Nino Scherrer
user
ninoscherrer
6621ced43ec06d6503eecde7
Alice Schoenauer Sebag
6621ced43ec06d6503eecde8
Patrick Schramowski
admin_assigned
2024-04-19T09:01:25.795Z
62d021a3dd7bdfc5e5c61c5c
https://cdn-avatars.huggingface.co/v1/production/uploads/62d021a3dd7bdfc5e5c61c5c/bnQW2SqirfGaQmI84HW_c.jpeg
Patrick Schramowski
user
PSaiml
6621ced43ec06d6503eecde9
Abolfazl Shahbazi
admin_assigned
2024-04-19T09:01:16.527Z
632a432774bc199012a0c84b
/avatars/f1658bf465d72fe167050fd00b650769.svg
Abolfazl Shahbazi
user
ashahba
6621ced43ec06d6503eecdea
Vin Sharma
admin_assigned
2024-04-19T09:01:08.609Z
651dbab62c6fac9717218917
/avatars/5d8eeacb4d37e716192ff693667cb328.svg
Vin Sharma
user
ciphertext
6621ced43ec06d6503eecdeb
Xudong Shen
admin_assigned
2024-04-19T09:01:01.731Z
63c628610be43255f36f2a40
https://cdn-avatars.huggingface.co/v1/production/uploads/1673930799006-noauth.jpeg
Xudong Shen
user
XudongShen
6621ced43ec06d6503eecdec
Vamsi Sistla
admin_assigned
2024-04-19T09:00:53.803Z
640a093996aae6497413e052
/avatars/d9a979c1cd83fb8c7cbfdec49c67f88c.svg
Vamsi Sistla
user
Vsistla
6621ced43ec06d6503eecded
Leonard Tang
admin_assigned
2024-04-19T09:00:45.603Z
64c13ee9e98a5e02c93459ee
https://cdn-avatars.huggingface.co/v1/production/uploads/64c13ee9e98a5e02c93459ee/o7huQQPXZ5vS8r5RVDvWP.png
Leonard Tang
user
leonardtang
6621ced43ec06d6503eecdee
Davide Testuggine
admin_assigned
2024-04-19T09:00:34.663Z
64e927131d8b651a9ac0bee4
/avatars/c74a3d2b8ffd75afcad35da39f5fad51.svg
Davide Testuggine
user
darktex
6621ced43ec06d6503eecdef
Vithursan Thangarasa
admin_assigned
2024-04-19T09:00:24.483Z
64236f9523b98d5ee6ea2f61
/avatars/e7f8156637aa8b0d36ce1954eac9dd77.svg
Vithursan Thangarasa
user
vithursant
6621ced43ec06d6503eecdf0
Elizabeth Anne Watkins
6621ced43ec06d6503eecdf1
Rebecca Weiss
admin_assigned
2024-04-19T09:00:07.191Z
6430688e0d7e3248d0616a64
/avatars/6d3ff97af3dd0da6f4781523e8cb2778.svg
Rebecca Weiss
user
rjweiss
6621ced43ec06d6503eecdf2
Chris Welty
6621ced43ec06d6503eecdf3
Tyler Wilbers
admin_assigned
2024-04-19T08:59:54.125Z
648bd280b1793ce134986343
/avatars/4c44a1b87e3a421a628a01956498a30d.svg
Tyler Wilbers
user
tybrs
6621ced43ec06d6503eecdf4
Adina Williams
admin_assigned
2024-04-19T08:59:46.979Z
6487618920fb2e386b73e5d2
https://cdn-avatars.huggingface.co/v1/production/uploads/6487618920fb2e386b73e5d2/ZnmBCV-gL3eMeDGC38Rz8.jpeg
Adina Williams
user
adinawilliams2
6621ced43ec06d6503eecdf5
Carole-Jean Wu
6621ced43ec06d6503eecdf6
Poonam Yadav
6621ced43ec06d6503eecdf7
Xianjun Yang
claimed_verified
2024-04-19T07:38:33.285Z
604137e5c15e823a685f2af2
https://cdn-avatars.huggingface.co/v1/production/uploads/604137e5c15e823a685f2af2/rrxZUyrbLpKqCmG6xtP8v.jpeg
Xianjun Yang
user
Xianjun
6621ced43ec06d6503eecdf8
Yi Zeng
admin_assigned
2024-04-19T08:59:30.689Z
63baf3da9726f7e58f8da2de
https://cdn-avatars.huggingface.co/v1/production/uploads/1673196486033-noauth.jpeg
Yi Zeng
user
yizeng
6621ced43ec06d6503eecdf9
Wenhui Zhang
admin_assigned
2024-04-19T08:58:46.371Z
658973b761f2dd8f6698c149
/avatars/a6225d23b4f6508e7a799c08efac5464.svg
Wenhui Zhang
user
Shenqibaobei
6621ced43ec06d6503eecdfa
Fedor Zhdanov
6621ced43ec06d6503eecdfb
Jiacheng Zhu
admin_assigned
2024-04-19T08:58:05.110Z
653801b167325b6218ddfdc8
https://cdn-avatars.huggingface.co/v1/production/uploads/653801b167325b6218ddfdc8/y3DjEDpcvgnRC30201dFc.jpeg
Jiacheng Zhu
user
JiachengZhu
6621ced43ec06d6503eecdfc
Percy Liang
admin_assigned
2024-04-19T08:57:31.374Z
6409651b9e9f790c905b2335
/avatars/1fb8c80b60f21f65a0a027319101f236.svg
Percy Liang
user
percyliang
6621ced43ec06d6503eecdfd
Peter Mattson
6621ced43ec06d6503eecdfe
Joaquin Vanschoren
admin_assigned
2024-04-19T08:57:17.608Z
6380c629e479bc19cbd4df0e
/avatars/146df6eb6b1a94e7bdf8c679bfd171af.svg
Joaquin Vanschoren
user
JoaquinVanschoren
Introducing v0.5 of the AI Safety Benchmark from MLCommons
This paper introduces v0.5 of the AI Safety Benchmark, which has been created
by the MLCommons AI Safety Working Group. The AI Safety Benchmark has been
designed to assess the safety risks of AI systems that use chat-tuned language
models. We introduce a principled approach to specifying and constructing the
benchmark, which for v0.5 covers only a single use case (an adult chatting to a
general-purpose assistant in English), and a limited set of personas (i.e.,
typical users, malicious users, and vulnerable users). We created a new
taxonomy of 13 hazard categories, of which 7 have tests in the v0.5 benchmark.
We plan to release version 1.0 of the AI Safety Benchmark by the end of 2024.
The v1.0 benchmark will provide meaningful insights into the safety of AI
systems. However, the v0.5 benchmark should not be used to assess the safety of
AI systems. We have sought to fully document the limitations, flaws, and
challenges of v0.5. This release of v0.5 of the AI Safety Benchmark includes
(1) a principled approach to specifying and constructing the benchmark, which
comprises use cases, types of systems under test (SUTs), language and context,
personas, tests, and test items; (2) a taxonomy of 13 hazard categories with
definitions and subcategories; (3) tests for seven of the hazard categories,
each comprising a unique set of test items, i.e., prompts. There are 43,090
test items in total, which we created with templates; (4) a grading system for
AI systems against the benchmark; (5) an openly available platform, and
downloadable tool, called ModelBench that can be used to evaluate the safety of
AI systems on the benchmark; (6) an example evaluation report which benchmarks
the performance of over a dozen openly available chat-tuned language models;
(7) a test specification for the benchmark.
The AI Safety Benchmark v0.5, developed by MLCommons, assesses the safety risks of chat-tuned language models using a taxonomy of 13 hazard categories and includes 43,090 test items.
chat-tuned language models hazard categories test items ModelBench evaluation report