SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of
  Large Language Model
The indexing-retrieval-generation paradigm of retrieval-augmented generation
(RAG) has been highly successful in solving knowledge-intensive tasks by
integrating external knowledge into large language models (LLMs). However, the
incorporation of external and unverified knowledge increases the vulnerability
of LLMs because attackers can perform attack tasks by manipulating knowledge.
In this paper, we introduce a benchmark named SafeRAG designed to evaluate the
RAG security. First, we classify attack tasks into silver noise, inter-context
conflict, soft ad, and white Denial-of-Service. Next, we construct RAG security
evaluation dataset (i.e., SafeRAG dataset) primarily manually for each task. We
then utilize the SafeRAG dataset to simulate various attack scenarios that RAG
may encounter. Experiments conducted on 14 representative RAG components
demonstrate that RAG exhibits significant vulnerability to all attack tasks and
even the most apparent attack task can easily bypass existing retrievers,
filters, or advanced LLMs, resulting in the degradation of RAG service quality.
Code is available at: https://github.com/IAAR-Shanghai/SafeRAG.
67a1bfc314cba2eba6da4b2b
Xun Liang
67a1bfc314cba2eba6da4b2c
Simin Niu
admin_assigned
2025-02-04T17:11:40.648Z
66daea8776dbaaa372eabec5
/avatars/1e5fbe4ff06bb6121c7029253b76b79f.svg
siminniu
user
siminniu
67a1bfc314cba2eba6da4b2d
Zhiyu Li
admin_assigned
2025-02-04T17:11:52.391Z
661ef43ff65a7cf84e2291e1
/avatars/cf90dba5934763693c800b3708ce4771.svg
Zhiyu (Drew) Li
user
zhiyuli
67a1bfc314cba2eba6da4b2e
Sensen Zhang
67a1bfc314cba2eba6da4b2f
Hanyu Wang
claimed_verified
2025-02-04T09:39:04.452Z
669e0b93c7cb0568dac6e92e
/avatars/a39ea77d7391f164af8a80f94f85f2ca.svg
hanyu Wang
user
UglyToilet
67a1bfc314cba2eba6da4b30
Feiyu Xiong
67a1bfc314cba2eba6da4b31
Jason Zhaoxin Fan
67a1bfc314cba2eba6da4b32
Bo Tang
67a1bfc314cba2eba6da4b33
Shichao Song
claimed_verified
2025-07-22T07:55:17.367Z
656f339a5273668d5b946b33
https://cdn-avatars.huggingface.co/v1/production/uploads/656f339a5273668d5b946b33/o2nBvQiOKKP5IfDmnpHP2.jpeg
Shichao Song
user
Ki-Seki
67a1bfc314cba2eba6da4b34
Mengwei Wang
67a1bfc314cba2eba6da4b35
Jiawei Yang
claimed_verified
2025-02-21T10:00:17.081Z
669e60ee8580d17cb60f8347
/avatars/37963b833228afe39cc24854c9326670.svg
yang jiawei
user
Dany-0
SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of
  Large Language Model
The indexing-retrieval-generation paradigm of retrieval-augmented generation
(RAG) has been highly successful in solving knowledge-intensive tasks by
integrating external knowledge into large language models (LLMs). However, the
incorporation of external and unverified knowledge increases the vulnerability
of LLMs because attackers can perform attack tasks by manipulating knowledge.
In this paper, we introduce a benchmark named SafeRAG designed to evaluate the
RAG security. First, we classify attack tasks into silver noise, inter-context
conflict, soft ad, and white Denial-of-Service. Next, we construct RAG security
evaluation dataset (i.e., SafeRAG dataset) primarily manually for each task. We
then utilize the SafeRAG dataset to simulate various attack scenarios that RAG
may encounter. Experiments conducted on 14 representative RAG components
demonstrate that RAG exhibits significant vulnerability to all attack tasks and
even the most apparent attack task can easily bypass existing retrievers,
filters, or advanced LLMs, resulting in the degradation of RAG service quality.
Code is available at: https://github.com/IAAR-Shanghai/SafeRAG.
The SafeRAG benchmark evaluates the security of RAG systems by simulating various attack scenarios, revealing significant vulnerabilities in RAG components.
retrieval-augmented generation RAG large language models external knowledge silver noise inter-context conflict soft ad white Denial-of-Service RAG security evaluation dataset retrievers filters