Do RAG Systems Suffer From Positional Bias?
Retrieval Augmented Generation enhances LLM accuracy by adding passages
retrieved from an external corpus to the LLM prompt. This paper investigates
how positional bias - the tendency of LLMs to weight information differently
based on its position in the prompt - affects not only the LLM's capability to
capitalize on relevant passages, but also its susceptibility to distracting
passages. Through extensive experiments on three benchmarks, we show how
state-of-the-art retrieval pipelines, while attempting to retrieve relevant
passages, systematically bring highly distracting ones to the top ranks, with
over 60% of queries containing at least one highly distracting passage among
the top-10 retrieved passages. As a result, the impact of the LLM positional
bias, which in controlled settings is often reported as very prominent by
related works, is actually marginal in real scenarios since both relevant and
distracting passages are, in turn, penalized. Indeed, our findings reveal that
sophisticated strategies that attempt to rearrange the passages based on LLM
positional preferences do not perform better than random shuffling.
6836c1815b96c1925376ecce
Florin Cuconasu
claimed_verified
2025-05-28T08:55:27.298Z
62bfff6788fdef8ecde8c45b
/avatars/637da88b8dce01bdc2a4c1319cc882e3.svg
Florin Cuconasu
user
florin-hf
6836c1815b96c1925376eccf
Simone Filice
admin_assigned
2025-05-28T12:25:37.486Z
652028f9974423bd3ea33ae2
/avatars/50b749f9a95a3b0fb7a2bfcf17c34295.svg
Simone Filice
user
filice
6836c1815b96c1925376ecd0
Guy Horowitz
admin_assigned
2025-05-28T12:25:43.599Z
63062839df993a789e649e00
/avatars/48a57ab7458a0c525da76b289a36a9f6.svg
Guy Horowitz
user
guyhoro95
6836c1815b96c1925376ecd1
Yoelle Maarek
admin_assigned
2025-05-28T12:25:49.349Z
677fa709d34d476c057cd4cb
/avatars/54fe4086b5f1e7af7f573d971f01a2c2.svg
Yoelle Maarek
user
yoelle
6836c1815b96c1925376ecd2
Fabrizio Silvestri
admin_assigned
2025-05-28T12:25:54.460Z
6075d282134c000d1ae10d54
/avatars/8691622e1f6325bc378b2268c119b9a0.svg
Fabrizio Silvestri
user
fabrizio-silvestri
Do RAG Systems Suffer From Positional Bias?
Retrieval Augmented Generation enhances LLM accuracy by adding passages
retrieved from an external corpus to the LLM prompt. This paper investigates
how positional bias - the tendency of LLMs to weight information differently
based on its position in the prompt - affects not only the LLM's capability to
capitalize on relevant passages, but also its susceptibility to distracting
passages. Through extensive experiments on three benchmarks, we show how
state-of-the-art retrieval pipelines, while attempting to retrieve relevant
passages, systematically bring highly distracting ones to the top ranks, with
over 60% of queries containing at least one highly distracting passage among
the top-10 retrieved passages. As a result, the impact of the LLM positional
bias, which in controlled settings is often reported as very prominent by
related works, is actually marginal in real scenarios since both relevant and
distracting passages are, in turn, penalized. Indeed, our findings reveal that
sophisticated strategies that attempt to rearrange the passages based on LLM
positional preferences do not perform better than random shuffling.
Retrieval Augmented Generation suffers from high distraction from top-ranked passages, rendering LLM positional bias less impactful than previously thought.
Retrieval Augmented Generation LLM positional bias relevant passages distracting passages retrieval pipelines