Distributional Semantics Tracing: A Framework for Explaining
  Hallucinations in Large Language Models
Large Language Models (LLMs) are prone to hallucination, the generation of
plausible yet factually incorrect statements. This work investigates the
intrinsic, architectural origins of this failure mode through three primary
contributions.First, to enable the reliable tracing of internal semantic
failures, we propose Distributional Semantics Tracing (DST), a unified
framework that integrates established interpretability techniques to produce a
causal map of a model's reasoning, treating meaning as a function of context
(distributional semantics). Second, we pinpoint the model's layer at which a
hallucination becomes inevitable, identifying a specific commitment
layer where a model's internal representations irreversibly diverge from
factuality. Third, we identify the underlying mechanism for these failures. We
observe a conflict between distinct computational pathways, which we interpret
using the lens of dual-process theory: a fast, heuristic associative
pathway (akin to System 1) and a slow, deliberate contextual pathway
(akin to System 2), leading to predictable failure modes such as
Reasoning Shortcut Hijacks. Our framework's ability to quantify the
coherence of the contextual pathway reveals a strong negative correlation
(rho = -0.863) with hallucination rates, implying that these failures are
predictable consequences of internal semantic weakness. The result is a
mechanistic account of how, when, and why hallucinations occur within the
Transformer architecture.
68e61bc3975ac4c405ef21c3
Gagan Bhatia
claimed_verified
2025-10-08T11:49:03.207Z
60394599033b61166496163b
https://cdn-avatars.huggingface.co/v1/production/uploads/1614366097007-noauth.jpeg
Gagan Bhatia
user
gagan3012
68e61bc3975ac4c405ef21c4
Somayajulu G Sripada
68e61bc3975ac4c405ef21c5
Kevin Allan
68e61bc3975ac4c405ef21c6
Jacobo Azcona
Distributional Semantics Tracing: A Framework for Explaining
  Hallucinations in Large Language Models
Large Language Models (LLMs) are prone to hallucination, the generation of
plausible yet factually incorrect statements. This work investigates the
intrinsic, architectural origins of this failure mode through three primary
contributions.First, to enable the reliable tracing of internal semantic
failures, we propose Distributional Semantics Tracing (DST), a unified
framework that integrates established interpretability techniques to produce a
causal map of a model's reasoning, treating meaning as a function of context
(distributional semantics). Second, we pinpoint the model's layer at which a
hallucination becomes inevitable, identifying a specific commitment
layer where a model's internal representations irreversibly diverge from
factuality. Third, we identify the underlying mechanism for these failures. We
observe a conflict between distinct computational pathways, which we interpret
using the lens of dual-process theory: a fast, heuristic associative
pathway (akin to System 1) and a slow, deliberate contextual pathway
(akin to System 2), leading to predictable failure modes such as
Reasoning Shortcut Hijacks. Our framework's ability to quantify the
coherence of the contextual pathway reveals a strong negative correlation
(rho = -0.863) with hallucination rates, implying that these failures are
predictable consequences of internal semantic weakness. The result is a
mechanistic account of how, when, and why hallucinations occur within the
Transformer architecture.
A framework called Distributional Semantics Tracing identifies the layers and pathways in Transformers where hallucinations occur, revealing a correlation between internal semantic coherence and hallucination rates.
Distributional Semantics Tracing commitment layer associative pathway contextual pathway dual-process theory Reasoning Shortcut Hijacks Transformer architecture