Reasoning Introduces New Poisoning Attacks Yet Makes Them More
  Complicated
Early research into data poisoning attacks against Large Language Models
(LLMs) demonstrated the ease with which backdoors could be injected. More
recent LLMs add step-by-step reasoning, expanding the attack surface to include
the intermediate chain-of-thought (CoT) and its inherent trait of decomposing
problems into subproblems. Using these vectors for more stealthy poisoning, we
introduce ``decomposed reasoning poison'', in which the attacker modifies only
the reasoning path, leaving prompts and final answers clean, and splits the
trigger across multiple, individually harmless components.
  Fascinatingly, while it remains possible to inject these decomposed poisons,
reliably activating them to change final answers (rather than just the CoT) is
surprisingly difficult. This difficulty arises because the models can often
recover from backdoors that are activated within their thought processes.
Ultimately, it appears that an emergent form of backdoor robustness is
originating from the reasoning capabilities of these advanced LLMs, as well as
from the architectural separation between reasoning and final answer
generation.
68c3e309fc1747b912403b52
Hanna Foerster
68c3e309fc1747b912403b53
Ilia Shumailov
68c3e309fc1747b912403b54
Yiren Zhao
68c3e309fc1747b912403b55
Harsh Chaudhari
68c3e309fc1747b912403b56
Jamie Hayes
68c3e309fc1747b912403b57
Robert Mullins
68c3e309fc1747b912403b58
Yarin Gal
Reasoning Introduces New Poisoning Attacks Yet Makes Them More
  Complicated
Early research into data poisoning attacks against Large Language Models
(LLMs) demonstrated the ease with which backdoors could be injected. More
recent LLMs add step-by-step reasoning, expanding the attack surface to include
the intermediate chain-of-thought (CoT) and its inherent trait of decomposing
problems into subproblems. Using these vectors for more stealthy poisoning, we
introduce ``decomposed reasoning poison'', in which the attacker modifies only
the reasoning path, leaving prompts and final answers clean, and splits the
trigger across multiple, individually harmless components.
  Fascinatingly, while it remains possible to inject these decomposed poisons,
reliably activating them to change final answers (rather than just the CoT) is
surprisingly difficult. This difficulty arises because the models can often
recover from backdoors that are activated within their thought processes.
Ultimately, it appears that an emergent form of backdoor robustness is
originating from the reasoning capabilities of these advanced LLMs, as well as
from the architectural separation between reasoning and final answer
generation.
Data poisoning attacks on Large Language Models can target the reasoning process by decomposing triggers, but the models' reasoning capabilities and architectural design provide a form of backdoor robustness.
Large Language Models data poisoning attacks backdoors step-by-step reasoning chain-of-thought decomposed reasoning poison reasoning path final answers emergent backdoor robustness