Exploiting Novel GPT-4 APIs
Language model attacks typically assume one of two extreme threat models:
full white-box access to model weights, or black-box access limited to a text
generation API. However, real-world APIs are often more flexible than just text
generation: these APIs expose "gray-box" access leading to new threat vectors.
To explore this, we red-team three new functionalities exposed in the GPT-4
APIs: fine-tuning, function calling and knowledge retrieval. We find that
fine-tuning a model on as few as 15 harmful examples or 100 benign examples can
remove core safeguards from GPT-4, enabling a range of harmful outputs.
Furthermore, we find that GPT-4 Assistants readily divulge the function call
schema and can be made to execute arbitrary function calls. Finally, we find
that knowledge retrieval can be hijacked by injecting instructions into
retrieval documents. These vulnerabilities highlight that any additions to the
functionality exposed by an API can create new vulnerabilities.
658a40c5d1331d552b250e8a
Kellin Pelrine
claimed_verified
2024-08-29T15:22:36.818Z
6596269ef8f4854a7781dba1
/avatars/680856920131c63c0ad6a67ed7db1db8.svg
Kellin Pelrine
user
kellinpelrine
658a40c5d1331d552b250e8b
Mohammad Taufeeque
658a40c5d1331d552b250e8c
Michał Zając
extracted_confirmed
2024-01-25T19:04:47.257Z
65b2b0884f699ea319d2570f
/avatars/42620ac19e28f1a6a11a9ed270a0e955.svg
Michał Zając
user
michalzajac
658a40c5d1331d552b250e8d
Euan McLean
658a40c5d1331d552b250e8e
Adam Gleave
claimed_verified
2024-01-12T08:06:45.599Z
6294db83b9d93a419906c9bb
/avatars/ccd716242a7a1b0024bcf92cb6134a8f.svg
Adam Gleave
user
AdamGleave
Exploiting Novel GPT-4 APIs
Language model attacks typically assume one of two extreme threat models:
full white-box access to model weights, or black-box access limited to a text
generation API. However, real-world APIs are often more flexible than just text
generation: these APIs expose "gray-box" access leading to new threat vectors.
To explore this, we red-team three new functionalities exposed in the GPT-4
APIs: fine-tuning, function calling and knowledge retrieval. We find that
fine-tuning a model on as few as 15 harmful examples or 100 benign examples can
remove core safeguards from GPT-4, enabling a range of harmful outputs.
Furthermore, we find that GPT-4 Assistants readily divulge the function call
schema and can be made to execute arbitrary function calls. Finally, we find
that knowledge retrieval can be hijacked by injecting instructions into
retrieval documents. These vulnerabilities highlight that any additions to the
functionality exposed by an API can create new vulnerabilities.
Exploration of gray-box threats on GPT-4 API functionalities, including fine-tuning, function calling, and knowledge retrieval, reveals new vulnerabilities that can enable harmful outputs and unauthorized actions.
fine-tuning function calling knowledge retrieval