PUMA: Secure Inference of LLaMA-7B in Five Minutes
With ChatGPT as a representative, tons of companies have began to provide
services based on large Transformers models. However, using such a service
inevitably leak users' prompts to the model provider. Previous studies have
studied secure inference for Transformer models using secure multiparty
computation (MPC), where model parameters and clients' prompts are kept secret.
Despite this, these frameworks are still limited in terms of model performance,
efficiency, and deployment. To address these limitations, we propose framework
PUMA to enable fast and secure Transformer model inference. Our framework
designs high quality approximations for expensive functions, such as GeLU and
Softmax, which significantly reduce the cost of secure inference while
preserving the model performance. Additionally, we design secure Embedding and
LayerNorm procedures that faithfully implement the desired functionality
without undermining the Transformer architecture. PUMA is about 2x faster than
the state-of-the-art MPC framework MPCFORMER(ICLR 2023) and has similar
accuracy as plaintext models without fine-tuning (which the previous works
failed to achieve).
  One more thing, PUMA can evaluate LLaMA-7B in around 5 minutes to generate 1
token. To our best knowledge, this is the first time that a model with such a
parameter size is able to be evaluated under MPC. PUMA has been open-sourced in
the Github repository of SecretFlow-SPU.
64bf315b4561d0aca2d5112d
Ye Dong
claimed_verified
2024-02-15T08:29:17.695Z
64ded395931a8e1c2c7bce03
/avatars/f70c6644b96741650fea587e2654b9c2.svg
Ye Dong
user
Ye000
64bf315b4561d0aca2d5112e
Wen-jie Lu
64bf315b4561d0aca2d5112f
Yancheng Zheng
64bf315b4561d0aca2d51130
Haoqi Wu
64bf315b4561d0aca2d51131
Derun Zhao
64bf315b4561d0aca2d51132
Jin Tan
claimed_verified
2023-08-02T20:30:31.905Z
64843d82dcece54afd738218
/avatars/c0b2b2eeafe8e8b58ebea4e5db03c454.svg
Jin Tan
user
jint001
64bf315b4561d0aca2d51133
Zhicong Huang
64bf315b4561d0aca2d51134
Cheng Hong
claimed_verified
2023-07-25T10:41:41.835Z
64bf6aa501f1983a86b7f990
/avatars/1d5514a37571aae547d071d4655e5957.svg
Cheng Hong
user
vincehong
64bf315b4561d0aca2d51135
Tao Wei
64bf315b4561d0aca2d51136
Wenguang Cheng
PUMA: Secure Inference of LLaMA-7B in Five Minutes
With ChatGPT as a representative, tons of companies have began to provide
services based on large Transformers models. However, using such a service
inevitably leak users' prompts to the model provider. Previous studies have
studied secure inference for Transformer models using secure multiparty
computation (MPC), where model parameters and clients' prompts are kept secret.
Despite this, these frameworks are still limited in terms of model performance,
efficiency, and deployment. To address these limitations, we propose framework
PUMA to enable fast and secure Transformer model inference. Our framework
designs high quality approximations for expensive functions, such as GeLU and
Softmax, which significantly reduce the cost of secure inference while
preserving the model performance. Additionally, we design secure Embedding and
LayerNorm procedures that faithfully implement the desired functionality
without undermining the Transformer architecture. PUMA is about 2x faster than
the state-of-the-art MPC framework MPCFORMER(ICLR 2023) and has similar
accuracy as plaintext models without fine-tuning (which the previous works
failed to achieve).
  One more thing, PUMA can evaluate LLaMA-7B in around 5 minutes to generate 1
token. To our best knowledge, this is the first time that a model with such a
parameter size is able to be evaluated under MPC. PUMA has been open-sourced in
the Github repository of SecretFlow-SPU.
PUMA framework accelerates secure Transformer model inference using high-quality approximations and secure procedures, achieving faster performance and maintaining accuracy compared to existing methods.
MPC Transformer models secure inference GeLU Softmax Embedding LayerNorm PUMA MPCFORMER LLaMA-7B