ConsumerBench: Benchmarking Generative AI Applications on End-User
  Devices
The recent shift in Generative AI (GenAI) applications from cloud-only
environments to end-user devices introduces new challenges in resource
management, system efficiency, and user experience. This paper presents
ConsumerBench, a comprehensive benchmarking framework designed to evaluate the
system efficiency and response time of GenAI models running on end-user
devices. Unlike existing benchmarks that assume exclusive model access on
dedicated GPUs, ConsumerBench simulates realistic multi-application scenarios
executing concurrently on constrained hardware. Furthermore, ConsumerBench
supports customizable workflows that simulate complex tasks requiring
coordination among multiple applications. ConsumerBench captures both
application-level metrics, including latency and Service Level Objective (SLO)
attainment, and system-level metrics like CPU/GPU utilization and memory
bandwidth. Through extensive experiments, ConsumerBench reveals inefficiencies
in resource sharing, unfair scheduling under greedy allocation, and performance
pitfalls of static model server configurations. The paper also provides
practical insights for model developers and system designers, highlighting the
benefits of custom kernels tailored to consumer-grade GPU architectures and the
value of implementing SLO-aware scheduling strategies.
685a33c50e4ad7e219758612
Yile Gu
685a33c50e4ad7e219758613
Rohan Kadekodi
claimed_verified
2025-06-26T09:33:04.791Z
67b63b314dde7e635d02e61d
/avatars/cf8909aa2642294405f40abfeb6caaec.svg
Rohan Kadekodi
user
rohankadekodi
685a33c50e4ad7e219758614
Hoang Nguyen
685a33c50e4ad7e219758615
Keisuke Kamahori
claimed_verified
2025-06-24T08:07:52.905Z
6304ac1a412a1b9d381ca378
/avatars/f4724eb5afc2a3b0e61e6da7bfa7be27.svg
Keisuke Kamahori
user
kamahori
685a33c50e4ad7e219758616
Yiyu Liu
685a33c50e4ad7e219758617
Baris Kasikci
ConsumerBench: Benchmarking Generative AI Applications on End-User
  Devices
The recent shift in Generative AI (GenAI) applications from cloud-only
environments to end-user devices introduces new challenges in resource
management, system efficiency, and user experience. This paper presents
ConsumerBench, a comprehensive benchmarking framework designed to evaluate the
system efficiency and response time of GenAI models running on end-user
devices. Unlike existing benchmarks that assume exclusive model access on
dedicated GPUs, ConsumerBench simulates realistic multi-application scenarios
executing concurrently on constrained hardware. Furthermore, ConsumerBench
supports customizable workflows that simulate complex tasks requiring
coordination among multiple applications. ConsumerBench captures both
application-level metrics, including latency and Service Level Objective (SLO)
attainment, and system-level metrics like CPU/GPU utilization and memory
bandwidth. Through extensive experiments, ConsumerBench reveals inefficiencies
in resource sharing, unfair scheduling under greedy allocation, and performance
pitfalls of static model server configurations. The paper also provides
practical insights for model developers and system designers, highlighting the
benefits of custom kernels tailored to consumer-grade GPU architectures and the
value of implementing SLO-aware scheduling strategies.
ConsumerBench evaluates GenAI system efficiency and response time on end-user devices through a comprehensive benchmarking framework, emphasizing realistic multi-application scenarios and customizable workflows.
Generative AI ConsumerBench system efficiency response time benchmarking framework multi-application scenarios application-level metrics latency Service Level Objective SLO system-level metrics CPU utilization GPU utilization memory bandwidth greedy allocation static model server configurations custom kernels SLO-aware scheduling strategies