Low Precision Streaming PCA
Sanjoy Dasgupta, Syamantak Kumar, Shourya Pandey, Purnamrita Sarkar
Low-Precision Streaming PCA addresses the challenge of estimating the top principal component in a streaming setting under limited precision. We establish an information‐theoretic lower bound on the quantization resolution required to attain a desired accuracy in the top eigenvector. We study Oja's algorithm for streaming PCA under linear and nonlinear stochastic quantization. The quantized variants incorporate unbiased stochastic quantization of the weight vector and the data-induced updates. Under mild moment and spectral-gap assumptions on the data distribution, we show that a minibatch version achieves the lower bound up to logarithmic factors under both schemes. This leads to a \textit{dimension-free} quantization error in the nonlinear quantization setting. Empirical evaluations on synthetic streams validate our theoretical findings and demonstrate that our low-precision methods closely track the performance of standard Oja’s algorithm.