Scalable Generalized Bayesian Online Neural Network Training for Sequential Decision Making
Gerardo Duran-Martin, Leandro Sánchez-Betancourt, Alvaro Cartea, Kevin Murphy
We introduce scalable algorithms for online learning andgeneralized Bayesian inference of neural network parameters,designed for sequential decision making tasks.Our methods combine the strengths of frequentist and Bayesian filtering, which include fast low-rank updates via a block-diagonal approximation of the parameter error covariance, and a well-defined posterior predictive distribution that we use for decision making.More precisely, our main method updates a low-rank error covariance for the hidden layers parameters, and a full-rank errorcovariance  for the final layer parameters.Although this characterizes an improper posterior,we show that the resulting posterior  predictive distribution is well-defined.Our methods update all network parameters online, with no need for replay buffers or offline retraining.We show, empirically, that our methods achieve a competitive tradeoffbetween speed and accuracy on (non-stationary) contextual bandit problemsand Bayesian optimization problems.