What does gradient variance tell us about the optimality of vector fields?
Teodora Reu, Sixtine Dromigny, Michael Bronstein, Francisco Vargas
We investigate the underlying cause of common failure modes behind the state-of-the-art fixed-point generative models, Rectified Flows and Schr\"odinger Bridge Matching. First, we introduce a gradient‐variance diagnostic that directly measures how learned vector fields induce transport errors after integration. Contrary to common belief, we show that low‐variance (i.e., well‐aligned) interpolants can still yield high transport error. We also demonstrate that standard neural network architectures (e.g., MLPs or CNNs) are fundamentally limited, failing to exactly represent even Gaussian-to-Gaussian transport. While prior work on Rectified Flows notes that strict convergence to optimal couplings is not guaranteed, it overlooks a key limitation: with noiseless interpolants, further iterations do not improve transport. If the pairings are already straight, the process stagnates; if they are deterministic but not straight, the vector field simply memorizes these pairings without achieving true optimal transport. We prove the existence of such memorizing vector fields.