Seeing through Uncertainty: Robust Task-Oriented Optimization in Visual Navigation
Yiyuan Pan, Yunzhe XU, Zhe Liu, Hesheng Wang
Visual navigation is essential for autonomous agents, yet data scarcity poses a fundamental bottleneck that severely impairs the generalization of learned policies to unseen scenarios. Existing visual navigation agents depend on complex predictive or reasoning modules that become counterproductive and data-hungry in such limited-data regimes. This paper introduces NeuRO, a novel hybrid framework that pioneers the integration of networks with downstream task-based optimization to tackle this critical problem. NeuRO addresses core difficulties in this integration: (i) it empowers the network to translate inherently unreliable visual predictions under data scarcity into calibrated convex uncertainty sets, which then directly inform and constrain the downstream optimization problem, using Partially Input Convex Neural Networks (PICNNs) via a conformal method; and (ii) it reformulates the partially observable task as a generalizable robust optimization problem to effectively leverage these uncertainty-aware representations to derive robust policies. Extensive experiments on both unordered and sequential MultiON tasks demonstrate that NeuRO establishes state-of-the-art performance, particularly in generalization to unseen environments. Our work thus presents a significant advancement for developing robust, generalizable autonomous agents.