Robust Camera Pose Estimation for Mobile Robots with Applications to Non-Line-of-Sight Tracking

Robust camera pose estimation is fundamental to autonomous navigation, robotic perception, and non-line-of-sight (NLOS) tracking. While conventional visual odometry and Simultaneous Localization and Mapping (SLAM) techniques rely heavily on discriminative feature correspondences in texture-rich environments, they often fail in feature-poor conditions, such as low-light, foggy, or textureless scenes. This dissertation proposes novel methodologies to improve pose estimation robustness in these challenging environments by leveraging multi-modal sensor fusion, geometric constraints, and learning-based feature matching.

First, this dissertation presents a Visual-Inertial Odometry (VIO) framework that integrates 3D points, lines, and planes as geometric primitives in an Extended Kalman Filtering (EKF) pipeline. By directly incorporating structural elements into pose estimation, this framework mitigates the limitations of sparse visual features in degraded conditions. The approach is validated using real-world experiments with an instrumented unmanned aerial vehicle (UAV), demonstrating superior pose accuracy compared to traditional feature-based methods.

Second, this dissertation introduces a Stereo Visual Odometry technique with an Attention Graph Neural Network, designed to enhance feature matching under adverse weather and dynamic lighting conditions. By incorporating a deep-learning-based point and line matching mechanism, this approach significantly improves robustness in low-visibility scenarios. Experimental results on synthetic and real-world datasets confirm its effectiveness in reducing trajectory drift.

Finally, these methodologies are extended to dynamic Non-Line-of-Sight (NLOS) tracking, where a mobile robot estimates the trajectory of an object outside its camera’s field of view using scattered light information. The proposed approach includes a novel transformer-based NLOS-Patch Network, which extracts geometric priors from relay surfaces and refines object trajectories using an optimization-based inference pipeline. The tracking framework is evaluated on both synthetic and real-world datasets and validated on in-the-wild scenes with a UAV, showing its potential for applications in surveillance, search-and-rescue, and autonomous exploration.

Together, these contributions advance the field of robust camera pose estimation by enabling reliable localization in visually challenging scenarios. The proposed techniques pave the way for more resilient robotic perception systems capable of operating in real-world conditions where conventional methods often fail.