GHOST: Ground-projected Hypotheses from Observed Structure-from-Motion Trajectories

Tomasz Frelek, Rohan Patil, Akshar Tumu, Henrik I. Christensen

We present a scalable self-supervised approach for segmenting feasible vehicle trajectories from monocular images for autonomous driving in complex urban environments. Leveraging large-scale dashcam videos, we treat recorded ego-vehicle motion as implicit supervision and recover camera trajectories via monocular structure-from-motion, projecting them onto the ground plane to generate spatial masks of traversed regions without manual annotation. These automatically generated labels are used to train a deep segmentation network that predicts motion-conditioned path proposals from a single RGB image at run time, without explicit modeling of road or lane markings. Trained on diverse, unconstrained internet data, the model implicitly captures scene layout, lane topology, and intersection structure, and generalizes across varying camera configurations. We evaluate our approach on NuScenes, demonstrating reliable trajectory prediction, and further show transfer to an electric scooter platform through light fine-tuning. Our results indicate that large-scale ego-motion distillation yields structured and generalizable path proposals beyond the demonstrated trajectory, enabling trajectory hypothesis estimation via image segmentation.

• University of California, Los Angeles
• University of California, San Diego
• University of California, San Diego
• University of Wisconsin-Madison
• D-Robotics
• Xspark AI
• Deemos Technologies
• Peking University
• Shanghai Jiao Tong University
• ShanghaiTech University
• Shenzhen University
• The University of Hong Kong
• Tsinghua University
• University of California, San Diego
• SenseTime
• Carnegie Mellon University
• Nanyang Technological University
• University of California, Berkeley
• University of California, San Diego
• University of California, San Diego
• NVIDIA
• Meta Platforms
• University of California, San Diego
• USC Physical Superintelligence Lab