In robotics and computer vision, there are many inference problems that come down to fusing the information from many different sensors, or a small number of sensors deployed over time. Examples include mapping the world around an autonomous system and reconstructing a 3D model of the world from images. I will review how these and other problems can be posed in terms of factor graphs, which provide a graphical language in which to develop and collaborate on such problems. Inference is done by converting these graphs to another graph type using a simple elimination game. Online inference gives rise to yet another graphical construct called a Bayes tree, which has been applied widely on a variety of autonomous systems. The talk will emphasize the advantages and intuition that come with these graphical languages, leading to both offline and online sensor fusion algorithms. One of our current research directions is exploring the use of code generation to create high-performance, certifiable code to support autonomous systems in the real world.

Frank Dellaert is a professor in the School of Interactive Computing at the Georgia Institute of Technology, where he has taught since 2001. While on leave from Georgia Tech in 2016-2018, he served as a technical project lead at Facebook Reality Labs. Before that, he completed a stint as chief scientist at Skydio, a startup founded by MIT grads to create a fully autonomous camera drone. Prof. Dellaert’s research interests lie in the overlap of robotics and computer vision, and he is particularly interested in graphical model techniques to solve large-scale problems in mapping and 3D reconstruction. The GTSAM toolbox embodies many of the ideas his research group has worked on in the past few years and is available for download at https://gtsam.org.