What do robots need to know?: World representations for autonomous systems

Abstract:

For a robot to act autonomously in the world, it must first understand it. But what does that understanding look like, and how should it be structured? This talk explores the question of world representation from the perspective of autonomous systems operating in complex, real-world environments, asking not just what robots can perceive, but what they need to know to act effectively.

We trace a research thread that begins with dynamic scene understanding through moving object detection and simultaneous multi-object tracking and classification, where probabilistic methods enable robots to reason jointly about the identity and motion of multiple agents. Alongside this, we address perception in autonomous driving, from stereo- and LiDAR-based motion detection to dense outdoor perception, semantic occupancy grids, and driver behaviour modelling, illustrating how rich world representations underpin safe operation in unstructured, populated environments. Moving from perception to action, we discuss view planning strategies that actively direct a robot's sensors toward task-relevant information, and learning-informed motion planning approaches that embed prior experience into the planning process to handle constrained workspaces and goal uncertainty. Each of these contributions highlights how the demands of the task shape the representation a robot must maintain.

Throughout, a consistent theme emerges: effective autonomy requires layered world representations that integrate geometry, semantics, dynamics, and structure. We reflect on the design principles that make such representations tractable and useful, and consider how they connect to the broader ambitions of embodied AI, where perception, memory, and action are tightly coupled in agents that must think, learn, and act in the physical world.

Bio:

Dr. Victor Romero Cano is a Senior Lecturer in the School of Computational and Mathematical Sciences at Cardiff University, where he is a member of the Human-Centred Computing Research Unit, co-leads the Cardiff University Computational Robotics (CUCR) research team, and manages the Human Centred Computing and Robotics (HCCR) Laboratory. His research focuses on robotics, embodied AI, machine learning, robot perception, and autonomous systems, with applications ranging from autonomous driving to human-centred robotics. He received his PhD from the University of Sydney, following an MSc from the University of São Paulo and a degree in Mechatronics Engineering from Universidad Autónoma de Occidente, Colombia. His career spans both academia and industry across Australia, France, Colombia, Croatia, and the United Kingdom, including positions at the Rio Tinto Centre for Mine Automation, Inria Grenoble, Universidad Autónoma de Occidente, and Rimac Technology. Dr. Romero Cano is a Senior Member of IEEE and the IEEE Robotics and Automation Society. He also serves as an Associate Editor for IEEE Robotics and Automation Letters (RA-L).