Do Robots Know What They See – or See What They Know?

Our visual sense enables us to perform a large range of complex tasks by interacting with objects and the environment. Similary, robots need strong visual capabilities to detect, reconstruct, and interact with objects, and classical machine learning methods use annotated training data to infer information for known object classes. However, these models often fail to extrapolate on new objects and environments. This talk introduces a new approach for robotic perception. By learning object-centric 3D representations, the system builds a deeper understanding of object structure, allowing it to generalize its knowledge to previously unseen instances. This effectively reverses the traditional paradigm from "seeing is knowing" to "knowing is seeing". As a result, robots can quickly detect, reconstruct, and interact with novel objects, only from a few given views. During the talk, this method will be demonstrated live on a new object, showing its potential for fast 3D reconstruction and robotic interaction.

Tracks

Speaker

Rudolph Triebel

Head of the Department Perception an Cognition (DLR) and Professor for Intelligent Robot Perception at KIT, German Aerospace Center