Geometric Deep Learning for Embodied AI
Imagine a human waking up, cooking breakfast, changing clothes, and driving to work. If all these tasks could be performed by an embodied agent, what level of understanding would that agent need? As humans, we can perform these everyday tasks so easily, often without much conscious thought (although we might occasionally overboil the coffee). Our decision-making and chains of thought often remain focused on a relatively small part of the world around us.
Geometric deep learning is a broad field that systematically studies groups, graphs, geodesics, and related mathematical structures (Bronstein et al., 2021). Its central idea is to understand the intrinsic structures underlying deep learning models and make better use of them.
I am interested in systematically understanding how to empower embodied AI with these intrinsic properties, for example, by enabling agents to understand hands, objects, and scenes. There is still a large unexplored space worth investigating.
To abstract and communicate human knowledge, people have invented books, pictures, and movies. However, these are essentially projections of human knowledge and perception. What if we could uncover the principles behind these projections and understand the underlying structures they represent? This is a research direction that particularly interests me.
Riemannian geometry has already been applied in many areas, including neural network architectures, motion dynamics, and humanoid robotics. However, I wonder whether we can develop a more unified and general representation of these structures, which might eventually provide a path toward general-purpose robotics.