Learning Intuitive Physics Through Objects

Although recent years have seen striking advances in artificial intelligence (AI), it is increasingly remarked that most of this progress is limited to narrow domains. Less success has been gained in capturing what are broadly referred to as commonsense aspects of thought. Perhaps most emblematic of common sense is the realm of understanding known as `intuitive physics,' which enables our pragmatic engagement with the physical world. Despite significant effort, current AI systems pale in their understanding of intuitive physics, in comparison to even very young children. In the present work, we address this AI problem, specifically by drawing on the field of developmental psychology. We make two contributions. First, we introduce a deep learning system that learns intuitive physics directly from visual data, using object-level representations inspired by studies of visual cognition in children. Second, we introduce and open-source a machine-learning dataset designed to evaluate conceptual understanding of intuitive physics, importing from developmental psychology the powerful violation-of-expectation (VoE) paradigm. Our AI model shows clear VoE effects when probed on a diverse set of physical concepts, a result which -- consistent with ideas from developmental psychology -- depends critically on the involvement of object-level representations. We consider the implications of these modeling results both for AI and for research on human cognition.

Authors' notes