Constellation: Learning relational abstractions over objects for compositional imagination

Imagination and abstraction are fundamental to intelligence. Abstractions allow reasoning and planning at a level separated from immediate experience, while novel configurations of content is possible with imagination. We introduce Constellation, a network that learns relational abstractions of static visual scenes, and generalise these abstractions over sensory particularities, thus offering a basis for abstract reasoning. We further show that this basis, along with language association, provides a means to imagine sensory content in new ways. This work is a first step in the explicit representation of visual relationships and using them for complex cognitive procedures.

Authors' notes