The notion of concept has been studied for centuries, by philosophers, linguists, cognitive scientists, and researchers in artificial intelligence (Margolis & Laurence, 1999). There is a large literature on formal, mathematical models of concepts, including a whole sub-field of AI—Formal Concept Analysis—devoted to this topic (Ganter & Obiedkov, 2016). Recently, researchers in machine learning have begun to investigate how methods from representation learning can be used to induce concepts from raw perceptual data (Higgins, Sonnerat, et al., 2018). The goal of this report is to provide a formal account of concepts which is compatible with this latest work in deep learning.
Since the concepts literature is so large, and covers so many disciplines, we will not attempt to survey the whole field, but rather provide links to the parts of the literature which are especially relevant to our own work. Good places to start for an introduction to concepts include Margolis and Laurence (1999, 2015, 2019), Murphy (2002), and G¨ardenfors (2014). The main technical goal of this report is to show how techniques from representation learning can be married with a lattice-theoretic formulation of conceptual spaces. The mathematics of partial orders and lattices is a standard tool for modelling conceptual spaces (Ch.2, Mitchell (1997), Ganter and Obiedkov (2016)); however, there is no formal work that we are aware of which defines a conceptual lattice on top of a representation that is induced using unsupervised deep learning (Goodfellow et al., 2016). Higgins, Sonnerat, et al. (2018) do this to a degree, but here we provide a much more comprehensive and formal account. G¨ardenfors (2000) offers a geometric account which fits naturally with representation learning, and we will be drawing some inspiration from G¨ardenfors’ work, but with more of a focus on how concepts can be ordered. The advantages of partially-ordered lattice structures are that these provide natural mechanisms for use in concept discovery algorithms, through the meets and joins of the lattice. Finally, although we do not provide much background in terms of the concepts literature, we do attempt some rigour in the mathematical presentation of lattices and partial orders, which will be based heavily on Davey and Priestley (2002).
Overall, our aim is to provide a formal framework for developing practical conceptual discovery and reasoning systems which are grounded in perception and action (Harnad, 1990), thereby overcoming a fundamental deficiency in formal representation systems which are either constructed manually by a knowledge engineer (Ch.8, Russell and Norvig (2003)) or induced automatically from purely text-based resources (Banko et al., 2007). Note that the main aim of this report is a mathematical one; how to realize the framework in practice, and how the framework relates to the vast literature on concepts—across philosophy, psychology, linguistics and AI—are questions left largely for future work.