Humans are learning agents that acquire social group representations from experience. Here we discuss how to construct artificial agents capable of this feat. One approach, based on deep reinforcement learning, allows the necessary representations to self-organize. This minimizes the need for hand-engineering, improving robustness and scalability. It also enables ``virtual neuroscience'' research on the learned representations.