π2vec: Policy Representations with SuccessorFeatures

Abstract

This paper introduces π2vec, a method for representing black box policies as comparable feature vectors. Our method combines the strengths of foundation models that serve as generic and powerful state representations and successor features that can model the future occurrence of the states for a policy. π2vec represents the behavior of policies by capturing the statistics of the features from a pretrained model with the help of successor feature framework. We focus on the offline setting where policies and their representations are trained on a fixed dataset of trajectories. Finally, we employ linear regression on π2vec vector representations to predict the performance of held out policies. The synergy of these techniques results in a method for efficient policy evaluation in resource constrained environments.

Authors

Gianluca Scarpellini*, Ksenia Konyushkova, Claudio Fantacci, Tom Paine, Yutian Chen, Misha Denil

Venue

ICLR 2024