  We present an information-theoretic framework to learn fixed-dimensional
embeddings for tasks in reinforcement learning. We leverage the idea that two
tasks are similar if observing an agent's performance on one task reduces our
uncertainty about its performance on the other. This intuition is captured by
our information-theoretic criterion which uses a diverse agent population as an
approximation for the space of agents to measure similarity between tasks in
sequential decision-making settings. In addition to qualitative assessment, we
empirically demonstrate the effectiveness of our techniques based on task
embeddings by quantitative comparisons against strong baselines on two
application scenarios: predicting an agent's performance on a new task by
observing its performance on a small quiz of tasks, and selecting tasks with
desired characteristics from a given set of options.
