  Transfer learning (TL) is a promising way to improve the sample efficiency of
reinforcement learning. However, how to efficiently transfer knowledge across
tasks with different state-action spaces is investigated at an early stage.
Most previous studies only addressed the inconsistency across different state
spaces by learning a common feature space, without considering that similar
actions in different action spaces of related tasks share similar semantics. In
this paper, we propose a method to learning action embeddings by leveraging
this idea, and a framework that learns both state embeddings and action
embeddings to transfer policy across tasks with different state and action
spaces. Our experimental results on various tasks show that the proposed method
can not only learn informative action embeddings but accelerate policy
learning.
