  To recognize objects of the unseen classes, most existing Zero-Shot
Learning(ZSL) methods first learn a compatible projection function between the
common semantic space and the visual space based on the data of source seen
classes, then directly apply it to the target unseen classes. However, in real
scenarios, the data distribution between the source and target domain might not
match well, thus causing the well-known \textbf{domain shift} problem. Based on
the observation that visual features of test instances can be separated into
different clusters, we propose a new visual structure constraint on class
centers for transductive ZSL, to improve the generality of the projection
function (i.e. alleviate the above domain shift problem). Specifically, three
different strategies (symmetric Chamfer-distance, Bipartite matching distance,
and Wasserstein distance) are adopted to align the projected unseen semantic
centers and visual cluster centers of test instances. We also propose a new
training strategy to handle the real cases where many unrelated images exist in
the test dataset, which is not considered in previous methods. Experiments on
many widely used datasets demonstrate that the proposed visual structure
constraint can bring substantial performance gain consistently and achieve
state-of-the-art results. The source code is available at
\url{https://github.com/raywzy/VSC}.
