  Graph convolutional networks (GCNs) are the most commonly used methods for
skeleton-based action recognition and have achieved remarkable performance.
Generating adjacency matrices with semantically meaningful edges is
particularly important for this task, but extracting such edges is challenging
problem. To solve this, we propose a hierarchically decomposed graph
convolutional network (HD-GCN) architecture with a novel hierarchically
decomposed graph (HD-Graph). The proposed HD-GCN effectively decomposes every
joint node into several sets to extract major structurally adjacent and distant
edges, and uses them to construct an HD-Graph containing those edges in the
same semantic spaces of a human skeleton. In addition, we introduce an
attention-guided hierarchy aggregation (A-HA) module to highlight the dominant
hierarchical edge sets of the HD-Graph. Furthermore, we apply a new six-way
ensemble method, which uses only joint and bone stream without any motion
stream. The proposed model is evaluated and achieves state-of-the-art
performance on four large, popular datasets. Finally, we demonstrate the
effectiveness of our model with various comparative experiments.
