  Exterior contour and interior structure are both vital features for
classifying objects. However, most of the existing methods consider exterior
contour feature and internal structure feature separately, and thus fail to
function when classifying patchy image structures that have similar contours
and flexible structures. To address above limitations, this paper proposes a
novel Multi-Orientation Region Transform (MORT), which can effectively
characterize both contour and structure features simultaneously, for patchy
image structure classification. MORT is performed over multiple orientation
regions at multiple scales to effectively integrate patchy features, and thus
enables a better description of the shape in a coarse-to-fine manner. Moreover,
the proposed MORT can be extended to combine with the deep convolutional neural
network techniques, for further enhancement of classification accuracy. Very
encouraging experimental results on the challenging ultra-fine-grained cultivar
recognition task, insect wing recognition task, and large variation butterfly
recognition task are obtained, which demonstrate the effectiveness and
superiority of the proposed MORT over the state-of-the-art methods in
classifying patchy image structures. Our code and three patchy image structure
datasets are available at: https://github.com/XiaohanYu-GU/MReT2019.
