  Learning representations with diversified information remains as an open
problem. Towards learning diversified representations, a new approach, termed
Information Competing Process (ICP), is proposed in this paper. Aiming to
enrich the information carried by feature representations, ICP separates a
representation into two parts with different mutual information constraints.
The separated parts are forced to accomplish the downstream task independently
in a competitive environment which prevents the two parts from learning what
each other learned for the downstream task. Such competing parts are then
combined synergistically to complete the task. By fusing representation parts
learned competitively under different conditions, ICP facilitates obtaining
diversified representations which contain rich information. Experiments on
image classification and image reconstruction tasks demonstrate the great
potential of ICP to learn discriminative and disentangled representations in
both supervised and self-supervised learning settings.
