  Coding, which targets compressing and reconstructing data, and intelligence,
often regarded at an abstract computational level as being centered around
model learning and prediction, interweave recently to give birth to a series of
significant progress. The recent trends demonstrate the potential homogeneity
of these two fields, especially when deep-learning models aid these two
categories for better probability modeling. For better understanding and
describing from a unified perspective, inspired by the basic generally
recognized principles in cognitive psychology, we formulate a novel problem of
Coding for Intelligence from the category theory view. Based on the three
axioms: existence of ideal coding, existence of practical coding, and
compactness promoting generalization, we derive a general framework to
understand existing methodologies, namely that, coding captures the intrinsic
relationships of objects as much as possible, while ignoring information
irrelevant to downstream tasks. This framework helps identify the challenges
and essential elements in solving the specific derived Minimal Description
Length (MDL) optimization problem from a broader range, providing opportunities
to build a more intelligent system for handling multiple tasks/applications
with coding ideas/tools. Centering on those elements, we systematically review
recent processes of towards optimizing the MDL problem in more comprehensive
ways from data, model, and task perspectives, and reveal their impacts on the
potential CfI technical routes. After that, we also present new technique paths
to fulfill CfI and provide potential solutions with preliminary experimental
evidence. Last, further directions and remaining issues are discussed as well.
The discussion shows our theory can reveal many phenomena and insights about
large foundation models, which mutually corroborate with recent practices in
feature learning.
