  The recently proposed Bayesian Flow Networks~(BFNs) show great potential in
modeling parameter spaces, offering a unified strategy for handling continuous,
discretized, and discrete data. However, BFNs cannot learn high-level semantic
representation from the parameter space since {common encoders, which encode
data into one static representation, cannot capture semantic changes in
parameters.} This motivates a new direction: learning semantic representations
hidden in the parameter spaces to characterize mixed-typed noisy data.
{Accordingly, we propose a representation learning framework named ParamReL,
which operates in the parameter space to obtain parameter-wise latent semantics
that exhibit progressive structures. Specifically, ParamReL proposes a
\emph{self-}encoder to learn latent semantics directly from parameters, rather
than from observations. The encoder is then integrated into BFNs, enabling
representation learning with various formats of observations. Mutual
information terms further promote the disentanglement of latent semantics and
capture meaningful semantics simultaneously.} We illustrate {conditional
generation and reconstruction} in ParamReL via expanding BFNs, and extensive
{quantitative} experimental results demonstrate the {superior effectiveness} of
ParamReL in learning parameter representation.
