  A methodology is developed to realized optimal channel input conditional
distributions, which maximize the finite-time horizon directed information, for
channels with memory and feedback, by information lossless randomized
strategies. The methodology is applied to general Time-Varying Multiple Input
Multiple Output (MIMO) Gaussian Linear Channel Models (G-LCMs) with memory,
subject to average transmission cost constraints of quadratic form. The
realizations of optimal distributions by randomized strategies are shown to
exhibit a decomposion into a deterministic part and a random part. The
decomposition reveals the dual role of randomized strategies, to control the
channel output process and to transmit new information over the channels.
Moreover, a separation principle is shown between the computation of the
optimal deterministic part and the random part of the randomized strategies.
The dual role of randomized strategies generalizes the
Linear-Quadratic-Gaussian (LQG) stochastic optimal control theory to directed
information pay-offs. The characterizations of feedback capacity are obtained
from the per unit time limits of finite-time horizon directed information,
without imposing \'a priori assumptions, such as, stability of channel models
or ergodicity of channel input and output processes. For time-invariant MIMO
G-LCMs with memory, it is shown that whether feedback increases capacity, is
directly related to the channel parameters and the transmission cost function,
through the solutions of Riccati matrix equations, and moreover for unstable
channels, feedback capacity is non-zero, provided the power exceeds a critical
level.
