  Joint pushing and caching is recognized as an efficient remedy to the problem
of spectrum scarcity incurred by tremendous mobile data traffic. In this paper,
by exploiting storage resources at end users and predictability of user demand
processes, we design the optimal joint pushing and caching policy to maximize
bandwidth utilization, which is of fundamental importance to mobile telecom
carriers. In particular, we formulate the stochastic optimization problem as an
infinite horizon average cost Markov Decision Process (MDP), for which there
generally exist only numerical solutions without many insights. By structural
analysis, we show how the optimal policy achieves a balance between the current
transmission cost and the future average transmission cost. In addition, we
show that the optimal average transmission cost decreases with the cache size,
revealing a tradeoff between the cache size and the bandwidth utilization.
Then, due to the fact that obtaining a numerical optimal solution suffers the
curse of dimensionality and implementing it requires a centralized controller
and global system information, we develop a decentralized policy with
polynomial complexity w.r.t. the numbers of users and files as well as cache
size, by a linear approximation of the value function and optimization
relaxation techniques. Next, we propose an online decentralized algorithm to
implement the proposed low-complexity decentralized policy using the technique
of Q-learning, when priori knowledge of user demand processes is not available.
Finally, using numerical results, we demonstrate the advantage of the proposed
solutions over some existing designs. The results in this paper offer useful
guidelines for designing practical cache-enabled content-centric wireless
networks.
