  Content caching in wireless networks provides a substantial opportunity to
trade off low cost memory storage with energy consumption, yet finding the
optimal causal policy with low computational complexity remains a challenge.
This paper models the Joint Pushing and Caching (JPC) problem as a Markov
Decision Process (MDP) and provides a solution to determine the optimal
randomized policy. A novel approach to decouple the influence from buffer
occupancy and user requests is proposed to turn the high-dimensional
optimization problem into three low-dimensional ones. Furthermore, a
non-iterative algorithm to solve one of the sub-problems is presented,
exploiting a structural property we found as \textit{generalized monotonicity},
and hence significantly reduces the computational complexity. The result
attains close performance in comparison with theoretical bounds from
non-practical policies, while benefiting from higher time efficiency than the
unadapted MDP solution.
