  360$^o$ video is an essential component of VR/AR/MR systems that provides
immersive experience to the users. However, 360$^o$ video is associated with
high bandwidth requirements. The required bandwidth can be reduced by
exploiting the fact that users are interested in viewing only a part of the
video scene and that users request viewports that overlap with each other.
Motivated by the findings of recent works where the benefits of caching video
tiles at edge servers instead of caching entire 360$^o$ videos were shown, in
this paper, we introduce the concept of virtual viewports that have the same
number of tiles with the original viewports. The tiles forming these viewports
are the most popular ones for each video and are determined by the users'
requests. Then, we propose a proactive caching scheme that assumes unknown
videos' and viewports' popularity. Our scheme determines which videos to cache
as well as which is the optimal virtual viewport per video. Virtual viewports
permit to lower the dimensionality of the cache optimization problem. To solve
the problem, we first formulate the content placement of 360$^o$ videos in edge
cache networks as a Markov Decision Process (MDP), and then we determine the
optimal caching placement using the Deep Q-Network (DQN) algorithm. The
proposed solution aims at maximizing the overall quality of the 360$^o$ videos
delivered to the end-users by caching the most popular 360$^o$ videos at base
quality along with a virtual viewport in high quality. We extensively evaluate
the performance of the proposed system and compare it with that of known
systems such as LFU, LRU, FIFO, over both synthetic and real 360$^o$ video
traces. The results reveal the large benefits coming from proactive caching of
virtual viewports instead of the original ones in terms of the overall quality
of the rendered viewports, the cache hit ratio, and the servicing cost.
