  Pulsar Timing Array (PTA) observations have recently gathered substantial
evidence for the existence of a gravitational wave background in the nHz
frequency band. Searching for anisotropies in this signal is key to determining
its origin, and in particular to distinguish possible astrophysical from
cosmological sources. In this work, we assess the sensitivity of current and
future pulsar timing arrays to such anisotropies using the full covariance
matrix of pulsar timing delays. While current day pulsar timing arrays can only
set mildly informative constraints on the dipole and quadrupole, we show that
percent level accuracy for several low multipoles can be achieved in the near
future. Moreover, we demonstrate that anisotropies in the gravitational wave
background and the Hellings-Downs angular correlation, indicating the presence
of GWs, are approximately uncorrelated, and can hence be reconstructed
independently. These results can be reproduced with
\href{https://github.com/Mauropieroni/fastPTA}{\texttt{fastPTA}}, a publicly
available Python code to forecast the constraining power of PTA configurations.
