  Photometric Stereo (PS) under outdoor illumination remains a challenging,
ill-posed problem due to insufficient variability in illumination. Months-long
capture sessions are typically used in this setup, with little success on
shorter, single-day time intervals. In this paper, we investigate the solution
of outdoor PS over a single day, under different weather conditions. First, we
investigate the relationship between weather and surface reconstructability in
order to understand when natural lighting allows existing PS algorithms to
work. Our analysis reveals that partially cloudy days improve the conditioning
of the outdoor PS problem while sunny days do not allow the unambiguous
recovery of surface normals from photometric cues alone. We demonstrate that
calibrated PS algorithms can thus be employed to reconstruct Lambertian
surfaces accurately under partially cloudy days. Second, we solve the ambiguity
arising in clear days by combining photometric cues with prior knowledge on
material properties, local surface geometry and the natural variations in
outdoor lighting through a CNN-based, weakly-calibrated PS technique. Given a
sequence of outdoor images captured during a single sunny day, our method
robustly estimates the scene surface normals with unprecedented quality for the
considered scenario. Our approach does not require precise geolocation and
significantly outperforms several state-of-the-art methods on images with real
lighting, showing that our CNN can combine efficiently learned priors and
photometric cues available during a single sunny day.
