  Probing the evolution of physical systems at the femto- or attosecond
timescale with light requires accurate characterization of ultrashort optical
pulses. The time profiles of such pulses are usually retrieved by methods
utilizing optical nonlinearities, which require significant signal powers and
operate in a limited spectral
range\cite{Trebino_Review_of_Scientific_Instruments97,Walmsley_Review_09}. We
present a linear self-referencing characterization technique based on time
domain localization of the pulse spectral components, operated in the
single-photon regime. Accurate timing of the spectral slices is achieved with
standard single photon detectors, rendering the technique applicable in any
spectral range from near infrared to deep UV. Using detection electronics with
about $70$ ps response, we retrieve the temporal profile of a picowatt pulse
train with $\sim10$ fs resolution, setting a new scale of sensitivity in
ultrashort pulse characterization.
