  We detect millihertz quasi-periodic oscillations (mHz QPOs) using the Rossi
X-ray Time Explorer (RXTE) from the atoll neutron-star (NS) low-mass X-ray
binaries 4U 1608--52 and Aql X--1. From the analysis of all RXTE observations
of 4U 1608--52 and Aql X--1, we find mHz QPOs with a significance level
$>3\sigma$ in 49 and 47 observations, respectively. The QPO frequency is
constrained between $\sim$ 4.2 and 13.4 mHz. These types of mHz QPOs have been
interpreted as being the result of marginally stable nuclear burning of He on
the NS surface. We also report the discovery of a downward frequency drift in
three observations of 4U 1608--52, making it the third source that shows this
behaviour. We only find strong evidence of frequency drift in one occasion in
Aql X--1, probably because the observations were too short to measure a
significant drift. Finally, the mHz QPOs are mainly detected when both sources
are in the soft or intermediate states; the cases that show frequency drift
only occur when the sources are in intermediate states. Our results are
consistent with the phenomenology observed for the NS systems 4U 1636--53 and
EXO 0748--676, suggesting that all four sources can reach the conditions for
marginally stable burning of He on the NS surface. These conditions depend on
the source state in the same manner in all four systems.
