The Dutch club produced one of the greatest Champions League comebacks as a 4-1 victory at the Bernabeu dumped the holders out 5-3 on aggregate.
With Real leading 2-1 from the first leg, Ajax travelled to Madrid needing to score at least two goals to reach their first quarter-final in 16 years - they achieved that inside 18 minutes as Hakim Ziyech and David Neres put the visitors in control.
A sumptuous third from Dusan Tadic (62) was allowed to stand after a VAR review and, though Marco Asensio sparked hopes of a late Real rally, they were extinguished 18 minutes from time as Lasse Schone's audacious free-kick from an impossible angle looped over Thibaut Courtois.
The victory saw Ajax win a Champions League knockout tie for the first time in 22 years and sealed a first quarter-final appearance since 2003.
"This was pretty close to perfection," coach Ten Hag told Dutch broadcaster Veronica TV.
"Over the two matches we definitely deserve to progress to the next round.
"I have no influence on the draw, so I'm not going to think about it.
"All the possible opponents are big teams - but so were Real Madrid."