  The first stars are believed to have formed a few hundred million years after
the big bang in so-called dark matter minihalos with masses ~10^6 M_sun. Their
radiation lit up the Universe for the first time, and the supernova explosions
that ended their brief lives enriched the intergalactic medium with the first
heavy elements. Influenced by their feedback, the first galaxies assembled in
halos with masses ~10^8 M_sun, and hosted the first metal-enriched stellar
populations. In this review, I summarize the theoretical progress made in the
field of high-redshift star and galaxy formation since the turn of the
millennium, with an emphasis on numerical simulations. These have become the
method of choice to understand the multi-scale, multi-physics problem posed by
structure formation in the early Universe. In the first part of the review, I
focus on the formation of the first stars in minihalos - in particular the
post-collapse phase, where disk fragmentation, protostellar evolution, and
radiative feedback become important. I also discuss the influence of additional
physical processes, such as magnetic fields and streaming velocities. In the
second part of the review, I summarize the various feedback mechanisms exerted
by the first stars, followed by a discussion of the first galaxies and the
various physical processes that operate in them.
