  General relativity has passed all solar system experiments and neutron star
based tests, such as binary pulsar observations, with flying colors. A more
exotic arena for testing general relativity is in systems that contain one or
more black holes. Black holes are the most compact objects in the universe,
providing probes of the strongest-possible gravitational fields. We are
motivated to study strong-field gravity since many theories give large
deviations from general relativity only at large field strengths, while
recovering the weak-field behavior. In this article, we review how one can
probe general relativity and various alternative theories of gravity by using
electromagnetic waves from a black hole with an accretion disk, and
gravitational waves from black hole binaries. We first review model-independent
ways of testing gravity with electromagnetic/gravitational waves from a black
hole system. We then focus on selected examples of theories that extend general
relativity in rather simple ways. Some important characteristics of general
relativity include (but are not limited to) (i) only tensor gravitational
degrees of freedom, (ii) the graviton is massless, (iii) no quadratic or higher
curvatures in the action, and (iv) the theory is 4 dimensional. Altering a
characteristic leads to a different extension of general relativity: (i)
scalar-tensor theories, (ii) massive gravity theories, (iii) quadratic gravity,
and (iv) theories with large extra dimensions. Within each theory, we describe
black hole solutions, their properties, and current and projected constraints
on each theory using black hole-based tests of gravity. We close this review by
listing some of the open problems in model-independent tests and within each
specific theory.
