  A precise link is derived between scalar-graviton S-matrix elements and
expectation values of operators in a worldline quantum field theory (WQFT),
both used to describe classical scattering of a pair of black holes. The link
is formally provided by a worldline path integral representation of the
graviton-dressed scalar propagator, which may be inserted into a traditional
definition of the S-matrix in terms of time-ordered correlators. To calculate
expectation values in the WQFT a new set of Feynman rules is introduced which
treats the gravitational field $h_{\mu\nu}(x)$ and position $x_i^\mu(\tau_i)$
of each black hole on equal footing. Using these both the next-order classical
gravitational radiation $\langle h^{\mu\nu}(k)\rangle$ (previously unknown) and
deflection $\Delta p_i^\mu$ from a binary black hole scattering event are
obtained. The latter can also be obtained from the eikonal phase of a $2\to2$
scalar S-matrix, which we show to correspond to the free energy of the WQFT.
