  We build an effective-one-body (EOB) Hamiltonian at third post-Newtonian
(3PN) order in scalar-tensor (ST) and Einstein-scalar-Gauss-Bonnet (ESGB)
theories of gravity. The latter is an extension of general relativity that
predicts scalar hair for black holes. We start from the known two-body
Lagrangian at 3PN order, and use order-reduction methods to construct its
ordinary Hamiltonian counterpart. We then reduce the conservative two-body
dynamics to the (nongeodesic) motion of a test particle in an effective metric
by means of canonical transformations. The resulting EOB Hamiltonian is a
modification of the general relativistic Hamiltonian, and already at 3PN order,
it must account for nonlocal-in-time tail contributions. We include the latter
beyond circular orbits and up to sixth order in the binary's orbital
eccentricity. We finally calculate the orbital frequency at the innermost
stable circular orbit (ISCO) of binary black holes in the shift-symmetric ESGB
model. Our work extends F.L. Juli\'e and N. Deruelle [Phys. Rev. D 95, 124054
(2017)], and it is an essential step toward the accurate modeling of
gravitational waveforms beyond general relativity.
