  We study the dynamics of gauge theory and general relativity using fields of
local observers, thus maintaining local Lorentz symmetry despite a space/time
splitting of fields. We start with Yang--Mills theory, where observer fields
are defined as normalized future-timelike vector fields. We then define
observers without a fixed geometry, and find these play two related roles in
general relativity: splitting fields into spatial and temporal parts, and
"breaking" gauge symmetry, effectively reducing the spacetime SO(n,1)
connection to an observer-dependent spatial SO(n) connection. In both gauge
theory and gravity, the observer field reduces the action to canonical form,
without using gauge fixing. In the 4d gravity case, the result is a manifestly
Lorentz covariant counterpart of the Ashtekar-Barbero formulation. We also
explain how this leads geometrically to a picture of general relativity in
terms of "observer space" rather than spacetime---a setting where both
spacetime symmetry and the dynamical description are simultaneously available.
