  The higher derivative corrections in double field theory are revisited to
first order in $\alpha'$. In first order perturbation theory around flat space,
the gauge algebra is $\alpha'$ corrected, governed by two parameters $a, b$.
One parameter choice corresponds to bosonic string theory, another to heterotic
string theory. These results are generalized to second order in perturbation
theory by a Noether procedure. Using consistency conditions derived from the
Jacobi identity, it is shown that for general $a, b$ there is no generalized
metric formulation. A manifestly background independent formulation is instead
given by a frame formalism, with $\alpha'$-deformed frame transformations, as
proposed by Marques and Nunez. Their construction is slightly generalized to an
$\alpha'$-deformed $GL(D)\times GL(D)$ frame formalism. The perturbation theory
around flat space is matched to second order with the results obtained by the
Noether procedure. We also obtain a formulation based on the `non-symmetric'
metric ${\cal E}$, the sum of metric and $b$-field. The transformations of
${\cal E}$ under $O(D,D)$ receive non-trivial $\alpha'$ corrections, unless
$a=-b$, in which case there is a generalized metric formulation, in agreement
with previous results in the literature.
