  We determine higher-derivative terms in the open superstring effective action
with U(N) gauge group up to and including order alpha'^4 as can be extracted
from 4 boson, 2 boson - 2 fermion and 4 fermion string scattering amplitudes.
This yields corrections to the non-abelian Born-Infeld action involving higher
derivatives as is relevant for studying D-branes beyond the slowly varying
field approximation. While at order alpha'^2 the action has recently been shown
to be a symmetrised trace, this no longer is true at order alpha'^3 or
alpha'^4. We argue that these terms including higher derivatives are as
important in a low-energy expansion as e.g. the much-discussed alpha'^4 F^6
terms. In particular a computation of the fluctuation spectra at order alpha'^4
has to take into account these non-symmetrised trace higher-derivative terms
computed here.
