  In this paper, we develop an elementary and unified treatment, in the spirit
of Rivi\`ere and Struwe (Comm. Pure. Appl. Math. 2008), to explore regularity
of weak solutions of higher order geometric elliptic systems in critical
dimensions without using conservation law. As a result, we obtain an interior
H\"older continuity for solutions of the higher order elliptic system of de
Longueville and Gastel \cite{deLongueville-Gastel-2019} in critical dimensions
$$\Delta^{k}u=\sum_{i=0}^{k-1}\Delta^{i}\left\langle V_{i},du\right\rangle
+\sum_{i=0}^{k-2}\Delta^{i}\delta\left(w_{i}du\right) \quad \text{in }
B^{2k},$$ under critical regularity assumptions on the coefficient functions.
This verifies an expectation of Rivi\`ere, and provides an affirmative answer
to an open question of Struwe in dimension four when $k=2$. The H\"older
continuity is also an improvement of the continuity result of Lamm and
Rivi\`ere and de Longueville and Gastel.
