  We prove that constant scalar curvature K\"ahler metric "adjacent" to a fixed
K\"ahler class is unique up to isomorphism. This extends the uniqueness theorem
of Donaldson and Chen-Tian, and formally fits into the infinite dimensional
G.I.T picture described by Donaldson. We prove that the Calabi flow near a cscK
metric exists globally and converges uniformly to a cscK metric in a polynomial
rate. Viewed in a K\"ahler class, the Calabi flow is also shown to be
asymptotic to a smooth geodesic ray at infinity. This latter fact is also
interesting in the finite dimensional analogue, where we show that the downward
gradient flow of the Kempf-Ness function in a semi-stable orbit is asymptotic
to the direction of optimal degeneration.
