  Spatial noncommutativity is similar and can even be related to the
non-Abelian nature of multiple D-branes. But they have so far seemed
independent of each other. Reflecting this decoupling, the algebra of matrix
valued fields on noncommutative space is thought to be the simple tensor
product of constant matrix algebra and the Moyal-Weyl deformation. We propose
scenarios in which the two become intertwined and inseparable. Therefore the
usual separation of ordinary or noncommutative space from the internal discrete
space responsible for non-Abelian symmetry is really the exceptional case of an
unified structure. We call it non-Abelian geometry. This general structure
emerges when multiple D-branes are configured suitably in a flat but varying B
field background, or in the presence of non-Abelian gauge field background. It
can also occur in connection with Taub-NUT geometry. We compute the deformed
product of matrix valued functions using the lattice string quantum mechanical
model developed earlier. The result is a new type of associative algebra
defining non-Abelian geometry. Possible supergravity dual is also discussed.
