  In this work we propose a generalization of the Hadamard product between two
matrices to a tensor-valued, multi-linear product between k matrices for any $k
\ge 1$. A multi-linear dual operator to the generalized Hadamard product is
presented. It is a natural generalization of the Diag x operator, that maps a
vector $x \in \R^n$ into the diagonal matrix with x on its main diagonal.
Defining an action of the $n \times n$ orthogonal matrices on the space of
k-dimensional tensors, we investigate its interactions with the generalized
Hadamard product and its dual. The research is motivated, as illustrated
throughout the paper, by the apparent suitability of this language to describe
the higher-order derivatives of spectral functions and the tools needed to
compute them. For more on the later we refer the reader to [14] and [15], where
we use the language and properties developed here to study the higher-order
derivatives of spectral functions.
