  The distance matrix of a dataset $X$ of $n$ points with respect to a distance
function $f$ represents all pairwise distances between points in $X$ induced by
$f$. Due to their wide applicability, distance matrices and related families of
matrices have been the focus of many recent algorithmic works. We continue this
line of research and take a broad view of algorithm design for distance
matrices with the goal of designing fast algorithms, which are specifically
tailored for distance matrices, for fundamental linear algebraic primitives.
Our results include efficient algorithms for computing matrix-vector products
for a wide class of distance matrices, such as the $\ell_1$ metric for which we
get a linear runtime, as well as an $\Omega(n^2)$ lower bound for any algorithm
which computes a matrix-vector product for the $\ell_{\infty}$ case, showing a
separation between the $\ell_1$ and the $\ell_{\infty}$ metrics. Our upper
bound results, in conjunction with recent works on the matrix-vector query
model, have many further downstream applications, including the fastest
algorithm for computing a relative error low-rank approximation for the
distance matrix induced by $\ell_1$ and $\ell_2^2$ functions and the fastest
algorithm for computing an additive error low-rank approximation for the
$\ell_2$ metric, in addition to applications for fast matrix multiplication
among others. We also give algorithms for constructing distance matrices and
show that one can construct an approximate $\ell_2$ distance matrix in time
faster than the bound implied by the Johnson-Lindenstrauss lemma.
