  We introduce a method for transforming low-order tensors into higher-order
tensors and apply it to tensors defined by graphs and hypergraphs. The
transformation proceeds according to a surgery-like procedure that splits
vertices, creates and absorbs virtual edges and inserts new vertices and edges.
We show that tensor surgery is capable of preserving the low rank structure of
an initial tensor decomposition and thus allows to prove nontrivial upper
bounds on tensor rank, border rank and asymptotic rank of the final tensors. We
illustrate our method with a number of examples. Tensor surgery on the triangle
graph, which corresponds to the matrix multiplication tensor, leads to
nontrivial rank upper bounds for all odd cycle graphs, which correspond to the
tensors of iterated matrix multiplication. In the asymptotic setting we obtain
upper bounds in terms of the matrix multiplication exponent $\omega$ and the
rectangular matrix multiplication parameter $\alpha$. These bounds are optimal
if $\omega$ equals two. We also give examples that illustrate that tensor
surgery on general graphs might involve the absorption of virtual hyperedges
and we provide an example of tensor surgery on a hypergraph. Besides its
relevance in algebraic complexity theory, our work has applications in quantum
information theory and communication complexity.
