  In this paper, we explore a process called neural teleportation, a
mathematical consequence of applying quiver representation theory to neural
networks. Neural teleportation "teleports" a network to a new position in the
weight space and preserves its function. This phenomenon comes directly from
the definitions of representation theory applied to neural networks and it
turns out to be a very simple operation that has remarkable properties. We shed
light on surprising and counter-intuitive consequences neural teleportation has
on the loss landscape. In particular, we show that teleportation can be used to
explore loss level curves, that it changes the local loss landscape, sharpens
global minima and boosts back-propagated gradients at any moment during the
learning process. Our results can be reproduced with the code available here:
https://github.com/vitalab/neuralteleportation
