  A well-studied coloring problem is to assign colors to the edges of a graph
$G$ so that, for every pair of vertices, all edges of at least one shortest
path between them receive different colors. The minimum number of colors
necessary in such a coloring is the strong rainbow connection number
($\src(G)$) of the graph. When proving upper bounds on $\src(G)$, it is natural
to prove that a coloring exists where, for \emph{every} shortest path between
every pair of vertices in the graph, all edges of the path receive different
colors. Therefore, we introduce and formally define this more restricted edge
coloring number, which we call \emph{very strong rainbow connection number}
($\vsrc(G)$).
  In this paper, we give upper bounds on $\vsrc(G)$ for several graph classes,
some of which are tight. These immediately imply new upper bounds on $\src(G)$
for these classes, showing that the study of $\vsrc(G)$ enables meaningful
progress on bounding $\src(G)$. Then we study the complexity of the problem to
compute $\vsrc(G)$, particularly for graphs of bounded treewidth, and show this
is an interesting problem in its own right. We prove that $\vsrc(G)$ can be
computed in polynomial time on cactus graphs; in contrast, this question is
still open for $\src(G)$. We also observe that deciding whether $\vsrc(G) = k$
is fixed-parameter tractable in $k$ and the treewidth of $G$. Finally, on
general graphs, we prove that there is no polynomial-time algorithm to decide
whether $\vsrc(G) \leq 3$ nor to approximate $\vsrc(G)$ within a factor
$n^{1-\varepsilon}$, unless P$=$NP.
