  We introduce an invariant of a hyperbolic knot which is a map $\alpha\mapsto
\boldsymbol{\Phi}_\alpha(h)$ from $\mathbb{Q}/\mathbb{Z}$ to matrices with
entries in $\overline{\mathbb{Q}}[[h]]$ and with rows and columns indexed by
the boundary parabolic ${\rm SL}_2(\mathbb{C})$ representations of the
fundamental group of the knot. These matrix invariants have a rich structure:
(a) their $(\sigma_0,\sigma_1)$ entry, where $\sigma_0$ is the trivial and
$\sigma_1$ the geometric representation, is the power series expansion of the
Kashaev invariant of the knot around the root of unity ${\rm e}^{2\pi{\rm i}
\alpha}$ as an element of the Habiro ring, and the remaining entries belong to
generalized Habiro rings of number fields; (b) the first column is given by the
perturbative power series of Dimofte-Garoufalidis; (c) the columns of
$\boldsymbol{\Phi}$ are fundamental solutions of a linear $q$-difference
equation; (d) the matrix defines an ${\rm SL}_2(\mathbb{Z})$-cocycle $W_\gamma$
in matrix-valued functions on $\mathbb{Q}$ that conjecturally extends to a
smooth function on $\mathbb{R}$ and even to holomorphic functions on suitable
complex cut planes, lifting the factorially divergent series
$\boldsymbol{\Phi}(h)$ to actual functions. The two invariants
$\boldsymbol{\Phi}$ and $W_\gamma$ are related by a refined quantum modularity
conjecture which we illustrate in detail for the three simplest hyperbolic
knots, the $4_1$, $5_2$ and $(-2,3,7)$ pretzel knots. This paper has two
sequels, one giving a different realization of our invariant as a matrix of
convergent $q$-series with integer coefficients and the other studying its
Habiro-like arithmetic properties in more depth.
