  We derive explicit asymptotic formulae for the joint moments of the $n_1$-th
and $n_2$-th derivatives of the characteristic polynomials of CUE random
matrices for any non-negative integers $n_1, n_2$. These formulae are expressed
in terms of determinants whose entries involve modified Bessel functions of the
first kind. We also express them in terms of two types of combinatorial sums.
Similar results are obtained for the analogue of Hardy's $Z$-function. We use
these formulae to formulate general conjectures for the joint moments of the
$n_1$-th and $n_2$-th derivatives of the Riemann zeta-function and of Hardy's
$Z$-function. Our conjectures are supported by comparison with results obtained
previously in the number theory literature.
