  In this paper, we develop a novel procedure for low-rank tensor regression,
namely \emph{\underline{I}mportance \underline{S}ketching \underline{L}ow-rank
\underline{E}stimation for \underline{T}ensors} (ISLET). The central idea
behind ISLET is \emph{importance sketching}, i.e., carefully designed sketches
based on both the responses and low-dimensional structure of the parameter of
interest. We show that the proposed method is sharply minimax optimal in terms
of the mean-squared error under low-rank Tucker assumptions and under
randomized Gaussian ensemble design. In addition, if a tensor is low-rank with
group sparsity, our procedure also achieves minimax optimality. Further, we
show through numerical study that ISLET achieves comparable or better
mean-squared error performance to existing state-of-the-art methods while
having substantial storage and run-time advantages including capabilities for
parallel and distributed computing. In particular, our procedure performs
reliable estimation with tensors of dimension $p = O(10^8)$ and is $1$ or $2$
orders of magnitude faster than baseline methods.
