  We examine a class of embeddings based on structured random matrices with
orthogonal rows which can be applied in many machine learning applications
including dimensionality reduction and kernel approximation. For both the
Johnson-Lindenstrauss transform and the angular kernel, we show that we can
select matrices yielding guaranteed improved performance in accuracy and/or
speed compared to earlier methods. We introduce matrices with complex entries
which give significant further accuracy improvement. We provide geometric and
Markov chain-based perspectives to help understand the benefits, and empirical
results which suggest that the approach is helpful in a wider range of
applications.
