  Low-depth random circuit codes possess many desirable properties for quantum
error correction but have so far only been analyzed in the code capacity
setting where it is assumed that encoding gates and syndrome measurements are
noiseless. In this work, we design a fault-tolerant distillation protocol for
preparing encoded states of one-dimensional random circuit codes even when all
gates and measurements are subject to noise. This is sufficient for
fault-tolerant quantum memory since these encoded states can then be used as
ancillas for Steane error correction. We show through numerical simulations
that our protocol can correct erasure errors up to an error rate of $2\%$. In
addition, we also extend results in the code capacity setting by developing a
maximum likelihood decoder for depolarizing noise similar to work by Darmawan
et al. As in their work, we formulate the decoding problem as a tensor network
contraction and show how to contract the network efficiently by exploiting the
low-depth structure. Replacing the tensor network with a so-called ''tropical''
tensor network, we also show how to perform minimum weight decoding. With these
decoders, we are able to numerically estimate the depolarizing error threshold
of finite-rate random circuit codes and show that this threshold closely
matches the hashing bound even when the decoding is sub-optimal.
