  Deciphering historical substitution ciphers is a challenging problem. Example
problems that have been previously studied include detecting cipher type,
detecting plaintext language, and acquiring the substitution key for segmented
ciphers. However, attacking unsegmented, space-free ciphers is still a
challenging task. Segmentation (i.e. finding substitution units) is the first
step towards cracking those ciphers. In this work, we propose the first
automatic methods to segment those ciphers using Byte Pair Encoding (BPE) and
unigram language models. Our methods achieve an average segmentation error of
2\% on 100 randomly-generated monoalphabetic ciphers and 27\% on 3 real
homophonic ciphers. We also propose a method for solving non-deterministic
ciphers with existing keys using a lattice and a pretrained language model. Our
method leads to the full solution of the IA cipher; a real historical cipher
that has not been fully solved until this work.
