  We propose speaker separation using speaker inventories and estimated speech
(SSUSIES), a framework leveraging speaker profiles and estimated speech for
speaker separation. SSUSIES contains two methods, speaker separation using
speaker inventories (SSUSI) and speaker separation using estimated speech
(SSUES). SSUSI performs speaker separation with the help of speaker inventory.
By combining the advantages of permutation invariant training (PIT) and speech
extraction, SSUSI significantly outperforms conventional approaches. SSUES is a
widely applicable technique that can substantially improve speaker separation
performance using the output of first-pass separation. We evaluate the models
on both speaker separation and speech recognition metrics.
