  Segmental duplications (SDs), or low-copy repeats (LCR), are segments of DNA
greater than 1 Kbp with high sequence identity that are copied to other regions
of the genome. SDs are among the most important sources of evolution, a common
cause of genomic structural variation, and several are associated with diseases
of genomic origin. Despite their functional importance, SDs present one of the
major hurdles for de novo genome assembly due to the ambiguity they cause in
building and traversing both state-of-the-art overlap-layout-consensus and de
Bruijn graphs. This causes SD regions to be misassembled, collapsed into a
unique representation, or completely missing from assembled reference genomes
for various organisms. In turn, this missing or incorrect information limits
our ability to fully understand the evolution and the architecture of the
genomes. Despite the essential need to accurately characterize SDs in
assemblies, there is only one tool that has been developed for this purpose,
called Whole Genome Assembly Comparison (WGAC). WGAC is comprised of several
steps that employ different tools and custom scripts, which makes it difficult
and time consuming to use. Thus there is still a need for algorithms to
characterize within-assembly SDs quickly, accurately, and in a user friendly
manner.
  Here we introduce a SEgmental Duplication Evaluation Framework (SEDEF) to
rapidly detect SDs through sophisticated filtering strategies based on Jaccard
similarity and local chaining. We show that SEDEF accurately detects SDs while
maintaining substantial speed up over WGAC that translates into practical run
times of minutes instead of weeks. Notably, our algorithm captures up to 25%
pairwise error between segments, where previous studies focused on only 10%,
allowing us to more deeply track the evolutionary history of the genome.
  SEDEF is available at https://github.com/vpc-ccg/sedef
