  Noncoding RNAs are integral to a wide range of biological processes,
including translation, gene regulation, host-pathogen interactions and
environmental sensing. While genomics is now a mature field, our capacity to
identify noncoding RNA elements in bacterial and archaeal genomes is hampered
by the difficulty of de novo identification. The emergence of new technologies
for characterizing transcriptome outputs, notably RNA-seq, are improving
noncoding RNA identification and expression quantification. However, a major
challenge is to robustly distinguish functional outputs from transcriptional
noise. To establish whether annotation of existing transcriptome data has
effectively captured all functional outputs, we analysed over 400 publicly
available RNA-seq datasets spanning 37 different Archaea and Bacteria. Using
comparative tools, we identify close to a thousand highly-expressed candidate
noncoding RNAs. However, our analyses reveal that capacity to identify
noncoding RNA outputs is strongly dependent on phylogenetic sampling.
Surprisingly, and in stark contrast to protein-coding genes, the phylogenetic
window for effective use of comparative methods is perversely narrow:
aggregating public datasets only produced one phylogenetic cluster where these
tools could be used to robustly separate unannotated noncoding RNAs from a null
hypothesis of transcriptional noise. Our results show that for the full
potential of transcriptomics data to be realized, a change in experimental
design is paramount: effective transcriptomics requires phylogeny-aware
sampling.
