<p>An unmapped SAM file format allows for read groups. You can use <code>samtools import</code> to create unmapped SAM files from FASTQ files:</p>
<pre><code>samtools import -R virus-1-2 in12.fq &gt; 12_reads.sam
samtools import -R virus-1-2-3 in123.fq &gt; 123_reads.sam
samtools merge out.sam 12_reads.sam 123_reads.sam
</code></pre>

CC BY-SA 4.0
2022-08-20T12:28:07.313Z
Perfect, thank you. It can be manipulated via Biopython `SamtoolsCalmdCommandline`
M__
CC BY-SA 4.0
2022-08-20T06:24:19.657Z
2022-08-20T06:24:19.657Z
gringer
<p>I'm awarding 'solved' to @gringer, because thats the context of the question.</p>
<p>The best solution I could find is stacking <code>Stockholm (sth)</code>  formats together. In this example the RNA families are associated with one structure:</p>
<pre><code># STOCKHOLM 1.0
#=GF ID    UPSK
AF035635.1/619-641             UGAGUUCUCGAUCUCUAAAAUCG
M24804.1/82-104                UGAGUUCUCUAUCUCUAAAAUCG
J04373.1/6212-6234             UAAGUUCUCGAUCUUUAAAAUCG
M24803.1/1-23                  UAAGUUCUCGAUCUCUAAAAUCG
#=GC SS_cons                   .AAA....&lt;&lt;&lt;&lt;aaa....&gt;&gt;&gt;&gt;
//
</code></pre>
<p><code>Biopython</code> allows <code>sth</code> format to be input.</p>
<pre><code>from Bio import AlignIO
from pathlib import Path

file = 'test.sth'
path = '/Volumes/mypath/testSth'
alignment = AlignIO.read(open(Path(path, file)), &quot;stockholm&quot;)
print(&quot;Alignment length %i&quot; % alignment.get_alignment_length())
print (alignment.column_annotations['secondary_structure'])
</code></pre>
<p>Output,</p>
<pre><code>Alignment length 23 
.AAA....&lt;&lt;&lt;&lt;aaa....&gt;&gt;&gt;&gt; # this is the RNA structure 
</code></pre>
<p>If we look the <code>alignment</code> object via</p>
<pre><code>from pprint import pprint
pprint(vars(alignment)) # or via print(alignment.__dict__)
</code></pre>
<p>The output</p>
<blockquote>
<p>{'_annotations': {},  '<strong>_per_col_annotations</strong>': {<strong>'secondary_structure'</strong>:
'.AAA....&lt;&lt;&lt;&lt;aaa....&gt;&gt;&gt;&gt;'},<br />
'_records':
[SeqRecord(seq=Seq('UGAGUUCUCGAUCUCUAAAAUCG'),
id='AF035635.1/619-641', name='AF035635.1',
description='AF035635.1/619-641', dbxrefs=[]),
SeqRecord(seq=Seq('UGAGUUCUCUAUCUCUAAAAUCG'), id='M24804.1/82-104', name='M24804.1', description='M24804.1/82-104',
dbxrefs=[]),
SeqRecord(seq=Seq('UAAGUUCUCGAUCUUUAAAAUCG'), id='J04373.1/6212-6234', name='J04373.1',
description='J04373.1/6212-6234', dbxrefs=[]),
SeqRecord(seq=Seq('UAAGUUCUCGAUCUCUAAAAUCG'), id='M24803.1/1-23', name='M24803.1', description='M24803.1/1-23',
dbxrefs=[])],  'annotations': {}} {<strong>'secondary_structure'</strong>:
'.AAA....&lt;&lt;&lt;&lt;aaa....&gt;&gt;&gt;&gt;'}</p>
</blockquote>
<p>The reason <code>vars(alignment)</code> is required is to internally construct an alignment object in Stockholm format, here's the first part of the skeleton 'pseudo-code',</p>
<pre><code>from Bio.Align import MultipleSeqAlignment
from Bio.SeqRecord import SeqRecord

outAlign = MultipleSeqAlignment([])
record in alignment:
    name = record.id 
    myseq = record.seq
    sr = SeqRecord(Seq(name), myseq, '', '')
    outAlign.append(sr)
</code></pre>
<p>The above code constructs a basic alignment, the bit thats missing is to add the following key/value to the <code>alignment</code> object of <code>{'secondary_structure': '.AAA....&lt;&lt;&lt;&lt;aaa....&gt;&gt;&gt;&gt;'}</code>.</p>
<p>Its complicated, but you get use to it.</p>
<hr />
<p><a href="https://github.com/biopython/biopython/blob/master/Bio/AlignIO/StockholmIO.py" rel="nofollow noreferrer">https://github.com/biopython/biopython/blob/master/Bio/AlignIO/StockholmIO.py</a></p>

CC BY-SA 4.0
2022-08-24T21:25:06.087Z
2022-09-01T20:05:08.297Z
2022-09-01T20:05:08.297Z
M__
<p>Is there a sequence format allowing multiple sequences to be grouped  across a flat file? The specific application is stacking separate nucleotide sequences against a given RNA secondary structure.</p>
<p>To rephrase the same question I'm looking to stack viral nucleotide data in different groups across a single flat file. FastaQ provides an example:</p>
<p>Traditional fastQ,</p>
<pre><code>@sequence_name
AGCTGAT
+
...(quality score)

@sequence2
AGCTGAC
+
...(quality score)
</code></pre>
<p>What I'm looking for is a format,</p>
<pre><code>@Gp1_virus1-2-3
AGACGTAG
+
AGAGACAA
+
AGAGACAT

@Gp2_virus1-2
AGATGTAG
+
AGATACAA
</code></pre>
<p>In the above example there are two groups comprising 3 (group 1) and 2 (group 2)  individual sequences (viruses). Here 'group' represents different structures.</p>
<p>Stockholme format has potential. My understanding of this format, which is basic, is it stacks a sequence, together with structural information. What is sought is stacking multiple sequences and then a structure format. Any clarification would help.</p>
<p>There are multiple XML formats, which may be relevant.</p>
<p>Just to mention a 'flat file' is needed to allow manual inspection. Dumping JSON is a solution, but isn't friendly for optical inspection.</p>
<hr />
<p><strong>Application</strong>
The immediate downstream application is structure/sequence viewers. I think at least one. viewer will accept Stockholme 1.0 format. rFAM is very similar (@terdon's comment)</p>
<p><strong>Summarised Question</strong> The question could be rephrased as to whether multiple RNA families can be stacked inside one Stockholme 1.0 or else rFAM format?</p>
<p>Some of the members here are likely familiar with <strong>pFAM</strong>. So could multiple protein structures be stored and retrieved inside one <strong>pFAM</strong> file.</p>
<p>@gringer's contribution is very welcome.</p>

CC BY-SA 4.0
2022-08-22T11:38:45.400Z
What is this for? I mean, since you don't seem to need something that will be understood by a specific downstream program, can't you just define your own format?
terdon
CC BY-SA 4.0
2022-08-22T15:00:19.460Z
Thanks, @terdon I've updated the post. A hybrid solution may be needed (last resort).
M__
<fastq><file-formats><format-conversion><secondary-structure>
Sequence format: grouping sequences within a flat file
# Sequence format: grouping sequences within a flat file

(Asked by: username_0 on 2022-08-19)

Is there a sequence format allowing multiple sequences to be grouped  across a flat file? The specific application is stacking separate nucleotide sequences against a given RNA secondary structure.
To rephrase the same question I'm looking to stack viral nucleotide data in different groups across a single flat file. FastaQ provides an example:
Traditional fastQ,
@sequence_name
AGCTGAT
+
...(quality score)

@sequence2
AGCTGAC
+
...(quality score)

What I'm looking for is a format,
@Gp1_virus1-2-3
AGACGTAG
+
AGAGACAA
+
AGAGACAT

@Gp2_virus1-2
AGATGTAG
+
AGATACAA

In the above example there are two groups comprising 3 (group 1) and 2 (group 2)  individual sequences (viruses). Here 'group' represents different structures.
Stockholme format has potential. My understanding of this format, which is basic, is it stacks a sequence, together with structural information. What is sought is stacking multiple sequences and then a structure format. Any clarification would help.
There are multiple XML formats, which may be relevant.
Just to mention a 'flat file' is needed to allow manual inspection. Dumping JSON is a solution, but isn't friendly for optical inspection.

Application
The immediate downstream application is structure/sequence viewers. I think at least one. viewer will accept Stockholme 1.0 format. rFAM is very similar (@terdon's comment)
Summarised Question The question could be rephrased as to whether multiple RNA families can be stacked inside one Stockholme 1.0 or else rFAM format?
Some of the members here are likely familiar with pFAM. So could multiple protein structures be stored and retrieved inside one pFAM file.
@gringer's contribution is very welcome.

**Tags:** fastq, file-formats, format-conversion, secondary-structure
> **Comments:**
> **username_1**: What is this for? I mean, since you don't seem to need something that will be understood by a specific downstream program, can't you just define your own format?  
> **username_0**: Thanks, @terdon I've updated the post. A hybrid solution may be needed (last resort).  

---

## 2 answers

### Answer by username_2 on 2022-08-20. Score: 2 | Accepted answer

An unmapped SAM file format allows for read groups. You can use samtools import to create unmapped SAM files from FASTQ files:
samtools import -R virus-1-2 in12.fq > 12_reads.sam
samtools import -R virus-1-2-3 in123.fq > 123_reads.sam
samtools merge out.sam 12_reads.sam 123_reads.sam

> **Comments:**
> **username_0**: Perfect, thank you. It can be manipulated via Biopython `SamtoolsCalmdCommandline`  

---

### Answer by username_0 on 2022-08-24. Score: 0

I'm awarding 'solved' to @gringer, because thats the context of the question.
The best solution I could find is stacking Stockholm (sth)  formats together. In this example the RNA families are associated with one structure:
# STOCKHOLM 1.0
#=GF ID    UPSK
AF035635.1/619-641             UGAGUUCUCGAUCUCUAAAAUCG
M24804.1/82-104                UGAGUUCUCUAUCUCUAAAAUCG
J04373.1/6212-6234             UAAGUUCUCGAUCUUUAAAAUCG
M24803.1/1-23                  UAAGUUCUCGAUCUCUAAAAUCG
#=GC SS_cons                   .AAA....<<<>>>
//

Biopython allows sth format to be input.
from Bio import AlignIO
from pathlib import Path

file = 'test.sth'
path = '/Volumes/mypath/testSth'
alignment = AlignIO.read(open(Path(path, file)), "stockholm")
print("Alignment length %i" % alignment.get_alignment_length())
print (alignment.column_annotations['secondary_structure'])

Output,
Alignment length 23 
.AAA....<<<>>> # this is the RNA structure 

If we look the alignment object via
from pprint import pprint
pprint(vars(alignment)) # or via print(alignment.__dict__)

The output

{'_annotations': {},  '_per_col_annotations': {'secondary_structure':
'.AAA....<<<>>>'},
'_records':
[SeqRecord(seq=Seq('UGAGUUCUCGAUCUCUAAAAUCG'),
id='AF035635.1/619-641', name='AF035635.1',
description='AF035635.1/619-641', dbxrefs=[]),
SeqRecord(seq=Seq('UGAGUUCUCUAUCUCUAAAAUCG'), id='M24804.1/82-104', name='M24804.1', description='M24804.1/82-104',
dbxrefs=[]),
SeqRecord(seq=Seq('UAAGUUCUCGAUCUUUAAAAUCG'), id='J04373.1/6212-6234', name='J04373.1',
description='J04373.1/6212-6234', dbxrefs=[]),
SeqRecord(seq=Seq('UAAGUUCUCGAUCUCUAAAAUCG'), id='M24803.1/1-23', name='M24803.1', description='M24803.1/1-23',
dbxrefs=[])],  'annotations': {}} {'secondary_structure':
'.AAA....<<<>>>'}

The reason vars(alignment) is required is to internally construct an alignment object in Stockholm format, here's the first part of the skeleton 'pseudo-code',
from Bio.Align import MultipleSeqAlignment
from Bio.SeqRecord import SeqRecord

outAlign = MultipleSeqAlignment([])
record in alignment:
    name = record.id 
    myseq = record.seq
    sr = SeqRecord(Seq(name), myseq, '', '')
    outAlign.append(sr)

The above code constructs a basic alignment, the bit thats missing is to add the following key/value to the alignment object of {'secondary_structure': '.AAA....<<<>>>'}.
Its complicated, but you get use to it.

https://github.com/biopython/biopython/blob/master/Bio/AlignIO/StockholmIO.py


---

