<p>You can run CI on a snakemake workflow when provoding a minimal example dataset. Maybe an example from the snakemake-workflows repo will be helpful: <a href="https://github.com/snakemake-workflows/dna-seq-varlociraptor" rel="noreferrer">https://github.com/snakemake-workflows/dna-seq-varlociraptor</a></p>

CC BY-SA 4.0
2020-09-04T10:37:57.517Z
2020-09-04T10:37:57.517Z
jafors
<p>This is what I do when I want to check whether a program runs from start to finish and produces the expected output on test data. <a href="https://github.com/dariober/fxstat" rel="nofollow noreferrer">Here</a> I used it to test a C program but the idea can be applied also to a snakemake pipeline as in your case.</p>
<p>The idea is pretty simple really: use python's <code>subprocess</code> module to run the pipeline then check the results using the <code>unittest</code> module (or your favourite unit test package).</p>
<p>For example, save this in <code>test.py</code>:</p>
<pre><code>#!/usr/bin/env python3

import unittest
import os
import subprocess as sp

class Tester(unittest.TestCase):

    def testPipelineMakesStuff(self):
        # Run the pipeline
        p= sp.Popen('snakemake [some parameters]', shell=True, stdout= sp.PIPE, stderr= sp.PIPE)
        
        # Optionally, get stdout and stderr
        stdout, stderr= p.communicate()

        # Check exits code and other expected output            
        self.assertEqual(0, p.returncode)
        self.assertTrue(os.path.isfile('some.pipeline.output.txt'))

        ...Maybe read some.pipeline.output.txt and check it looks ok

    def testPipelineReturnsWithError(self):
        p= sp.Popen('snakemake [some parameters]', shell=True, stdout= sp.PIPE, stderr= sp.PIPE)
        stdout, stderr= p.communicate()
        self.assertTrue(p.returncode != 0)
        self.assertTrue('Something went wrong' in stderr.decode())
    
    def ...more test cases...

if __name__ == '__main__':
    unittest.main()
</code></pre>
<p>Run this script as:</p>
<pre><code> python test.py 
</code></pre>
<p>and check all tests pass ok. Of course, you can embed <code>python test.py</code> in travis or other continuous integration system as I've done <a href="https://github.com/dariober/fxstat/blob/master/.travis.yml" rel="nofollow noreferrer">here</a>.</p>

CC BY-SA 4.0
2020-09-10T13:14:04.080Z
2020-09-10T13:14:04.080Z
dariober
<p>I've been playing with snakemake, which is really powerful, however, I'd like to create a 'unit test' for the whole pipeline. e.g. a way to call the pipeline that tests that all the rules work together, all the steps complete, etc.</p>
<p>I assumed I could find some guides about how to do this, but I don't find any clear information about how to tests pipelines.</p>
<p>Any advice or suggestions would be welcome.</p>

CC BY-SA 4.0
2020-09-04T10:00:15.477Z
I made a feature request here: https://github.com/snakemake/snakemake/issues/599
Dan Bolser
<snakemake><reproducibility>
How do I write tests for a snakemake pipeline?
# How do I write tests for a snakemake pipeline?

(Asked by: username_0 on 2020-09-04)

I've been playing with snakemake, which is really powerful, however, I'd like to create a 'unit test' for the whole pipeline. e.g. a way to call the pipeline that tests that all the rules work together, all the steps complete, etc.

I assumed I could find some guides about how to do this, but I don't find any clear information about how to tests pipelines.

Any advice or suggestions would be welcome.

**Tags:** snakemake, reproducibility
> **Comments:**
> **username_0**: I made a feature request here: https://github.com/snakemake/snakemake/issues/599  

---

## 2 answers

### Answer by username_1 on 2020-09-04. Score: 5

You can run CI on a snakemake workflow when provoding a minimal example dataset. Maybe an example from the snakemake-workflows repo will be helpful: <https://github.com/snakemake-workflows/dna-seq-varlociraptor>


---

### Answer by username_2 on 2020-09-10. Score: 2

This is what I do when I want to check whether a program runs from start to finish and produces the expected output on test data. Here I used it to test a C program but the idea can be applied also to a snakemake pipeline as in your case.
The idea is pretty simple really: use python's subprocess module to run the pipeline then check the results using the unittest module (or your favourite unit test package).
For example, save this in test.py:
#!/usr/bin/env python3

import unittest
import os
import subprocess as sp

class Tester(unittest.TestCase):

    def testPipelineMakesStuff(self):
        # Run the pipeline
        p= sp.Popen('snakemake [some parameters]', shell=True, stdout= sp.PIPE, stderr= sp.PIPE)
        
        # Optionally, get stdout and stderr
        stdout, stderr= p.communicate()

        # Check exits code and other expected output            
        self.assertEqual(0, p.returncode)
        self.assertTrue(os.path.isfile('some.pipeline.output.txt'))

        ...Maybe read some.pipeline.output.txt and check it looks ok

    def testPipelineReturnsWithError(self):
        p= sp.Popen('snakemake [some parameters]', shell=True, stdout= sp.PIPE, stderr= sp.PIPE)
        stdout, stderr= p.communicate()
        self.assertTrue(p.returncode != 0)
        self.assertTrue('Something went wrong' in stderr.decode())
    
    def ...more test cases...

if __name__ == '__main__':
    unittest.main()

Run this script as:
 python test.py 

and check all tests pass ok. Of course, you can embed python test.py in travis or other continuous integration system as I've done here.


---

