<blockquote>
<p>it seems cumbersome to create them to ensure execution order in my case</p>
</blockquote>
<p>It took me a while to figure out, but it did not seem cumbersome to me.</p>
<p>You can re-use your rules, see below, and make Step 1 of S2 depend on Step 2 of S1 like so:</p>
<pre><code>SAMPLES = ['S1', 'S2']


wildcard_constraints:
    sample = &quot;S[12]&quot;

rule all:
  input: expand(&quot;{sample}.done&quot;, sample = SAMPLES)



rule step1:
  input: data=&quot;{sample}&quot;
  output: data=&quot;{sample}.step1.done&quot;
  shell: &quot;touch {output.data}&quot;

rule step2:
  input: data=&quot;{sample}.step1.done&quot;
  output: data=&quot;{sample}.done&quot;
  shell: &quot;touch {output.data}&quot;


# step2 of S1 has the flag file as output

use rule step2 as step2ForS1 with:
    input: data=&quot;S1.step1.done&quot;
    output: data=&quot;S1.done&quot;, flag=touch(&quot;flags/step2-s1-done&quot;)

# step1 of S2 needs the flag file as input

use rule step1 as step1ForS2 with:
    input: data=&quot;S2&quot;, flag=&quot;flags/step2-s1-done&quot;
    output: data=&quot;S2.step1.done&quot;


</code></pre>

CC BY-SA 4.0
2022-10-14T22:47:41.430Z
2022-10-14T22:48:52.220Z
2022-10-14T22:48:52.220Z
mvtango
<p>The problem that you are encountering, is that when snakemake determines the order to run a set of rules it first builds a <a href="https://en.wikipedia.org/wiki/Directed_acyclic_graph" rel="nofollow noreferrer">Directed Acyclic Graph</a> (DAG) of all the rules. This shows which rules are are dependant on each other, and what order they can be safely executed in, without breaking downstream dependencies. </p>

<p>In general, if you want tasks to execute one after another which do not share an explicit file dependency, you can use a <a href="https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#flag-files" rel="nofollow noreferrer">Flag File</a> as an ouput for <code>rule 1</code> then reference it as an input in <code>rule 2</code>. </p>

<p>Unfortunately, I do not think this will help you in your case. You are expanding on the <code>SAMPLES</code> list and creating a wildcard. in the <code>DAG</code> this creates two competely separate execution branches which have no dependencies with each other. <code>step1</code> and <code>step2</code> are guaranteed to occur in order for each sample, however there is no guarantees between samples. </p>

<p>Is there a specific reason why you need all of <code>S2</code> tasks to be run after <code>S1</code>? If so, then your best bet would be to create explicit rules for sample 1 and sample 2, using the <code>Flag File</code> to incidate when <code>step2</code> for <code>S1</code> is done.</p>

CC BY-SA 4.0
2020-05-04T15:54:10.610Z
In typical analysis, we would like to see some results for the first sample, and make sure it is correct. So I would like to see S1 processed all the way first. Thanks for offering the Flag File feature. But it seems cumbersome to create them to ensure execution order in my case.
zhanxw
CC BY-SA 4.0
2020-05-05T00:11:06.190Z
@zhanxw your need seems to mostly center around QA of the pipeline, and making sure that its producing the right outputs. I often have to validate a pipeline before running 10s or 1000s of samples. The best way I found is to not try to gate the pipeline  like you are suggesting, but just run it once with a single sample,  then run it again with all of the remaining samples once you are satisfied.
Patrick Magee
CC BY-SA 4.0
2020-05-05T03:19:53.450Z
I see your points: run one sample at a time. I can do that, but that is not convenient as I need to write another script to run snakemake on each sample. I am curious if snakemake has already some solutions. Thanks.
zhanxw
CC BY-SA 4.0
2020-05-04T15:24:01.677Z
2020-05-04T15:24:01.677Z
Patrick Magee
<p>I think you should use the <a href="https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#priorities" rel="noreferrer">priority</a> directive to give precedence to downstream rule(s).</p>

<p>Here below I give higher priority to <code>step2</code> so when the first sample has completed <code>step1</code> snakemake will run step2 on that sample rather than submitting another sample to step1. </p>

<pre><code>SAMPLES = ['S1', 'S2', 'S3']

wildcard_constraints:
    sample= '|'.join([re.escape(x) for x in SAMPLES])

rule all:
    input: expand("{sample}.done", sample = SAMPLES)

rule step1:
    priority: 1
    output: "{sample}.step1.done"
    shell: 
        r"""
        sleep 5
        touch {output}
        """

rule step2:
    priority: 10
    input: "{sample}.step1.done"
    output: "{sample}.done"
    shell: 
        r"""
        sleep 5
        touch {output}
        """
</code></pre>

<p>(Note that without <code>wildcard_constraints</code> your/this code throws <code>AmbiguousRuleException</code>)</p>

CC BY-SA 4.0
2020-05-05T07:53:13.990Z
2020-05-05T07:53:13.990Z
dariober
<p>My goal is to process two samples, <code>S1</code> and <code>S2</code>, using two rules, <code>step1</code>, <code>step2</code>.
The Snakemake file is like:</p>

<pre><code>SAMPLES = ['S1', 'S2']

rule all:
  input: expand("{sample}.done", sample = SAMPLES)

rule step1:
  input: "{sample}"
  output: "{sample}.step1.done"
  shell: "touch {output}"

rule step2:
  input: "{sample}.step1.done"
  output: "{sample}.done"
  shell: "touch {output}"
</code></pre>

<p>The current execution order is: </p>

<pre><code>step1 on S1
step1 on S2
step2 on S1
step2 on S2
</code></pre>

<p>Is there a way to change the order to:</p>

<pre><code>step1 on S1
step2 on S1
step1 on S2
step2 on S2
</code></pre>

<p>Basically, I want to complete <code>S1</code>, then <code>S2</code>. 
Is there a way to specify the order of executions in Snakemake?</p>

<snakemake>
How to modify execution orders for Snakemake?
# How to modify execution orders for Snakemake?

(Asked by: username_0 on 2020-05-03)

My goal is to process two samples, S1 and S2, using two rules, step1, step2.
The Snakemake file is like:
SAMPLES = ['S1', 'S2']

rule all:
  input: expand("{sample}.done", sample = SAMPLES)

rule step1:
  input: "{sample}"
  output: "{sample}.step1.done"
  shell: "touch {output}"

rule step2:
  input: "{sample}.step1.done"
  output: "{sample}.done"
  shell: "touch {output}"

The current execution order is: 
step1 on S1
step1 on S2
step2 on S1
step2 on S2

Is there a way to change the order to:
step1 on S1
step2 on S1
step1 on S2
step2 on S2

Basically, I want to complete S1, then S2. 
Is there a way to specify the order of executions in Snakemake?

**Tags:** snakemake

---

## 3 answers

### Answer by username_3 on 2020-05-05. Score: 5 | Accepted answer

I think you should use the priority directive to give precedence to downstream rule(s).
Here below I give higher priority to step2 so when the first sample has completed step1 snakemake will run step2 on that sample rather than submitting another sample to step1. 
SAMPLES = ['S1', 'S2', 'S3']

wildcard_constraints:
    sample= '|'.join([re.escape(x) for x in SAMPLES])

rule all:
    input: expand("{sample}.done", sample = SAMPLES)

rule step1:
    priority: 1
    output: "{sample}.step1.done"
    shell: 
        r"""
        sleep 5
        touch {output}
        """

rule step2:
    priority: 10
    input: "{sample}.step1.done"
    output: "{sample}.done"
    shell: 
        r"""
        sleep 5
        touch {output}
        """

(Note that without wildcard_constraints your/this code throws AmbiguousRuleException)


---

### Answer by username_1 on 2022-10-14. Score: 1

it seems cumbersome to create them to ensure execution order in my case

It took me a while to figure out, but it did not seem cumbersome to me.
You can re-use your rules, see below, and make Step 1 of S2 depend on Step 2 of S1 like so:
SAMPLES = ['S1', 'S2']


wildcard_constraints:
    sample = "S[12]"

rule all:
  input: expand("{sample}.done", sample = SAMPLES)



rule step1:
  input: data="{sample}"
  output: data="{sample}.step1.done"
  shell: "touch {output.data}"

rule step2:
  input: data="{sample}.step1.done"
  output: data="{sample}.done"
  shell: "touch {output.data}"


# step2 of S1 has the flag file as output

use rule step2 as step2ForS1 with:
    input: data="S1.step1.done"
    output: data="S1.done", flag=touch("flags/step2-s1-done")

# step1 of S2 needs the flag file as input

use rule step1 as step1ForS2 with:
    input: data="S2", flag="flags/step2-s1-done"
    output: data="S2.step1.done"


---

### Answer by username_2 on 2020-05-04. Score: 0

The problem that you are encountering, is that when snakemake determines the order to run a set of rules it first builds a [Directed Acyclic Graph](https://en.wikipedia.org/wiki/Directed_acyclic_graph) (DAG) of all the rules. This shows which rules are are dependant on each other, and what order they can be safely executed in, without breaking downstream dependencies.

In general, if you want tasks to execute one after another which do not share an explicit file dependency, you can use a [Flag File](https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#flag-files) as an ouput for `rule 1` then reference it as an input in `rule 2`.

Unfortunately, I do not think this will help you in your case. You are expanding on the `SAMPLES` list and creating a wildcard. in the `DAG` this creates two competely separate execution branches which have no dependencies with each other. `step1` and `step2` are guaranteed to occur in order for each sample, however there is no guarantees between samples.

Is there a specific reason why you need all of `S2` tasks to be run after `S1`? If so, then your best bet would be to create explicit rules for sample 1 and sample 2, using the `Flag File` to incidate when `step2` for `S1` is done.

> **Comments:**
> **username_0**: In typical analysis, we would like to see some results for the first sample, and make sure it is correct. So I would like to see S1 processed all the way first. Thanks for offering the Flag File feature. But it seems cumbersome to create them to ensure execution order in my case.  
> **username_2**: @zhanxw your need seems to mostly center around QA of the pipeline, and making sure that its producing the right outputs. I often have to validate a pipeline before running 10s or 1000s of samples. The best way I found is to not try to gate the pipeline  like you are suggesting, but just run it once with a single sample,  then run it again with all of the remaining samples once you are satisfied.  
> **username_0**: I see your points: run one sample at a time. I can do that, but that is not convenient as I need to write another script to run snakemake on each sample. I am curious if snakemake has already some solutions. Thanks.  

---

