<p>This is a Snakemake feature: For small files (less than 100000 bytes), checksums are compared to determine if a job needs to run if the input file is newer than output file:</p>
<p><a href="https://github.com/snakemake/snakemake/pull/1568" rel="noreferrer">https://github.com/snakemake/snakemake/pull/1568</a></p>
<p>With a size of 99999 bytes:</p>
<pre><code><span class="math-container">$ dd if=/dev/zero of=a bs=99999 count=1
1+0 records in
1+0 records out
99999 bytes (100 kB, 98 KiB) copied, 0.000502722 s, 199 MB/s
$</span> snakemake -c1 b
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job       count    min threads    max threads
------  -------  -------------  -------------
sample        1              1              1
total         1              1              1

Select jobs to execute...

[Mon Oct 31 14:38:00 2022]
rule sample:
    input: a
    output: b
    jobid: 0
    reason: Missing output files: b
    resources: tmpdir=/tmp

[Mon Oct 31 14:38:00 2022]
Finished job 0.
1 of 1 steps (100%) done
Complete log: .snakemake/log/2022-10-31T143800.486966.snakemake.log
<span class="math-container">$ dd if=/dev/zero of=a bs=99999 count=1
1+0 records in
1+0 records out
99999 bytes (100 kB, 98 KiB) copied, 0.000180857 s, 553 MB/s
$</span> snakemake -c1 b
Building DAG of jobs...
Nothing to be done (all requested files are present and up to date).
Complete log: .snakemake/log/2022-10-31T143813.285988.snakemake.log
</code></pre>
<p>With a size of 100000 bytes:</p>
<pre><code><span class="math-container">$ dd if=/dev/zero of=a bs=100000 count=1
1+0 records in
1+0 records out
100000 bytes (100 kB, 98 KiB) copied, 0.000173008 s, 578 MB/s
$</span> snakemake -c1 b
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job       count    min threads    max threads
------  -------  -------------  -------------
sample        1              1              1
total         1              1              1

Select jobs to execute...

[Mon Oct 31 14:41:14 2022]
rule sample:
    input: a
    output: b
    jobid: 0
    reason: Missing output files: b
    resources: tmpdir=/tmp

[Mon Oct 31 14:41:14 2022]
Finished job 0.
1 of 1 steps (100%) done
Complete log: .snakemake/log/2022-10-31T144114.211255.snakemake.log
<span class="math-container">$ dd if=/dev/zero of=a bs=100000 count=1
1+0 records in
1+0 records out
100000 bytes (100 kB, 98 KiB) copied, 5.35361 s, 18.7 kB/s
$</span> snakemake -c1 b
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job       count    min threads    max threads
------  -------  -------------  -------------
sample        1              1              1
total         1              1              1

Select jobs to execute...

[Mon Oct 31 14:41:26 2022]
rule sample:
    input: a
    output: b
    jobid: 0
    reason: Updated input files: a
    resources: tmpdir=/tmp

[Mon Oct 31 14:41:26 2022]
Finished job 0.
1 of 1 steps (100%) done
Complete log: .snakemake/log/2022-10-31T144125.949828.snakemake.log
</code></pre>

CC BY-SA 4.0
2022-11-04T01:01:01.823Z
Thanks!  Do you know if there is a another mechanism that snakemake uses for "flag files" (https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#flag-files)?
Jules B
CC BY-SA 4.0
2022-11-04T13:01:09.253Z
@JulesB Looks like they're just touched or created (see [handle_touch()](https://github.com/snakemake/snakemake/blob/v7.18.1/snakemake/dag.py#L670-L676)). Then [touch()](https://github.com/snakemake/snakemake/blob/v7.18.1/snakemake/io.py#L688-L726) basically just calls [os.utime()](https://docs.python.org/3/library/os.html#os.utime) or `touch --no-dereference` to set the file access and modified times to the current time.
Steve
CC BY-SA 4.0
2022-10-31T04:17:13.090Z
2022-10-31T04:17:13.090Z
Steve
<p>I'm trying to use snakemake to replace some make workflows, but I'm finding that snakemake does not use the file modification time to trigger a rule.</p>
<p>Here is my snakefile</p>
<pre><code>rule sample:
    input:
        &quot;a&quot;
    output:
        &quot;b&quot;
    shell:
        &quot;touch {output}&quot;
</code></pre>
<p>If I create <code>a</code> then run snakemake, <code>b</code> is created as expected:</p>
<pre><code><span class="math-container">$ touch a
$</span> snakemake -c1 b
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job      count    min threads    max threads
-----  -------  -------------  -------------
sample       1              1              1
total        1              1              1

Select jobs to execute...

[Sat Oct 29 14:36:43 2022]
rule sample:
    input: a
    output: b
    jobid: 0
    reason: Missing output files: b
    resources: tmpdir=/tmp

[Sat Oct 29 14:36:43 2022]
Finished job 0.
1 of 1 steps (100%) done
Complete log: .snakemake/log/2022-10-29T143643.334050.snakemake.log
$ ls -l a b
-rw-r--r-- 1 ... ... 0 Oct 29 14:36 a
-rw-r--r-- 1 ... ... 0 Oct 29 14:36 b
</code></pre>
<p>Now, if I touch <code>a</code> then ask snakemake to update <code>b</code>, nothing happens:</p>
<pre><code>$ touch a
$ snakemake -c1 b
Building DAG of jobs...
Nothing to be done (all requested files are present and up to date).
Complete log: .snakemake/log/2022-10-29T144005.589943.snakemake.log
$ ls -l a b
-rw-r--r-- 1 ... ... 0 Oct 29 14:40 a
-rw-r--r-- 1 ... ... 0 Oct 29 14:36 b
</code></pre>
<p>Modifying the contents of <code>a</code> triggers an update as expected:</p>
<pre><code><span class="math-container">$ echo new &gt; a
$</span> snakemake -c1 b
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job      count    min threads    max threads
-----  -------  -------------  -------------
sample       1              1              1
total        1              1              1

Select jobs to execute...

[Sat Oct 29 14:42:13 2022]
rule sample:
    input: a
    output: b
    jobid: 0
    reason: Updated input files: a
    resources: tmpdir=/tmp

[Sat Oct 29 14:42:13 2022]
Finished job 0.
1 of 1 steps (100%) done
Complete log: .snakemake/log/2022-10-29T144213.161985.snakemake.log
$ ls -l a b
-rw-r--r-- 1 ... ... 4 Oct 29 14:42 a
-rw-r--r-- 1 ... ... 0 Oct 29 14:42 b
</code></pre>
<p>I installed snakemake-minimal (version 7.17.1) from bioconda, am using python 3.9.13 from conda-forge, on a RHEL 7 system.</p>
<pre><code>$ snakemake --version
7.17.1
$ python --version
Python 3.9.13
$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.8 (Maipo)
</code></pre>
<p>I'm clearly misunderstanding something very basic about snakemake.  Any suggestions on how to debug this, descriptions of how to have snakemake pay attention to file modification times, or just an explanation of why snakemake behaves differently from make would be much appreciated!</p>
<p>thanks in advance,</p>

<snakemake>
Modification time does not trigger a snakemake rule
# Modification time does not trigger a snakemake rule

(Asked by: username_0 on 2022-10-29)

I'm trying to use snakemake to replace some make workflows, but I'm finding that snakemake does not use the file modification time to trigger a rule.
Here is my snakefile
rule sample:
    input:
        "a"
    output:
        "b"
    shell:
        "touch {output}"

If I create a then run snakemake, b is created as expected:
$ touch a
$ snakemake -c1 b
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job      count    min threads    max threads
-----  -------  -------------  -------------
sample       1              1              1
total        1              1              1

Select jobs to execute...

[Sat Oct 29 14:36:43 2022]
rule sample:
    input: a
    output: b
    jobid: 0
    reason: Missing output files: b
    resources: tmpdir=/tmp

[Sat Oct 29 14:36:43 2022]
Finished job 0.
1 of 1 steps (100%) done
Complete log: .snakemake/log/2022-10-29T143643.334050.snakemake.log
$ ls -l a b
-rw-r--r-- 1 ... ... 0 Oct 29 14:36 a
-rw-r--r-- 1 ... ... 0 Oct 29 14:36 b

Now, if I touch a then ask snakemake to update b, nothing happens:
$ touch a
$ snakemake -c1 b
Building DAG of jobs...
Nothing to be done (all requested files are present and up to date).
Complete log: .snakemake/log/2022-10-29T144005.589943.snakemake.log
$ ls -l a b
-rw-r--r-- 1 ... ... 0 Oct 29 14:40 a
-rw-r--r-- 1 ... ... 0 Oct 29 14:36 b

Modifying the contents of a triggers an update as expected:
$ echo new > a
$ snakemake -c1 b
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job      count    min threads    max threads
-----  -------  -------------  -------------
sample       1              1              1
total        1              1              1

Select jobs to execute...

[Sat Oct 29 14:42:13 2022]
rule sample:
    input: a
    output: b
    jobid: 0
    reason: Updated input files: a
    resources: tmpdir=/tmp

[Sat Oct 29 14:42:13 2022]
Finished job 0.
1 of 1 steps (100%) done
Complete log: .snakemake/log/2022-10-29T144213.161985.snakemake.log
$ ls -l a b
-rw-r--r-- 1 ... ... 4 Oct 29 14:42 a
-rw-r--r-- 1 ... ... 0 Oct 29 14:42 b

I installed snakemake-minimal (version 7.17.1) from bioconda, am using python 3.9.13 from conda-forge, on a RHEL 7 system.
$ snakemake --version
7.17.1
$ python --version
Python 3.9.13
$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.8 (Maipo)

I'm clearly misunderstanding something very basic about snakemake.  Any suggestions on how to debug this, descriptions of how to have snakemake pay attention to file modification times, or just an explanation of why snakemake behaves differently from make would be much appreciated!
thanks in advance,

**Tags:** snakemake

---

## 1 answers

### Answer by username_1 on 2022-10-31. Score: 5

This is a Snakemake feature: For small files (less than 100000 bytes), checksums are compared to determine if a job needs to run if the input file is newer than output file:
https://github.com/snakemake/snakemake/pull/1568
With a size of 99999 bytes:
$ dd if=/dev/zero of=a bs=99999 count=1
1+0 records in
1+0 records out
99999 bytes (100 kB, 98 KiB) copied, 0.000502722 s, 199 MB/s
$ snakemake -c1 b
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job       count    min threads    max threads
------  -------  -------------  -------------
sample        1              1              1
total         1              1              1

Select jobs to execute...

[Mon Oct 31 14:38:00 2022]
rule sample:
    input: a
    output: b
    jobid: 0
    reason: Missing output files: b
    resources: tmpdir=/tmp

[Mon Oct 31 14:38:00 2022]
Finished job 0.
1 of 1 steps (100%) done
Complete log: .snakemake/log/2022-10-31T143800.486966.snakemake.log
$ dd if=/dev/zero of=a bs=99999 count=1
1+0 records in
1+0 records out
99999 bytes (100 kB, 98 KiB) copied, 0.000180857 s, 553 MB/s
$ snakemake -c1 b
Building DAG of jobs...
Nothing to be done (all requested files are present and up to date).
Complete log: .snakemake/log/2022-10-31T143813.285988.snakemake.log

With a size of 100000 bytes:
$ dd if=/dev/zero of=a bs=100000 count=1
1+0 records in
1+0 records out
100000 bytes (100 kB, 98 KiB) copied, 0.000173008 s, 578 MB/s
$ snakemake -c1 b
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job       count    min threads    max threads
------  -------  -------------  -------------
sample        1              1              1
total         1              1              1

Select jobs to execute...

[Mon Oct 31 14:41:14 2022]
rule sample:
    input: a
    output: b
    jobid: 0
    reason: Missing output files: b
    resources: tmpdir=/tmp

[Mon Oct 31 14:41:14 2022]
Finished job 0.
1 of 1 steps (100%) done
Complete log: .snakemake/log/2022-10-31T144114.211255.snakemake.log
$ dd if=/dev/zero of=a bs=100000 count=1
1+0 records in
1+0 records out
100000 bytes (100 kB, 98 KiB) copied, 5.35361 s, 18.7 kB/s
$ snakemake -c1 b
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job       count    min threads    max threads
------  -------  -------------  -------------
sample        1              1              1
total         1              1              1

Select jobs to execute...

[Mon Oct 31 14:41:26 2022]
rule sample:
    input: a
    output: b
    jobid: 0
    reason: Updated input files: a
    resources: tmpdir=/tmp

[Mon Oct 31 14:41:26 2022]
Finished job 0.
1 of 1 steps (100%) done
Complete log: .snakemake/log/2022-10-31T144125.949828.snakemake.log

> **Comments:**
> **username_0**: Thanks!  Do you know if there is a another mechanism that snakemake uses for "flag files" (https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#flag-files)?  
> **username_1**: @JulesB Looks like they're just touched or created (see [handle_touch()](https://github.com/snakemake/snakemake/blob/v7.18.1/snakemake/dag.py#L670-L676)). Then [touch()](https://github.com/snakemake/snakemake/blob/v7.18.1/snakemake/io.py#L688-L726) basically just calls [os.utime()](https://docs.python.org/3/library/os.html#os.utime) or `touch --no-dereference` to set the file access and modified times to the current time.  

---

