<p>Although @JamesHawley answer was informative, I don't think it solved the initial issue requested by the OP. (<strong>edit</strong>: or maybe it partially did, I'm not sure as I got a bit confused with the wildcards on the initial question/answer. I'm anyway leaving this answer here, in case another way of explaining it is useful for others)</p>

<h3>Solution</h3>

<p>I think you only need to change the output of your <code>rule multiqc</code> from:</p>

<p><code>os.path.join(qc_dir, 'multiqc_report.html')</code></p>

<p>to</p>

<p><code>os.path.join(qc_dir, '{sample}.html')</code></p>

<p>This seemed counter-intuitive to me initially, but I think what's happening is that <code>snakemake</code> will look at <code>rule all</code> and see that you want an output called "qc/multiqc_report.html". And the only place where it can find something of that sort is in <code>rule multiqc</code>, by replacing the wildcard <code>{sample}</code> by the string <code>multiqc_report</code>. </p>

<p>It felt confusing to me, because the same <code>{sample}</code> wildcard is used in other rules (namely the rules you use to generate the outputs in your <code>analysed_dir</code>). But <code>snakemake</code> will be smart to replace <code>{sample}</code> with whatever it needs to recreate the output files you're requesting. </p>

<hr>

<h3>Longer explanation</h3>

<p>Here is a (hopefully) reproducible example, if it helps.</p>

<p>First create some fake directory structure with some files:</p>

<pre class="lang-sh prettyprint-override"><code>mkdir inputs
mkdir analysed_dir
mkdir qc_dir

touch inputs/file1.txt
touch inputs/file2.txt
</code></pre>

<p>Then, this would be the snakefile:</p>

<pre class="lang-py prettyprint-override"><code># Define your variables, fetch things from config, or csv, etc...
input_dir = "inputs"
output_dir = "outputs"
samples = ["file1.txt", "file2.txt"]


# Rule all defining all the output files required
rule all:
  input:
    expand("analysed_dir/{samples}", samples = samples),
    "qc_dir/multi_qc_report.html"

# this rule is equivalent to whichever rules process your data individually
rule analyse_each_file:
  input:
    "inputs/{sample}.txt"
  output:
    "analysed_dir/{sample}.txt"
  shell:
    "cat {input} &gt; {output}"

# This rule is equivalent to your multiqc step where multiple files result in a single output
rule multiqc:
  input:
    expand("analysed_dir/{samples}", samples = samples)
  output:
    "qc_dir/{sample}.html"
  shell:
    "cat {input} &gt; {output}"
</code></pre>

<p>to generate this pipeline:</p>

<p><a href="https://i.sstatic.net/9LnDy.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/9LnDy.png" alt="dag"></a></p>

<p>And then issuing:</p>

<p><code>snakemake --cluster "qsub -N {wildcards.sample}-{rule}" --jobs 10</code></p>

<p>Generates the following job names:</p>

<ul>
<li>file1-analyse_each_file</li>
<li>file2-analyse_each_file</li>
<li>multi_qc_report-multiqc</li>
</ul>

<p><strong>Edit with further explanation:</strong> </p>

<p>Notice in the DAG how the <code>sample</code> wildcard takes different values for the different rules. I think the snakemake logic is as follows:</p>

<ul>
<li>Starting with <strong><code>rule all</code></strong> I see that you want three files as input (the first two come from the <code>expand()</code> call): 

<ul>
<li><code>analysed_dir/file1.txt</code></li>
<li><code>analysed_dir/file2.txt</code></li>
<li><code>qc_dir/multi_qc_report.html</code></li>
</ul></li>
<li>Let me start with the first one <code>analysed_dir/file1.txt</code>. The file doesn't exist, so let me look in the other rules and see if I can find it in their output. Actually in <strong><code>rule analyse_each_file</code></strong> I see the output is set to <code>"analysed_dir/{sample}.txt"</code>, so if I replace <code>{sample}</code> by <code>file1.txt</code> I can recreate your file from it. Therefore, the input for that rule will be <code>"inputs/file1.txt"</code> (and this file already exists, so I can issue this job).</li>
<li>For the second file <code>analysed_dir/file2.txt</code>, well it's a similar case as with the previous one, so I'll issue a job with the same <strong><code>rule analyse_each_file</code></strong>, but this time <code>{sample}</code> = <code>file2.txt</code>.</li>
<li>Finally, you want a file called <code>qc_dir/multi_qc_report.html</code>. Where can I create such a file from? Well, the output of <strong><code>rule multiqc</code></strong> is set to <code>"qc_dir/{sample}.html"</code>, so if I make <code>{sample}</code> = <code>multi_qc_report</code>, then I can issue the job and produce your file of interest. In this case, the rule takes as input two files <code>"analysed_dir/file1.txt"</code> and <code>"analysed_dir/file2.txt"</code>, which I already know how to create from another rule. </li>
</ul>

<p>The key thing to realise is that <code>snakemake</code> <strong>defines the wildcard <code>{sample}</code> for each rule separately</strong>. So, in this example, <code>wildcard.sample</code> in <code>rule analyse_each_file</code> does not take on the same values as in <code>rule multiqc</code>.</p>

CC BY-SA 4.0
2020-01-15T13:08:06.340Z
Thank you for your comment, I still don't have a satisfying solution (meaning I now just leave out the sample wildcard from the qsub job names). For your solution I wonder, will Snakemake not complain about "qc_dir/{sample}.html" never being generated? I think the core issue is that the Multiqc rule takes in all samples so it is correct that there is no single "sample" wildcard. I was just looking for a way to set the sample wildcards to something like "all_samples" for the purpose of naming the qsubbed jobs. Am I making any sense?
Freek
CC BY-SA 4.0
2020-01-16T11:19:30.637Z
@Freek, when you say "I still don't have a satisfying solution", do you mean you tried my solution and it didn't work? (note, I've edited my solution to more correctly match your posted code using `os.path.join()` - see the **Solution** section at the top). I don't think `snakemake` will get confused, because it seems to me that _wildcards are defined within each rule individually_. See my further edit with expanded explanation at the end of the answer (below the DAG). I hope it helps.
hugot
CC BY-SA 4.0
2020-01-14T10:54:25.777Z
2020-01-16T11:23:18.193Z
2020-01-16T11:23:18.193Z
hugot
<p>Snakemake gets <code>wildcards</code> from the parsing the input/output file names.
For example, if you had</p>

<pre><code>rule sort:
    input: "{file}.bed"
    output: "{file}.sorted.bed"
    shell: "sort -k1,1 -k2,2n {input} &gt; {output}"
</code></pre>

<p>then the <code>wildcards</code> variable would be set as <code>wildcards = {"file": ...}</code>.</p>

<p>In your <code>multiqc</code> rule, you have the <code>{sample}</code> variable within your <code>expand()</code>, but this variable doesn't exist outside of <code>expand</code>'s local context. You don't actually have any wildcards there, nor in your output file (<code>qc_dir</code> and <code>rseqc_dir</code> are variables you define elsewhere, but not "wildcards" as Snakemake interprets them).</p>

<p>So you're getting the error <code>'Wildcards' object has no attribute 'sample'</code> because there are no wildcards for that rule.</p>

<p>Your rule is currently set up to process every sample that you have listed in  <code>samples['sample']</code>. If you want your <code>multiqc</code> rule to work for a single sample, simply remove the <code>sample=samples['sample']</code> and double brace the <code>{sample}</code> wildcard.</p>

<pre><code>rule multiqc:
    input:
        expand([os.path.join(analyzed_dir, '{{sample}}.stat', '{{sample}}.cnt'),
                os.path.join(rseqc_dir, '{{sample}}.bam_stat.txt'),
                os.path.join(rseqc_dir, '{{sample}}.clipping_profile.xls'),
                os.path.join(rseqc_dir, '{{sample}}.deletion_profile.txt'),
                os.path.join(rseqc_dir, '{{sample}}.infer_experiment.txt'),
                os.path.join(rseqc_dir, '{{sample}}.geneBodyCoverage.txt'),
                os.path.join(rseqc_dir, '{{sample}}.inner_distance.txt'),
                os.path.join(rseqc_dir, '{{sample}}.insertion_profile.xls'),
                os.path.join(rseqc_dir, '{{sample}}.junction.xls'),
                os.path.join(rseqc_dir, '{{sample}}.junctionSaturation_plot.r'),
                os.path.join(rseqc_dir, '{{sample}}.mismatch_profile.xls'),
                os.path.join(rseqc_dir, '{{sample}}.read_distribution.txt'),
                os.path.join(rseqc_dir, '{{sample}}.pos.DupRate.xls'),
                os.path.join(rseqc_dir, '{{sample}}.seq.DupRate.xls'),
                os.path.join(rseqc_dir, '{{sample}}.GC.xls'),
                os.path.join(rseqc_dir, '{{sample}}.NVC.xls'),
                os.path.join(rseqc_dir, '{{sample}}.qual.r'),
                os.path.join(rseqc_dir, '{{sample}}.RNA_fragment_size.txt'),
                os.path.join(rseqc_dir, '{{sample}}.STAR.genome.sorted.summary.txt'),
                os.path.join(fastqc_dir, '{{sample}}_R{read}_001_fastqc.html')],
                read=reads)
    output:
        # this {sample} is in single {} and is recognized as a wildcard
        os.path.join(qc_dir, '{sample}', 'multiqc_report.html') 
    conda:
        "envs/multiqc.yaml"
    shell:
        '''
        #!/bin/bash
        # using {wildcards.sample} here since MultiQC likes directories
        multiqc . --outdir qc/{wildcards.sample} --ignore .snakemake --force
        '''
</code></pre>

<p>When wildcards are recognized, you'll see them in the job list.</p>

<p>Running <code>snakemake 0067_P2018SEQE27R01_S1/multiqc_report.html</code>, for example, should give you something like:</p>

<pre><code>rule multiqc:
    input: qc/rseqc/0067_P2018SEQE27R01_S1.geneBodyCoverage.txt, ...
    output: qc/0067_P2018SEQE27R01_S1/multiqc_report.html
    jobid: 2
    wildcards: sample=0067_P2018SEQE27R01_S1
</code></pre>

CC BY-SA 4.0
2019-01-15T20:02:11.857Z
Thank you, nicely explained, I'll accept this answer. But, do you know of any way to temporarily set the wildcards.sample variable? So that I can use it for naming my multiqc job? I find it a shame that because of this one rule I can't have the {sample}_{rule} naming scheme anymore...
Freek
CC BY-SA 4.0
2019-01-15T22:59:06.870Z
Maybe you could try using multiple bash commands? Something like `multiqc . --outdir qc --ignore .snakemake --force && mv multiqc_report.html {wildcards.sample}_multiqc.html`?
James Hawley
CC BY-SA 4.0
2019-01-15T23:00:28.007Z
But I think that because multiqc scans the entire directory, all your files are going to get scanned anyway, and end up in each report. So I don't know if they're actually going to be any different, or if you're just going to end up with the same file under multiple names
James Hawley
CC BY-SA 4.0
2020-01-13T16:39:38.640Z
@JamesHawley, I thought the OP wanted many inputs (defined by the `expand()` call) to a single output (`qc/multiqc_report.html`), rather than your solution, which if I understand results in an output for every input. It should be possible to achieve what the OP asked for, and the [multiqc wrapper](https://snakemake-wrappers.readthedocs.io/en/stable/wrappers/multiqc.html) seems to suggest so. In which case, what was wrong with the OP's code? (I'm having the same error in a similar situation, thus my comment)
hugot
CC BY-SA 4.0
2020-01-13T16:58:16.047Z
I got it now: the problem wasn't with the rule's code itself, it was to do with the call to `--cluster`. So, there is no known solution for this problem of wanting to use the `wildcards.sample` in the call to `qsub` when there is one, but otherwise not? This would be particularly useful for log files.
hugot
CC BY-SA 4.0
2019-01-14T23:37:39.667Z
2019-01-15T14:46:22.620Z
2019-01-15T14:46:22.620Z
James Hawley
<p>I'm using Snakemake and I'm trying to get all my QC reports into multiqc but I get the following error:</p>

<pre><code>WorkflowError in line 120 of /rst1/2017-0205_illuminaseq/scratch/swo-406/test_snakemake_full/Snakefile:
'Wildcards' object has no attribute 'sample'
</code></pre>

<p>Rule 120 is the start of my multiqc rule which looks like this:</p>

<pre><code>rule multiqc:
    input:
        expand([os.path.join(analyzed_dir, '{sample}.stat', '{sample}.cnt'),
                os.path.join(rseqc_dir, '{sample}.bam_stat.txt'),
                os.path.join(rseqc_dir, '{sample}.clipping_profile.xls'),
                os.path.join(rseqc_dir, '{sample}.deletion_profile.txt'),
                os.path.join(rseqc_dir, '{sample}.infer_experiment.txt'),
                os.path.join(rseqc_dir, '{sample}.geneBodyCoverage.txt'),
                os.path.join(rseqc_dir, '{sample}.inner_distance.txt'),
                os.path.join(rseqc_dir, '{sample}.insertion_profile.xls'),
                os.path.join(rseqc_dir, '{sample}.junction.xls'),
                os.path.join(rseqc_dir, '{sample}.junctionSaturation_plot.r'),
                os.path.join(rseqc_dir, '{sample}.mismatch_profile.xls'),
                os.path.join(rseqc_dir, '{sample}.read_distribution.txt'),
                os.path.join(rseqc_dir, '{sample}.pos.DupRate.xls'),
                os.path.join(rseqc_dir, '{sample}.seq.DupRate.xls'),
                os.path.join(rseqc_dir, '{sample}.GC.xls'),
                os.path.join(rseqc_dir, '{sample}.NVC.xls'),
                os.path.join(rseqc_dir, '{sample}.qual.r'),
                os.path.join(rseqc_dir, '{sample}.RNA_fragment_size.txt'),
                os.path.join(rseqc_dir, '{sample}.STAR.genome.sorted.summary.txt'),
                os.path.join(fastqc_dir, '{sample}_R{read}_001_fastqc.html')],
                sample=samples['samples'], read=reads)
    output:
        os.path.join(qc_dir, 'multiqc_report.html') 
    conda:
        "envs/multiqc.yaml"
    shell:
        '''
        #!/bin/bash
        multiqc . --outdir qc --ignore .snakemake --force
        '''
</code></pre>

<p>My "rule all looks like this:"</p>

<pre><code>rule all:
    input:
        expand([os.path.join(analyzed_dir, '{sample}.genes.results'),
                os.path.join(qc_dir, 'multiqc_report.html')],
                sample=samples['samples'])
</code></pre>

<p>I don't see anything wrong with it, in fact, it is very similar to my "rule all" I used before and which worked. Can anyone see an error?</p>

<p>EDIT</p>

<p>I limited the rule to only one input, I still have the same error:</p>

<pre><code>rule multiqc:
    input:
        expand(os.path.join(rseqc_dir, '{sample}.geneBodyCoverage.txt'),
                sample=samples['samples'])
    output:
        os.path.join(qc_dir, 'multiqc_report.html') 
    conda:
        "envs/multiqc.yaml"
    shell:
        '''
        #!/bin/bash
        multiqc . --outdir qc --ignore .snakemake --force
        '''
</code></pre>

<p>This is the ouput, you can see it read the samples correctly:</p>

<pre><code>run_snakemake.sh 
/rst1/2017-0205_illuminaseq/scratch/swo-406/snakemake/lib/python3.6/site-packages/snakemake/workflow.py:12: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'.
  from collections import OrderedDict
Building DAG of jobs...
/rst1/2017-0205_illuminaseq/scratch/swo-406/snakemake/lib/python3.6/site-packages/snakemake/workflow.py:12: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'.
  from collections import OrderedDict
Building DAG of jobs...
Using shell: /bin/bash
Provided cluster nodes: 256
Job counts:
    count   jobs
    1   all
    1   multiqc
    2

[Mon Jan 14 17:46:40 2019]
rule multiqc:
    input: qc/rseqc/0067_P2018SEQE27R01_S1.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R02_S2.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R03_S3.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R04_S4.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R05_S5.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R06_S6.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R07_S7.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R08_S8.geneBodyCoverage.txt
    output: qc/multiqc_report.html
    jobid: 2

WorkflowError in line 119 of /rst1/2017-0205_illuminaseq/scratch/swo-406/test_snakemake_full/Snakefile:
'Wildcards' object has no attribute 'sample'
17:46 nlv24077@kiato /rst1/2017-0205_illuminaseq/scratch/swo-406/test_snakemake_full &gt; ll qc/rseqc/0067_P2018SEQE27R07_S7.geneBodyCoverage.txt
-rw-rw-r-- 1 nlv24077 2017-0205_illuminaseq 303 Jan  7 16:00 qc/rseqc/0067_P2018SEQE27R07_S7.geneBodyCoverage.txt
</code></pre>

<p>EDIT 2:</p>

<p>This is how I start Snakemake:</p>

<pre><code>&gt; cat run_snakemake.sh
source activate /rst1/2017-0205_illuminaseq/scratch/swo-406/snakemake
snakemake --dag | dot -T svg &gt; dag.svg
snakemake --cluster-config cluster.json --cluster "qsub -l nodes={cluster.nodes}:ppn={cluster.ppn} -N {wildcards.sample}_{rule}" --jobs 256 --use-conda --rerun-incomplete
</code></pre>

<p>EDIT 3:</p>

<p>Ok, that last remark (by @finswimmer) triggered me, I removed the "{wildcards.sample}_" from the above command and then it works. Or, works, I get a very different error, it is multiqc specific and solved as specified here: <a href="https://multiqc.info/docs/#locale-error-messages" rel="nofollow noreferrer">https://multiqc.info/docs/#locale-error-messages</a>. When adding the two lines as suggested this error disappears and everything works as expected, even with the long inputs list.</p>

<pre><code>RuntimeError: Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment. Consult https://click.palletsprojects.com/en/7.x/python3/ for mitigation steps.
</code></pre>

<p>Anyway, I'm guessing Snakemake does not understand how to name this task when it uses an entire list of {sample}'s... Is this a bug? What should be used? I guess it is impossible to use {sample} for naming for this rule. It's a shame I have to kill the naming for all rules because using sample_rule is very nice for keeping track.</p>

CC BY-SA 4.0
2019-01-14T00:09:40.980Z
Can you start by limiting the input to that rule to a couple files? That will at least help debug this.
Devon Ryan
CC BY-SA 4.0
2019-01-14T09:03:52.900Z
I don't know anything about snakemake but the problem seems to come from not declaring what a `sample` should be
llrs
CC BY-SA 4.0
2019-01-14T09:21:18.013Z
@llrs `sample` is defined in the `expand()` function. Oddly, there's no direct use of `wildcards` outside of that, which is usually where this sort of thing crops up.
Devon Ryan
CC BY-SA 4.0
2019-01-14T14:45:24.463Z
Is `samples['samples']` defined somewhere?
finswimmer
CC BY-SA 4.0
2019-01-14T17:32:26.997Z
And how does your command look like to start the workflow?
finswimmer
CC BY-SA 4.0
2019-01-14T21:07:29.417Z
I think you could use it, but not that way. But I think that would be another and please post how you solved it (I want to learn to use snakemake).
llrs
CC BY-SA 4.0
2019-01-14T21:47:55.143Z
Hi @llrs, the answer is in EDIT 2, there is my "start snakemake" script. I used the wildcards.sample to name the jobs on our qsub cluster, but I can't do that because my MultiQC does not have a single defined wildcards.sample (it has many). So I simply removed that part (the {wildcards.sample}) and now I only have the rules as the jobname, sadly. Does that help? How do you think I could use it then?
Freek
CC BY-SA 4.0
2019-01-15T08:04:57.483Z
@Freek Well, to signal that you got an answer here (on the stackexchange.com sites) we post an answer! You can post an answer to your own question (it is completely normal and encouraged). In the answer please explain how it solved your problem as you did in this comment.
llrs
CC BY-SA 4.0
2019-01-15T20:03:47.723Z
Ok, thanx, I'll remember that, for now James Hawley worded it nicely already :)
Freek
<snakemake>
'Wildcards' object has no attribute 'sample'
# 'Wildcards' object has no attribute 'sample'

(Asked by: username_0 on 2019-01-13)

I'm using Snakemake and I'm trying to get all my QC reports into multiqc but I get the following error:
WorkflowError in line 120 of /rst1/2017-0205_illuminaseq/scratch/swo-406/test_snakemake_full/Snakefile:
'Wildcards' object has no attribute 'sample'

Rule 120 is the start of my multiqc rule which looks like this:
rule multiqc:
    input:
        expand([os.path.join(analyzed_dir, '{sample}.stat', '{sample}.cnt'),
                os.path.join(rseqc_dir, '{sample}.bam_stat.txt'),
                os.path.join(rseqc_dir, '{sample}.clipping_profile.xls'),
                os.path.join(rseqc_dir, '{sample}.deletion_profile.txt'),
                os.path.join(rseqc_dir, '{sample}.infer_experiment.txt'),
                os.path.join(rseqc_dir, '{sample}.geneBodyCoverage.txt'),
                os.path.join(rseqc_dir, '{sample}.inner_distance.txt'),
                os.path.join(rseqc_dir, '{sample}.insertion_profile.xls'),
                os.path.join(rseqc_dir, '{sample}.junction.xls'),
                os.path.join(rseqc_dir, '{sample}.junctionSaturation_plot.r'),
                os.path.join(rseqc_dir, '{sample}.mismatch_profile.xls'),
                os.path.join(rseqc_dir, '{sample}.read_distribution.txt'),
                os.path.join(rseqc_dir, '{sample}.pos.DupRate.xls'),
                os.path.join(rseqc_dir, '{sample}.seq.DupRate.xls'),
                os.path.join(rseqc_dir, '{sample}.GC.xls'),
                os.path.join(rseqc_dir, '{sample}.NVC.xls'),
                os.path.join(rseqc_dir, '{sample}.qual.r'),
                os.path.join(rseqc_dir, '{sample}.RNA_fragment_size.txt'),
                os.path.join(rseqc_dir, '{sample}.STAR.genome.sorted.summary.txt'),
                os.path.join(fastqc_dir, '{sample}_R{read}_001_fastqc.html')],
                sample=samples['samples'], read=reads)
    output:
        os.path.join(qc_dir, 'multiqc_report.html') 
    conda:
        "envs/multiqc.yaml"
    shell:
        '''
        #!/bin/bash
        multiqc . --outdir qc --ignore .snakemake --force
        '''

My "rule all looks like this:"
rule all:
    input:
        expand([os.path.join(analyzed_dir, '{sample}.genes.results'),
                os.path.join(qc_dir, 'multiqc_report.html')],
                sample=samples['samples'])

I don't see anything wrong with it, in fact, it is very similar to my "rule all" I used before and which worked. Can anyone see an error?
EDIT
I limited the rule to only one input, I still have the same error:
rule multiqc:
    input:
        expand(os.path.join(rseqc_dir, '{sample}.geneBodyCoverage.txt'),
                sample=samples['samples'])
    output:
        os.path.join(qc_dir, 'multiqc_report.html') 
    conda:
        "envs/multiqc.yaml"
    shell:
        '''
        #!/bin/bash
        multiqc . --outdir qc --ignore .snakemake --force
        '''

This is the ouput, you can see it read the samples correctly:
run_snakemake.sh 
/rst1/2017-0205_illuminaseq/scratch/swo-406/snakemake/lib/python3.6/site-packages/snakemake/workflow.py:12: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'.
  from collections import OrderedDict
Building DAG of jobs...
/rst1/2017-0205_illuminaseq/scratch/swo-406/snakemake/lib/python3.6/site-packages/snakemake/workflow.py:12: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'.
  from collections import OrderedDict
Building DAG of jobs...
Using shell: /bin/bash
Provided cluster nodes: 256
Job counts:
    count   jobs
    1   all
    1   multiqc
    2

[Mon Jan 14 17:46:40 2019]
rule multiqc:
    input: qc/rseqc/0067_P2018SEQE27R01_S1.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R02_S2.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R03_S3.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R04_S4.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R05_S5.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R06_S6.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R07_S7.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R08_S8.geneBodyCoverage.txt
    output: qc/multiqc_report.html
    jobid: 2

WorkflowError in line 119 of /rst1/2017-0205_illuminaseq/scratch/swo-406/test_snakemake_full/Snakefile:
'Wildcards' object has no attribute 'sample'
17:46 nlv24077@kiato /rst1/2017-0205_illuminaseq/scratch/swo-406/test_snakemake_full > ll qc/rseqc/0067_P2018SEQE27R07_S7.geneBodyCoverage.txt
-rw-rw-r-- 1 nlv24077 2017-0205_illuminaseq 303 Jan  7 16:00 qc/rseqc/0067_P2018SEQE27R07_S7.geneBodyCoverage.txt

EDIT 2:
This is how I start Snakemake:
> cat run_snakemake.sh
source activate /rst1/2017-0205_illuminaseq/scratch/swo-406/snakemake
snakemake --dag | dot -T svg > dag.svg
snakemake --cluster-config cluster.json --cluster "qsub -l nodes={cluster.nodes}:ppn={cluster.ppn} -N {wildcards.sample}_{rule}" --jobs 256 --use-conda --rerun-incomplete

EDIT 3:
Ok, that last remark (by @finswimmer) triggered me, I removed the "{wildcards.sample}_" from the above command and then it works. Or, works, I get a very different error, it is multiqc specific and solved as specified here: https://multiqc.info/docs/#locale-error-messages. When adding the two lines as suggested this error disappears and everything works as expected, even with the long inputs list.
RuntimeError: Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment. Consult https://click.palletsprojects.com/en/7.x/python3/ for mitigation steps.

Anyway, I'm guessing Snakemake does not understand how to name this task when it uses an entire list of {sample}'s... Is this a bug? What should be used? I guess it is impossible to use {sample} for naming for this rule. It's a shame I have to kill the naming for all rules because using sample_rule is very nice for keeping track.

**Tags:** snakemake
> **Comments:**
> **username_1**: Can you start by limiting the input to that rule to a couple files? That will at least help debug this.  
> **username_2**: I don't know anything about snakemake but the problem seems to come from not declaring what a `sample` should be  
> **username_1**: @llrs `sample` is defined in the `expand()` function. Oddly, there's no direct use of `wildcards` outside of that, which is usually where this sort of thing crops up.  
> **username_3**: Is `samples['samples']` defined somewhere?  
> **username_3**: And how does your command look like to start the workflow?  
> **username_2**: I think you could use it, but not that way. But I think that would be another and please post how you solved it (I want to learn to use snakemake).  
> **username_0**: Hi @llrs, the answer is in EDIT 2, there is my "start snakemake" script. I used the wildcards.sample to name the jobs on our qsub cluster, but I can't do that because my MultiQC does not have a single defined wildcards.sample (it has many). So I simply removed that part (the {wildcards.sample}) and now I only have the rules as the jobname, sadly. Does that help? How do you think I could use it then?  
> **username_2**: @Freek Well, to signal that you got an answer here (on the stackexchange.com sites) we post an answer! You can post an answer to your own question (it is completely normal and encouraged). In the answer please explain how it solved your problem as you did in this comment.  
> **username_0**: Ok, thanx, I'll remember that, for now James Hawley worded it nicely already :)  

---

## 2 answers

### Answer by username_5 on 2019-01-14. Score: 5 | Accepted answer

Snakemake gets wildcards from the parsing the input/output file names.
For example, if you had
rule sort:
    input: "{file}.bed"
    output: "{file}.sorted.bed"
    shell: "sort -k1,1 -k2,2n {input} > {output}"

then the wildcards variable would be set as wildcards = {"file": ...}.
In your multiqc rule, you have the {sample} variable within your expand(), but this variable doesn't exist outside of expand's local context. You don't actually have any wildcards there, nor in your output file (qc_dir and rseqc_dir are variables you define elsewhere, but not "wildcards" as Snakemake interprets them).
So you're getting the error 'Wildcards' object has no attribute 'sample' because there are no wildcards for that rule.
Your rule is currently set up to process every sample that you have listed in  samples['sample']. If you want your multiqc rule to work for a single sample, simply remove the sample=samples['sample'] and double brace the {sample} wildcard.
rule multiqc:
    input:
        expand([os.path.join(analyzed_dir, '{{sample}}.stat', '{{sample}}.cnt'),
                os.path.join(rseqc_dir, '{{sample}}.bam_stat.txt'),
                os.path.join(rseqc_dir, '{{sample}}.clipping_profile.xls'),
                os.path.join(rseqc_dir, '{{sample}}.deletion_profile.txt'),
                os.path.join(rseqc_dir, '{{sample}}.infer_experiment.txt'),
                os.path.join(rseqc_dir, '{{sample}}.geneBodyCoverage.txt'),
                os.path.join(rseqc_dir, '{{sample}}.inner_distance.txt'),
                os.path.join(rseqc_dir, '{{sample}}.insertion_profile.xls'),
                os.path.join(rseqc_dir, '{{sample}}.junction.xls'),
                os.path.join(rseqc_dir, '{{sample}}.junctionSaturation_plot.r'),
                os.path.join(rseqc_dir, '{{sample}}.mismatch_profile.xls'),
                os.path.join(rseqc_dir, '{{sample}}.read_distribution.txt'),
                os.path.join(rseqc_dir, '{{sample}}.pos.DupRate.xls'),
                os.path.join(rseqc_dir, '{{sample}}.seq.DupRate.xls'),
                os.path.join(rseqc_dir, '{{sample}}.GC.xls'),
                os.path.join(rseqc_dir, '{{sample}}.NVC.xls'),
                os.path.join(rseqc_dir, '{{sample}}.qual.r'),
                os.path.join(rseqc_dir, '{{sample}}.RNA_fragment_size.txt'),
                os.path.join(rseqc_dir, '{{sample}}.STAR.genome.sorted.summary.txt'),
                os.path.join(fastqc_dir, '{{sample}}_R{read}_001_fastqc.html')],
                read=reads)
    output:
        # this {sample} is in single {} and is recognized as a wildcard
        os.path.join(qc_dir, '{sample}', 'multiqc_report.html') 
    conda:
        "envs/multiqc.yaml"
    shell:
        '''
        #!/bin/bash
        # using {wildcards.sample} here since MultiQC likes directories
        multiqc . --outdir qc/{wildcards.sample} --ignore .snakemake --force
        '''

When wildcards are recognized, you'll see them in the job list.
Running snakemake 0067_P2018SEQE27R01_S1/multiqc_report.html, for example, should give you something like:
rule multiqc:
    input: qc/rseqc/0067_P2018SEQE27R01_S1.geneBodyCoverage.txt, ...
    output: qc/0067_P2018SEQE27R01_S1/multiqc_report.html
    jobid: 2
    wildcards: sample=0067_P2018SEQE27R01_S1

> **Comments:**
> **username_0**: Thank you, nicely explained, I'll accept this answer. But, do you know of any way to temporarily set the wildcards.sample variable? So that I can use it for naming my multiqc job? I find it a shame that because of this one rule I can't have the {sample}_{rule} naming scheme anymore...  
> **username_5**: Maybe you could try using multiple bash commands? Something like `multiqc . --outdir qc --ignore .snakemake --force && mv multiqc_report.html {wildcards.sample}_multiqc.html`?  
> **username_5**: But I think that because multiqc scans the entire directory, all your files are going to get scanned anyway, and end up in each report. So I don't know if they're actually going to be any different, or if you're just going to end up with the same file under multiple names  
> **username_4**: @JamesHawley, I thought the OP wanted many inputs (defined by the `expand()` call) to a single output (`qc/multiqc_report.html`), rather than your solution, which if I understand results in an output for every input. It should be possible to achieve what the OP asked for, and the [multiqc wrapper](https://snakemake-wrappers.readthedocs.io/en/stable/wrappers/multiqc.html) seems to suggest so. In which case, what was wrong with the OP's code? (I'm having the same error in a similar situation, thus my comment)  
> **username_4**: I got it now: the problem wasn't with the rule's code itself, it was to do with the call to `--cluster`. So, there is no known solution for this problem of wanting to use the `wildcards.sample` in the call to `qsub` when there is one, but otherwise not? This would be particularly useful for log files.  

---

### Answer by username_4 on 2020-01-14. Score: 0

Although @JamesHawley answer was informative, I don't think it solved the initial issue requested by the OP. (edit: or maybe it partially did, I'm not sure as I got a bit confused with the wildcards on the initial question/answer. I'm anyway leaving this answer here, in case another way of explaining it is useful for others)
Solution
I think you only need to change the output of your rule multiqc from:
os.path.join(qc_dir, 'multiqc_report.html')
to
os.path.join(qc_dir, '{sample}.html')
This seemed counter-intuitive to me initially, but I think what's happening is that snakemake will look at rule all and see that you want an output called "qc/multiqc_report.html". And the only place where it can find something of that sort is in rule multiqc, by replacing the wildcard {sample} by the string multiqc_report. 
It felt confusing to me, because the same {sample} wildcard is used in other rules (namely the rules you use to generate the outputs in your analysed_dir). But snakemake will be smart to replace {sample} with whatever it needs to recreate the output files you're requesting. 

Longer explanation
Here is a (hopefully) reproducible example, if it helps.
First create some fake directory structure with some files:
mkdir inputs
mkdir analysed_dir
mkdir qc_dir

touch inputs/file1.txt
touch inputs/file2.txt

Then, this would be the snakefile:
# Define your variables, fetch things from config, or csv, etc...
input_dir = "inputs"
output_dir = "outputs"
samples = ["file1.txt", "file2.txt"]


# Rule all defining all the output files required
rule all:
  input:
    expand("analysed_dir/{samples}", samples = samples),
    "qc_dir/multi_qc_report.html"

# this rule is equivalent to whichever rules process your data individually
rule analyse_each_file:
  input:
    "inputs/{sample}.txt"
  output:
    "analysed_dir/{sample}.txt"
  shell:
    "cat {input} > {output}"

# This rule is equivalent to your multiqc step where multiple files result in a single output
rule multiqc:
  input:
    expand("analysed_dir/{samples}", samples = samples)
  output:
    "qc_dir/{sample}.html"
  shell:
    "cat {input} > {output}"

to generate this pipeline:

And then issuing:
snakemake --cluster "qsub -N {wildcards.sample}-{rule}" --jobs 10
Generates the following job names:

file1-analyse_each_file
file2-analyse_each_file
multi_qc_report-multiqc

Edit with further explanation: 
Notice in the DAG how the sample wildcard takes different values for the different rules. I think the snakemake logic is as follows:

Starting with rule all I see that you want three files as input (the first two come from the expand() call): 


analysed_dir/file1.txt
analysed_dir/file2.txt
qc_dir/multi_qc_report.html

Let me start with the first one analysed_dir/file1.txt. The file doesn't exist, so let me look in the other rules and see if I can find it in their output. Actually in rule analyse_each_file I see the output is set to "analysed_dir/{sample}.txt", so if I replace {sample} by file1.txt I can recreate your file from it. Therefore, the input for that rule will be "inputs/file1.txt" (and this file already exists, so I can issue this job).
For the second file analysed_dir/file2.txt, well it's a similar case as with the previous one, so I'll issue a job with the same rule analyse_each_file, but this time {sample} = file2.txt.
Finally, you want a file called qc_dir/multi_qc_report.html. Where can I create such a file from? Well, the output of rule multiqc is set to "qc_dir/{sample}.html", so if I make {sample} = multi_qc_report, then I can issue the job and produce your file of interest. In this case, the rule takes as input two files "analysed_dir/file1.txt" and "analysed_dir/file2.txt", which I already know how to create from another rule. 

The key thing to realise is that snakemake defines the wildcard {sample} for each rule separately. So, in this example, wildcard.sample in rule analyse_each_file does not take on the same values as in rule multiqc.

> **Comments:**
> **username_0**: Thank you for your comment, I still don't have a satisfying solution (meaning I now just leave out the sample wildcard from the qsub job names). For your solution I wonder, will Snakemake not complain about "qc_dir/{sample}.html" never being generated? I think the core issue is that the Multiqc rule takes in all samples so it is correct that there is no single "sample" wildcard. I was just looking for a way to set the sample wildcards to something like "all_samples" for the purpose of naming the qsubbed jobs. Am I making any sense?  
> **username_4**: @Freek, when you say "I still don't have a satisfying solution", do you mean you tried my solution and it didn't work? (note, I've edited my solution to more correctly match your posted code using `os.path.join()` - see the **Solution** section at the top). I don't think `snakemake` will get confused, because it seems to me that _wildcards are defined within each rule individually_. See my further edit with expanded explanation at the end of the answer (below the DAG). I hope it helps.  

---

