<p>Here's an example I wrote recently:</p>

<pre class="lang-java prettyprint-override"><code>#!/usr/bin/env nextflow

params.requestedCPU = 80
params.requestedMem = 50

maxcpus = Runtime.runtime.availableProcessors()
requestedCpus = Math.round((params.requestedCPU * maxcpus) / 100)

maxmem = Runtime.runtime.totalMemory() / 10241024
requestedMem = (Math.round((params.requestedMem * maxmem) / 100))

process foo {
    echo true
    cpus = requestedCpus
    memory = requestedMem.GB

    script:
        """
        echo <span class="math-container">$requestedCpus
 echo $</span>requestedMem
        """
}
</code></pre>

<p>You can pass the number of CPUs or Memory directly on the command line in percentage, e.g. <code>nextflow run main.nf --requestedCPU 100</code> to have all CPU usages. Hope this helps.</p>

CC BY-SA 4.0
2020-04-08T00:54:31.753Z
2020-04-08T00:54:31.753Z
user324810
<p>I have not read the full Nextflow manual in depth, but I think there is no Nextflow command to detect the maximum number of threads you can use.</p>

<p>However, you can try to automatize this task by using the <code>nproc</code> command.</p>

<p><code>nproc</code> gives you the number of processing units available. An idea would be to include something such <code>nproc - 1</code> as the max number of threads for Roary.  </p>

CC BY-SA 4.0
2019-05-29T14:46:02.450Z
Thanks, as far I could see there are ways to dynamically allocate resources (https://www.nextflow.io/docs/latest/process.html#dynamic-computing-resources) but not detect threads, so this is a nice workaround.
TW93
CC BY-SA 4.0
2019-05-29T12:20:39.397Z
2019-05-29T13:57:42.187Z
2019-05-29T13:57:42.187Z
jxx_fa
<p>I am creating a pipeline in Nextflow. One step is creating a pangenome with Roary. Roary takes threads as an argument and if no number of threads is supplied as an argument it defaults to one. </p>

<p>Is there a way in Nextflow to pass the maximum number of available threads as an argument? At the moment the best I can come up with is defining an arbitrarily large number of threads as an argument which will cap out and run with the max number of threads. I would like to know, is there a 'cleaner' or more canonical way of doing this?</p>

<p>At the moment my process looks like this:</p>

<pre><code>
process roary {

    publishDir "${params.outdir}/roary", mode: 'copy'

    cpus 200

    input:
    file gff from gff.collect()

    output:
    file("*") into roary
    file("pan_genome_reference.fa") into pan_genome
    file("pan_genome_sequences/*") into alignment_files
    file("gene_presence_absence.Rtab") into gene_presence
    set file("*accessory*"),
        file("*.Rtab"),
        file("_*"),
        file("*.txt"),
        file("*csv") into roary_out


    script:
    """
    roary -p <span class="math-container">${task.cpus} -e -n -v -z $</span>gff
    """
}
</code></pre>

<p>Thanks in advance. Also, apologies as I can see why this might not strictly be bioinformatics, but it's where Nextflow has the most traction.</p>

CC BY-SA 4.0
2019-05-29T13:01:21.097Z
Workflow management has a firm foothold in bioinformatics and is in my opinion directly on topic on this site.
Daniel Standage
<workflow-management><nextflow>
Autodetect max number of cores and pass as an argument in Nextflow
# Autodetect max number of cores and pass as an argument in Nextflow

(Asked by: username_0 on 2019-05-29)

I am creating a pipeline in Nextflow. One step is creating a pangenome with Roary. Roary takes threads as an argument and if no number of threads is supplied as an argument it defaults to one. 
Is there a way in Nextflow to pass the maximum number of available threads as an argument? At the moment the best I can come up with is defining an arbitrarily large number of threads as an argument which will cap out and run with the max number of threads. I would like to know, is there a 'cleaner' or more canonical way of doing this?
At the moment my process looks like this:

process roary {

    publishDir "${params.outdir}/roary", mode: 'copy'

    cpus 200

    input:
    file gff from gff.collect()

    output:
    file("*") into roary
    file("pan_genome_reference.fa") into pan_genome
    file("pan_genome_sequences/*") into alignment_files
    file("gene_presence_absence.Rtab") into gene_presence
    set file("*accessory*"),
        file("*.Rtab"),
        file("_*"),
        file("*.txt"),
        file("*csv") into roary_out


    script:
    """
    roary -p ${task.cpus} -e -n -v -z $gff
    """
}

Thanks in advance. Also, apologies as I can see why this might not strictly be bioinformatics, but it's where Nextflow has the most traction.

**Tags:** workflow-management, nextflow
> **Comments:**
> **username_1**: Workflow management has a firm foothold in bioinformatics and is in my opinion directly on topic on this site.  

---

## 2 answers

### Answer by username_3 on 2019-05-29. Score: 6 | Accepted answer

I have not read the full Nextflow manual in depth, but I think there is no Nextflow command to detect the maximum number of threads you can use.

However, you can try to automatize this task by using the `nproc` command.

`nproc` gives you the number of processing units available. An idea would be to include something such `nproc - 1` as the max number of threads for Roary.

> **Comments:**
> **username_0**: Thanks, as far I could see there are ways to dynamically allocate resources (https://www.nextflow.io/docs/latest/process.html#dynamic-computing-resources) but not detect threads, so this is a nice workaround.  

---

### Answer by username_2 on 2020-04-08. Score: 3

Here's an example I wrote recently:
#!/usr/bin/env nextflow

params.requestedCPU = 80
params.requestedMem = 50

maxcpus = Runtime.runtime.availableProcessors()
requestedCpus = Math.round((params.requestedCPU * maxcpus) / 100)

maxmem = Runtime.runtime.totalMemory() / 10241024
requestedMem = (Math.round((params.requestedMem * maxmem) / 100))

process foo {
    echo true
    cpus = requestedCpus
    memory = requestedMem.GB

    script:
        """
        echo $requestedCpus
 echo $requestedMem
        """
}

You can pass the number of CPUs or Memory directly on the command line in percentage, e.g. nextflow run main.nf --requestedCPU 100 to have all CPU usages. Hope this helps.


---

