<p>You don't need to create processes in a loop when you have channels. The <a href="https://www.nextflow.io/docs/latest/process.html#input-repeaters" rel="nofollow noreferrer">each qualifier</a> allows you to repeat the execution of a process for each item in a collection. For example, the contents of 'test.nf':</p>
<pre><code>some_inputs = Channel.of( 'prot', 'dna', 'rna' )
some_parameters = Channel.of( 5, 3, 2, 7, 1 )

process foo {

    tag { &quot;task: <span class="math-container">${task.index} - $</span>{x},${y}&quot; }

    input:
    val x from some_inputs
    each y from some_parameters

    output:
    tuple val(x), val(y) into receivers

    &quot;&quot;&quot;
    echo &quot;x: <span class="math-container">${x}; y: $</span>{y}&quot;
    &quot;&quot;&quot;
}

results = receivers.branch { x, y -&gt;
    
    five: y == 5
        return tuple( x, y )

    three: y == 3
        return tuple( x, y )

    two: y == 2
        return tuple( x, y )

    seven: y == 7
        return tuple( x, y )
    
    one: y == 1
        return tuple( x, y )
}

results.five.view { &quot;My result for param 5 is: <span class="math-container">$it" }
results.three.view { "My result for param 3 is: $it" }
results.two.view { "My result for param 2 is: $it" }
results.seven.view { "My result for param 7 is: $it" }
results.one.view { "My result for param 1 is: $</span>it&quot; }
</code></pre>
<p>Then running like:</p>
<pre><code>nextflow run -ansi-log false test.nf
</code></pre>
<p>Results:</p>
<pre><code>N E X T F L O W  ~  version 20.10.0
Launching `test.nf` [fabulous_franklin] - revision: 14c3290d2b
[6e/c12ce7] Submitted process &gt; foo (task: 7 - dna,3)
[93/616f93] Submitted process &gt; foo (task: 4 - prot,7)
[27/0bc3c4] Submitted process &gt; foo (task: 3 - prot,2)
[0d/0d0b51] Submitted process &gt; foo (task: 9 - dna,7)
[d4/f56d18] Submitted process &gt; foo (task: 6 - dna,5)
[b0/d1535f] Submitted process &gt; foo (task: 8 - dna,2)
[13/0a660a] Submitted process &gt; foo (task: 5 - prot,1)
[f0/1c2f57] Submitted process &gt; foo (task: 2 - prot,3)
[3f/3e893d] Submitted process &gt; foo (task: 1 - prot,5)
[19/2921de] Submitted process &gt; foo (task: 10 - dna,1)
My result for param 3 is: [dna, 3]
My result for param 7 is: [prot, 7]
My result for param 7 is: [dna, 7]
[a7/953a2b] Submitted process &gt; foo (task: 11 - rna,3)
[1c/0fd4ef] Submitted process &gt; foo (task: 12 - rna,5)
My result for param 1 is: [prot, 1]
[39/d21adf] Submitted process &gt; foo (task: 14 - rna,7)
My result for param 5 is: [dna, 5]
My result for param 2 is: [prot, 2]
My result for param 2 is: [dna, 2]
[f6/8509a7] Submitted process &gt; foo (task: 15 - rna,1)
My result for param 5 is: [prot, 5]
[6f/cee0ae] Submitted process &gt; foo (task: 13 - rna,2)
My result for param 3 is: [rna, 3]
My result for param 1 is: [dna, 1]
My result for param 2 is: [rna, 2]
My result for param 3 is: [prot, 3]
My result for param 1 is: [rna, 1]
My result for param 5 is: [rna, 5]
My result for param 7 is: [rna, 7]
</code></pre>
<p>The answer to your bonus question is to use the <code>task.index</code> variable in your script. I used this to <a href="https://www.nextflow.io/docs/latest/process.html#tag" rel="nofollow noreferrer">tag</a> your job, but it can be used anywhere in the process declaration. You likely won't need this variable though.</p>
<p>Usually, using the <a href="https://www.nextflow.io/docs/latest/operator.html#branch" rel="nofollow noreferrer">branch operator</a> is enough for most use cases. However,it's possible to implement your own branching function. The following uses the new DSL, DSL2, to automatically fork the 'receivers' channel:</p>
<pre><code>nextflow.enable.dsl=2

process foo {

    tag { &quot;task: <span class="math-container">${task.index} - $</span>{x},${y}&quot; }

    input:
    val x
    each y

    output:
    tuple val(x), val(y)

    &quot;&quot;&quot;
    echo &quot;x: <span class="math-container">${x}; y: $</span>{y}&quot;
    &quot;&quot;&quot;
}

def branchByElement( input_ch, elements, by=0 ) {

    def targets = [:]
    for( element in elements ) {
        def key = element
        targets[ key ] = { input_ch.filter { it[by] == key } }
    }
    return targets
}

workflow {

    some_inputs = [ 'prot', 'dna', 'rna' ]
    some_parameters = [ 5, 3, 2, 7, 1 ]

    inputs_ch = Channel.fromList( some_inputs )
    parameters_ch = Channel.fromList( some_parameters )

    foo( inputs_ch, parameters_ch )

    parameter_results = branchByElement( foo.out, some_parameters, by=1 )
    input_results = branchByElement( foo.out, some_inputs, by=0 )

    parameter_results[7].call().view { &quot;Param 7: <span class="math-container">$it" }
    input_results.dna().view { "Input dna: $</span>it&quot; }
}
</code></pre>
<p>Results:</p>
<pre><code>N E X T F L O W  ~  version 20.10.0
Launching `test.nf` [distraught_meninsky] - revision: 04df251752
[bb/32325f] Submitted process &gt; foo (task: 4 - prot,7)
[a0/19fdf5] Submitted process &gt; foo (task: 8 - dna,2)
[e9/4c1d09] Submitted process &gt; foo (task: 6 - dna,5)
[a3/dcb002] Submitted process &gt; foo (task: 2 - prot,3)
[db/7e612e] Submitted process &gt; foo (task: 5 - prot,1)
[cd/dfa84e] Submitted process &gt; foo (task: 9 - dna,7)
[d4/6a85f2] Submitted process &gt; foo (task: 7 - dna,3)
[f0/8e8bf4] Submitted process &gt; foo (task: 1 - prot,5)
[89/f20a28] Submitted process &gt; foo (task: 3 - prot,2)
Input dna: [dna, 2]
Param 7: [prot, 7]
Input dna: [dna, 5]
[53/056fe4] Submitted process &gt; foo (task: 10 - dna,1)
[f3/a61abf] Submitted process &gt; foo (task: 12 - rna,3)
Input dna: [dna, 7]
[f9/a568a8] Submitted process &gt; foo (task: 13 - rna,2)
Param 7: [dna, 7]
[06/72bb6c] Submitted process &gt; foo (task: 11 - rna,5)
Input dna: [dna, 3]
[a3/6d80d8] Submitted process &gt; foo (task: 14 - rna,7)
[ac/acdb5a] Submitted process &gt; foo (task: 15 - rna,1)
Param 7: [rna, 7]
Input dna: [dna, 1]
</code></pre>

CC BY-SA 4.0
2021-04-13T14:51:15.133Z
I think `each` is even more elegant than combine, do you also have a more elegant way to split the resulting channel than `branch` ?
Pallie
CC BY-SA 4.0
2021-04-14T14:02:01.260Z
@Pallie I think there is using a function and the new DSL2 - please see my update above.
Steve
CC BY-SA 4.0
2021-04-13T14:28:38.817Z
2021-04-14T14:00:36.763Z
2021-04-14T14:00:36.763Z
Steve
<p>Use channels, this is what nextflow's entire design is based on. I'm honestly surprised that spawning 5 processes with the same name in a loop even works.</p>
<p>Turn your parameters into regular channels, use the <code>combine</code> operator (<a href="https://www.nextflow.io/docs/latest/operator.html#combine" rel="nofollow noreferrer">https://www.nextflow.io/docs/latest/operator.html#combine</a>) to transform the 2 parameter channels into a channel that has each combination of params/inputs. Feed this channel into the  <code>foo</code>  process. Afterward use <code>branch</code> (<a href="https://www.nextflow.io/docs/latest/operator.html#branch" rel="nofollow noreferrer">https://www.nextflow.io/docs/latest/operator.html#branch</a>) to split your <code>foo</code> output channel into 5 different channels.</p>
<pre><code>typechan = Channel.from(5,3,2,7,1)
numchan = Channel.from('prot','dna', 'rna')

cartesian = numchan.combine(typechan)

process foo {
  input:
  val numtype from cartesian
  output:
  val numtype into bar 
  script:
  &quot;&quot;&quot;
  echo $numtype
  &quot;&quot;&quot;
}

bar.branch {
  one: it[1] == 1
  two: it[1] == 2
  three: it[1] == 3 
  five: it[1] == 5
  seven : it[1] == 7
}.set {result}
result.one.view()
<span class="math-container">```</span>
</code></pre>

CC BY-SA 4.0
2021-04-13T11:25:08.340Z
2021-04-13T11:25:08.340Z
Pallie
<p>I have set of ~5 processes which perform very similar tasks such that their differences can be parameterized. Rather than putting 5 items through some channel (which seems complicated), or defining 5 separate processes, it appears to be possible in Nextflow to declare the processes in a for loop, and output into an array of channels.</p>
<p>Below is a prototypical example and it seems to work as intended. (I understand this could be done differently with some channel cross product operator but I am just giving this as an example of a process in a for loop.)</p>
<p>Is there anything wrong with doing this? Is there a better way to do it?</p>
<p>Bonus question - the name of the process below is &quot;foo&quot;. I would like to have the index reflected in the name i.e. &quot;foo_$i&quot; but this doesn't seem possible by default - any way to do this??</p>
<pre><code>some_inputs = ['prot','dna', 'rna']

some_parameters = [5,3,2,7,1]

// Set up an array of channels to output to
receivers = new ArrayList&lt;Channel&gt;()
for (i = 0; i &lt; 5; i++) {
    receivers.add(Channel.create())
}

// Iterate 5 times to create 5 processes each with different parameters
for (i = 0; i &lt; 5; i++) {

    // WOW - process can be defined inside a for loop!!
    process &quot;foo&quot; {

      // Extract parameters here so correct $i value is used
      p = some_parameters[i]

      input:
      val x from Channel.from(some_inputs)

      output:
      val &quot;<span class="math-container">$x $</span>p&quot; into receivers[i] // Note that correct index $i seems to be used here

      &quot;&quot;&quot;
      &quot;&quot;&quot;
    }
}

// Confirm that each of the processes outputted independently with the correct parameter
receivers[0].view { &quot;Received[0]: <span class="math-container">$it" }
receivers[1].view { "Received[1]: $it" }
receivers[2].view { "Received[2]: $it" }
receivers[3].view { "Received[3]: $it" }
receivers[4].view { "Received[4]: $</span>it&quot; }
</code></pre>
<p>Output - note that the parameter is consistent within each input channel i.e. working correctly:</p>
<pre><code>N E X T F L O W  ~  version 20.10.0
Launching `play.nf` [fabulous_keller] - revision: c89e2b2281
executor &gt;  local (15)
[10/b477ec] process &gt; foo (3) [100%] 3 of 3 ✔
[16/8282ba] process &gt; foo (2) [100%] 3 of 3 ✔
[db/f7327e] process &gt; foo (1) [100%] 3 of 3 ✔
[fb/886f6f] process &gt; foo (1) [100%] 3 of 3 ✔
[46/c34ed3] process &gt; foo (1) [100%] 3 of 3 ✔
Received[1]: rna 3
Received[1]: prot 3
Received[4]: rna 1
Received[2]: rna 2
Received[0]: dna 5
Received[2]: dna 2
Received[2]: prot 2
Received[0]: prot 5
Received[3]: rna 7
Received[0]: rna 5
Received[1]: dna 3
Received[3]: dna 7
Received[4]: dna 1
Received[3]: prot 7
Received[4]: prot 1
</code></pre>

<nextflow>
Nextflow: Can I parameterize similar processes inside a for loop?
# Nextflow: Can I parameterize similar processes inside a for loop?

(Asked by: username_0 on 2021-04-13)

I have set of ~5 processes which perform very similar tasks such that their differences can be parameterized. Rather than putting 5 items through some channel (which seems complicated), or defining 5 separate processes, it appears to be possible in Nextflow to declare the processes in a for loop, and output into an array of channels.
Below is a prototypical example and it seems to work as intended. (I understand this could be done differently with some channel cross product operator but I am just giving this as an example of a process in a for loop.)
Is there anything wrong with doing this? Is there a better way to do it?
Bonus question - the name of the process below is "foo". I would like to have the index reflected in the name i.e. "foo_$i" but this doesn't seem possible by default - any way to do this??
some_inputs = ['prot','dna', 'rna']

some_parameters = [5,3,2,7,1]

// Set up an array of channels to output to
receivers = new ArrayList()
for (i = 0; i < 5; i++) {
    receivers.add(Channel.create())
}

// Iterate 5 times to create 5 processes each with different parameters
for (i = 0; i < 5; i++) {

    // WOW - process can be defined inside a for loop!!
    process "foo" {

      // Extract parameters here so correct $i value is used
      p = some_parameters[i]

      input:
      val x from Channel.from(some_inputs)

      output:
      val "$x $p" into receivers[i] // Note that correct index $i seems to be used here

      """
      """
    }
}

// Confirm that each of the processes outputted independently with the correct parameter
receivers[0].view { "Received[0]: $it" }
receivers[1].view { "Received[1]: $it" }
receivers[2].view { "Received[2]: $it" }
receivers[3].view { "Received[3]: $it" }
receivers[4].view { "Received[4]: $it" }

Output - note that the parameter is consistent within each input channel i.e. working correctly:
N E X T F L O W  ~  version 20.10.0
Launching `play.nf` [fabulous_keller] - revision: c89e2b2281
executor >  local (15)
[10/b477ec] process > foo (3) [100%] 3 of 3 ✔
[16/8282ba] process > foo (2) [100%] 3 of 3 ✔
[db/f7327e] process > foo (1) [100%] 3 of 3 ✔
[fb/886f6f] process > foo (1) [100%] 3 of 3 ✔
[46/c34ed3] process > foo (1) [100%] 3 of 3 ✔
Received[1]: rna 3
Received[1]: prot 3
Received[4]: rna 1
Received[2]: rna 2
Received[0]: dna 5
Received[2]: dna 2
Received[2]: prot 2
Received[0]: prot 5
Received[3]: rna 7
Received[0]: rna 5
Received[1]: dna 3
Received[3]: dna 7
Received[4]: dna 1
Received[3]: prot 7
Received[4]: prot 1

**Tags:** nextflow

---

## 2 answers

### Answer by username_2 on 2021-04-13. Score: 4 | Accepted answer

Use channels, this is what nextflow's entire design is based on. I'm honestly surprised that spawning 5 processes with the same name in a loop even works.
Turn your parameters into regular channels, use the combine operator (https://www.nextflow.io/docs/latest/operator.html#combine) to transform the 2 parameter channels into a channel that has each combination of params/inputs. Feed this channel into the  foo  process. Afterward use branch (https://www.nextflow.io/docs/latest/operator.html#branch) to split your foo output channel into 5 different channels.
typechan = Channel.from(5,3,2,7,1)
numchan = Channel.from('prot','dna', 'rna')

cartesian = numchan.combine(typechan)

process foo {
  input:
  val numtype from cartesian
  output:
  val numtype into bar 
  script:
  """
  echo $numtype
  """
}

bar.branch {
  one: it[1] == 1
  two: it[1] == 2
  three: it[1] == 3 
  five: it[1] == 5
  seven : it[1] == 7
}.set {result}
result.one.view()
```


---

### Answer by username_1 on 2021-04-13. Score: 3

You don't need to create processes in a loop when you have channels. The each qualifier allows you to repeat the execution of a process for each item in a collection. For example, the contents of 'test.nf':
some_inputs = Channel.of( 'prot', 'dna', 'rna' )
some_parameters = Channel.of( 5, 3, 2, 7, 1 )

process foo {

    tag { "task: ${task.index} - ${x},${y}" }

    input:
    val x from some_inputs
    each y from some_parameters

    output:
    tuple val(x), val(y) into receivers

    """
    echo "x: ${x}; y: ${y}"
    """
}

results = receivers.branch { x, y ->
    
    five: y == 5
        return tuple( x, y )

    three: y == 3
        return tuple( x, y )

    two: y == 2
        return tuple( x, y )

    seven: y == 7
        return tuple( x, y )
    
    one: y == 1
        return tuple( x, y )
}

results.five.view { "My result for param 5 is: $it" }
results.three.view { "My result for param 3 is: $it" }
results.two.view { "My result for param 2 is: $it" }
results.seven.view { "My result for param 7 is: $it" }
results.one.view { "My result for param 1 is: $it" }

Then running like:
nextflow run -ansi-log false test.nf

Results:
N E X T F L O W  ~  version 20.10.0
Launching `test.nf` [fabulous_franklin] - revision: 14c3290d2b
[6e/c12ce7] Submitted process > foo (task: 7 - dna,3)
[93/616f93] Submitted process > foo (task: 4 - prot,7)
[27/0bc3c4] Submitted process > foo (task: 3 - prot,2)
[0d/0d0b51] Submitted process > foo (task: 9 - dna,7)
[d4/f56d18] Submitted process > foo (task: 6 - dna,5)
[b0/d1535f] Submitted process > foo (task: 8 - dna,2)
[13/0a660a] Submitted process > foo (task: 5 - prot,1)
[f0/1c2f57] Submitted process > foo (task: 2 - prot,3)
[3f/3e893d] Submitted process > foo (task: 1 - prot,5)
[19/2921de] Submitted process > foo (task: 10 - dna,1)
My result for param 3 is: [dna, 3]
My result for param 7 is: [prot, 7]
My result for param 7 is: [dna, 7]
[a7/953a2b] Submitted process > foo (task: 11 - rna,3)
[1c/0fd4ef] Submitted process > foo (task: 12 - rna,5)
My result for param 1 is: [prot, 1]
[39/d21adf] Submitted process > foo (task: 14 - rna,7)
My result for param 5 is: [dna, 5]
My result for param 2 is: [prot, 2]
My result for param 2 is: [dna, 2]
[f6/8509a7] Submitted process > foo (task: 15 - rna,1)
My result for param 5 is: [prot, 5]
[6f/cee0ae] Submitted process > foo (task: 13 - rna,2)
My result for param 3 is: [rna, 3]
My result for param 1 is: [dna, 1]
My result for param 2 is: [rna, 2]
My result for param 3 is: [prot, 3]
My result for param 1 is: [rna, 1]
My result for param 5 is: [rna, 5]
My result for param 7 is: [rna, 7]

The answer to your bonus question is to use the task.index variable in your script. I used this to tag your job, but it can be used anywhere in the process declaration. You likely won't need this variable though.
Usually, using the branch operator is enough for most use cases. However,it's possible to implement your own branching function. The following uses the new DSL, DSL2, to automatically fork the 'receivers' channel:
nextflow.enable.dsl=2

process foo {

    tag { "task: ${task.index} - ${x},${y}" }

    input:
    val x
    each y

    output:
    tuple val(x), val(y)

    """
    echo "x: ${x}; y: ${y}"
    """
}

def branchByElement( input_ch, elements, by=0 ) {

    def targets = [:]
    for( element in elements ) {
        def key = element
        targets[ key ] = { input_ch.filter { it[by] == key } }
    }
    return targets
}

workflow {

    some_inputs = [ 'prot', 'dna', 'rna' ]
    some_parameters = [ 5, 3, 2, 7, 1 ]

    inputs_ch = Channel.fromList( some_inputs )
    parameters_ch = Channel.fromList( some_parameters )

    foo( inputs_ch, parameters_ch )

    parameter_results = branchByElement( foo.out, some_parameters, by=1 )
    input_results = branchByElement( foo.out, some_inputs, by=0 )

    parameter_results[7].call().view { "Param 7: $it" }
    input_results.dna().view { "Input dna: $it" }
}

Results:
N E X T F L O W  ~  version 20.10.0
Launching `test.nf` [distraught_meninsky] - revision: 04df251752
[bb/32325f] Submitted process > foo (task: 4 - prot,7)
[a0/19fdf5] Submitted process > foo (task: 8 - dna,2)
[e9/4c1d09] Submitted process > foo (task: 6 - dna,5)
[a3/dcb002] Submitted process > foo (task: 2 - prot,3)
[db/7e612e] Submitted process > foo (task: 5 - prot,1)
[cd/dfa84e] Submitted process > foo (task: 9 - dna,7)
[d4/6a85f2] Submitted process > foo (task: 7 - dna,3)
[f0/8e8bf4] Submitted process > foo (task: 1 - prot,5)
[89/f20a28] Submitted process > foo (task: 3 - prot,2)
Input dna: [dna, 2]
Param 7: [prot, 7]
Input dna: [dna, 5]
[53/056fe4] Submitted process > foo (task: 10 - dna,1)
[f3/a61abf] Submitted process > foo (task: 12 - rna,3)
Input dna: [dna, 7]
[f9/a568a8] Submitted process > foo (task: 13 - rna,2)
Param 7: [dna, 7]
[06/72bb6c] Submitted process > foo (task: 11 - rna,5)
Input dna: [dna, 3]
[a3/6d80d8] Submitted process > foo (task: 14 - rna,7)
[ac/acdb5a] Submitted process > foo (task: 15 - rna,1)
Param 7: [rna, 7]
Input dna: [dna, 1]

> **Comments:**
> **username_2**: I think `each` is even more elegant than combine, do you also have a more elegant way to split the resulting channel than `branch` ?  
> **username_1**: @Pallie I think there is using a function and the new DSL2 - please see my update above.  

---

