<p>Be aware, that dynamic build steps could cause some problems in some build steps, e.g. when you call an other job:</p>

<pre><code>pipeline {
    stages {
        stage('Test') {
            steps {
                script {
                    def tests = [:]
                    for (f in findFiles(glob: '**/html/*.html')) {
                        // Create temp variable, otherwise the name will be the last value of the for loop
                        def name = f
                        tests["${name}"] = {
                            build job: "${name}"
                        }
                    }
                    parallel tests
                }
            }
        }       
    }
}
</code></pre>

CC BY-SA 4.0
2020-04-08T10:26:09.370Z
I don't know why but i used stage and echo inside tests, but it will giving stage name and echo as null until I used a local variable like you have used name above. Do you know why that is? Thanks for your answer though!
Kaushik
CC BY-SA 4.0
2019-08-12T10:06:36.227Z
2019-08-12T10:06:36.227Z
Christian Gripp
<p>It's much easier to use scripted Pipelines to do this since you can use arbitrary Groovy, but you should still be able to do this with declarative Pipelines using the <a href="https://jenkins.io/doc/pipeline/steps/pipeline-utility-steps/#code-findfiles-code-find-files-in-the-workspace" rel="nofollow noreferrer"><code>findFiles</code></a> step.</p>

CC BY-SA 3.0
2018-01-15T22:52:40.987Z
2018-01-15T22:52:40.987Z
jayhendren
<p>This also works, if you want to stay within the <code>Declarative Pipeline</code> space</p>

<pre><code>// declare our vars outside the pipeline
def tests = [:]
def files

pipeline {
    agent any
    stages {
        stage('1') {
            steps {
                script {
                    // we've declared the variable, now we give it the values
                    files = findFiles(glob: '**/html/*.html')
                    // Loop through them
                    files.each { f -&gt;
                        // add each object from the 'files' loop to the 'tests' array
                        tests[f] = {
                            // we're already in the script{} block, so do our advanced stuff here
                            echo f.toString()
                        }
                    }
                    // Still within the 'Script' block, run the parallel array object
                    parallel tests
                }
            }
        }       
    }
}
</code></pre>

CC BY-SA 4.0
2019-01-29T19:57:39.723Z
If you want to allocate each parallel task to different Jenkins nodes then simply wrap the actions in a `node {}` block, like this:

`tests[f] = {
    node {
        echo f.toString()
    }
}`
primetheus
CC BY-SA 4.0
2018-11-15T01:24:34.300Z
2018-12-19T21:45:09.707Z
2018-12-19T21:45:09.707Z
primetheus
<p>Here is a solution where you can setup the maximum number of concurrent tasks running at once with <code>collate(n)</code>.</p>
<pre><code>pipeline {
  stages {
    stage(&quot;Parallel&quot;) {
      steps {
        script {
          def list = [&quot;alpha&quot;, &quot;beta&quot;, &quot;gamma&quot;, &quot;delta&quot;, &quot;epsilon&quot;, &quot;zeta&quot;, &quot;eta&quot;, &quot;theta&quot;, &quot;iota&quot;, &quot;kappa&quot;]

          // Split the list into batches of 4
          def parallelBatches = list.collate(4);
          // [[&quot;alpha&quot;, &quot;beta&quot;, &quot;gamma&quot;, &quot;delta&quot;], [&quot;epsilon&quot;, &quot;zeta&quot;, &quot;eta&quot;, &quot;theta&quot;], [&quot;iota&quot;, &quot;kappa&quot;]]

          // This allows to run a maximum of 4 tasks in parallel (at once)
          for (parallelBatch in parallelBatches) {
            def tasks = [:]

            for (task in parallelBatch) {
              tasks[&quot;${task}&quot;] = {
                stage(&quot;${task}&quot;) {
                  echo '${task}'
                }
              }
            }
            parallel tasks

            // Once the batch is over, the outer loop move onto the next batch
          }
        }
      }
    }
  }
}
</code></pre>

CC BY-SA 4.0
2022-10-07T08:52:04.587Z
2022-10-07T08:52:04.587Z
shellwhale
<p>Managed to solve it with the following code:</p>

<pre><code>pipeline {
    agent { label "master"}
    stages {
        stage('1') {
            steps {
                script {
                    def tests = [:]
                    for (f in findFiles(glob: '**/html/*.html')) {
                        tests["${f}"] = {
                            node {
                                stage("${f}") {
                                    echo '${f}'
                                }
                            }
                        }
                    }
                    parallel tests
                }
            }
        }       
    }
}
</code></pre>

CC BY-SA 3.0
2018-04-10T16:15:40.517Z
Please also check out the official Pipeline examples - https://jenkins.io/doc/pipeline/examples/#parallel-multiple-nodes
phedoreanu
CC BY-SA 3.0
2018-04-10T16:24:56.657Z
@phedoreanu i'm using declarative pipeline...
thclpr
CC BY-SA 3.0
2018-04-16T09:38:54.890Z
@phedoreanu I have rejected your edit, editing code should have good reasons, your comment isn't enough for me to allow an edit of this kind on an answer which was a self solution. I think you should have commented to discuss the matter with the answer author before doing this edit.
Tensibai
CC BY-SA 3.0
2018-04-16T09:40:48.103Z
@phedoreanu I you think you have a better derivative work, then please write your own answer and explain why it is better (in error handling, templates, etc.) instead.
Tensibai
CC BY-SA 4.0
2019-04-11T10:13:41.723Z
Hi, I figured out the same after a few failing attempts. My only problem right now is that if I put two stage{..} sections inside a node for some reasons the workflow stage chart and Blu Ocean get confused. For instance in the workflow stage chart I get NaNy NaNd and in Blue Ocean I get only the first stage.
Giuseppe Salvatore
CC BY-SA 4.0
2022-10-06T11:49:10.070Z
How would you limit this for, let's say 10 concurrents builds running at once ?
shellwhale
CC BY-SA 3.0
2018-01-16T23:45:09.720Z
2018-04-16T09:41:07.233Z
2018-04-16T09:41:07.233Z
thclpr
<p>Currently, I'm going to need an implementation that must find all files within a directory and start a parallel task for every file found.</p>

<p>Is it possible to achieve this using declarative pipelines?</p>

<pre><code>pipeline {
    agent any
    stages {
        stage("test") {
            steps {
                dir ("file_path") {
                    // find all files with complete path
                    parallel (
                        // execute parallel tasks for each file found.
                        // this must be dynamic
                        }
                    }
                }
            }
        }
    }
}
</code></pre>

CC BY-SA 4.0
2019-02-27T18:06:54.747Z
How can I do if i want to run multiple steps sequentaly and not parallel?
Frank Escobar
CC BY-SA 4.0
2020-01-29T12:00:57.053Z
Sure, but that way you cannot generate the parallel tasks dynamically, for example, depending on some files on the repository.
Raúl Salinas-Monteagudo
<jenkins><jenkinsfile>
How to properly achieve dynamic parallel action with a declarative pipeline?
thclpr
# How to properly achieve dynamic parallel action with a declarative pipeline?

(Asked by: username_0 on 2018-01-12)

Currently, I'm going to need an implementation that must find all files within a directory and start a parallel task for every file found.
Is it possible to achieve this using declarative pipelines?
pipeline {
    agent any
    stages {
        stage("test") {
            steps {
                dir ("file_path") {
                    // find all files with complete path
                    parallel (
                        // execute parallel tasks for each file found.
                        // this must be dynamic
                        }
                    }
                }
            }
        }
    }
}

**Tags:** jenkins, jenkinsfile

> **Comments:**
> **username_1**: How can I do if i want to run multiple steps sequentaly and not parallel?  
> **username_2**: Sure, but that way you cannot generate the parallel tasks dynamically, for example, depending on some files on the repository.  

---

## 5 answers

### Answer by username_0 on 2018-01-16. Score: 40 | Accepted answer

Managed to solve it with the following code:
pipeline {
    agent { label "master"}
    stages {
        stage('1') {
            steps {
                script {
                    def tests = [:]
                    for (f in findFiles(glob: '**/html/*.html')) {
                        tests["${f}"] = {
                            node {
                                stage("${f}") {
                                    echo '${f}'
                                }
                            }
                        }
                    }
                    parallel tests
                }
            }
        }       
    }
}

> **Comments:**
> **username_8**: Please also check out the official Pipeline examples - https://jenkins.io/doc/pipeline/examples/#parallel-multiple-nodes  
> **username_0**: @username_8 i'm using declarative pipeline...  
> **username_9**: @username_8 I have rejected your edit, editing code should have good reasons, your comment isn't enough for me to allow an edit of this kind on an answer which was a self solution. I think you should have commented to discuss the matter with the answer author before doing this edit.  
> **username_9**: @username_8 I you think you have a better derivative work, then please write your own answer and explain why it is better (in error handling, templates, etc.) instead.  
> **username_10**: Hi, I figured out the same after a few failing attempts. My only problem right now is that if I put two stage{..} sections inside a node for some reasons the workflow stage chart and Blu Ocean get confused. For instance in the workflow stage chart I get NaNy NaNd and in Blue Ocean I get only the first stage.  
> **username_7**: How would you limit this for, let's say 10 concurrents builds running at once ?  

---

### Answer by username_6 on 2018-11-15. Score: 15

This also works, if you want to stay within the Declarative Pipeline space
// declare our vars outside the pipeline
def tests = [:]
def files

pipeline {
    agent any
    stages {
        stage('1') {
            steps {
                script {
                    // we've declared the variable, now we give it the values
                    files = findFiles(glob: '**/html/*.html')
                    // Loop through them
                    files.each { f ->
                        // add each object from the 'files' loop to the 'tests' array
                        tests[f] = {
                            // we're already in the script{} block, so do our advanced stuff here
                            echo f.toString()
                        }
                    }
                    // Still within the 'Script' block, run the parallel array object
                    parallel tests
                }
            }
        }       
    }
}

> **Comments:**
> **username_6**: If you want to allocate each parallel task to different Jenkins nodes then simply wrap the actions in a `node {}` block, like this:  `tests[f] = {     node {         echo f.toString()     } }`  

---

### Answer by username_3 on 2019-08-12. Score: 7

Be aware, that dynamic build steps could cause some problems in some build steps, e.g. when you call an other job:
pipeline {
    stages {
        stage('Test') {
            steps {
                script {
                    def tests = [:]
                    for (f in findFiles(glob: '**/html/*.html')) {
                        // Create temp variable, otherwise the name will be the last value of the for loop
                        def name = f
                        tests["${name}"] = {
                            build job: "${name}"
                        }
                    }
                    parallel tests
                }
            }
        }       
    }
}

> **Comments:**
> **username_4**: I don't know why but i used stage and echo inside tests, but it will giving stage name and echo as null until I used a local variable like you have used name above. Do you know why that is? Thanks for your answer though!  

---

### Answer by username_5 on 2018-01-15. Score: 2

It's much easier to use scripted Pipelines to do this since you can use arbitrary Groovy, but you should still be able to do this with declarative Pipelines using the [`findFiles`](https://jenkins.io/doc/pipeline/steps/pipeline-utility-steps/#code-findfiles-code-find-files-in-the-workspace) step.


---

### Answer by username_7 on 2022-10-07. Score: 1

Here is a solution where you can setup the maximum number of concurrent tasks running at once with collate(n).
pipeline {
  stages {
    stage("Parallel") {
      steps {
        script {
          def list = ["alpha", "beta", "gamma", "delta", "epsilon", "zeta", "eta", "theta", "iota", "kappa"]

          // Split the list into batches of 4
          def parallelBatches = list.collate(4);
          // [["alpha", "beta", "gamma", "delta"], ["epsilon", "zeta", "eta", "theta"], ["iota", "kappa"]]

          // This allows to run a maximum of 4 tasks in parallel (at once)
          for (parallelBatch in parallelBatches) {
            def tasks = [:]

            for (task in parallelBatch) {
              tasks["${task}"] = {
                stage("${task}") {
                  echo '${task}'
                }
              }
            }
            parallel tasks

            // Once the batch is over, the outer loop move onto the next batch
          }
        }
      }
    }
  }
}


---

