<p>The reason you can't use <code>dotenv</code> variables in the <code>rules</code> clause(s) is that <code>rules</code> are evaluated before the pipeline even starts executing.</p>
<p>If you have a <code>rules</code> clause like</p>
<pre class="lang-yml prettyprint-override"><code>rules:
  - if: $CI_COMMIT_TAG === 'v1.0.0'
  when: never
  - when: always
</code></pre>
<p>Gitlab CI will check the following things:</p>
<ol>
<li>Is the pipeline a <code>Tag Pipeline</code> (from the <code>CI_PIPELINE_SOURCE</code> variable)? If no, <code>CI_COMMIT_TAG</code> won't exist, so the clause doesn't exist, and <code>when:always</code> is the only rule. Add the job to the pipeline.</li>
<li>If the pipeline is a tag pipeline, is the tag === 'v1.0.0'? Then use <code>when:never</code>. Don't add the job to the pipeline.</li>
<li>Tag pipeline and the tag !== 'v1.0.0', use <code>when:never</code> and add the job to the pipeline.</li>
</ol>
<p>All these things happen before the pipeline is in the <code>created</code> state, so therefore it cannot depend on a variable that is created once the pipeline starts running.</p>
<p>Knowing this, the way I would solve the problem is to mark the job in Stage B as a manual job like it already is, but when you want to run it automatically, call the Gitlab Jobs API from the Stage A job to run the Stage B job.</p>
<p>This will use a couple of API Calls:</p>
<ol>
<li>First, we need to get the jobs for this particular Pipeline:</li>
</ol>
<pre class="lang-bash prettyprint-override"><code>curl --header &quot;PRIVATE-TOKEN: &lt;your_access_token&gt;&quot; &quot;https://gitlab.example.com/api/v4/projects/$CI_PROJECT_ID/pipelines/$CI_PIPELINE_IID/jobs?scope=manual&quot;
</code></pre>
<p>This will return all the <code>manual</code> jobs from the pipeline. If you have more than one, you can retrieve the correct one by looking at the value for <code>stage</code> and <code>name</code> from the response. Here's an example from the docs:</p>
<pre class="lang-json prettyprint-override"><code>[
  {
    &quot;commit&quot;: {
      &quot;author_email&quot;: &quot;admin@example.com&quot;,
      &quot;author_name&quot;: &quot;Administrator&quot;,
      &quot;created_at&quot;: &quot;2015-12-24T16:51:14.000+01:00&quot;,
      &quot;id&quot;: &quot;0ff3ae198f8601a285adcf5c0fff204ee6fba5fd&quot;,
      &quot;message&quot;: &quot;Test the CI integration.&quot;,
      &quot;short_id&quot;: &quot;0ff3ae19&quot;,
      &quot;title&quot;: &quot;Test the CI integration.&quot;
    },
    &quot;coverage&quot;: null,
    &quot;allow_failure&quot;: false,
    &quot;created_at&quot;: &quot;2015-12-24T15:51:21.727Z&quot;,
    &quot;started_at&quot;: &quot;2015-12-24T17:54:24.729Z&quot;,
    &quot;finished_at&quot;: &quot;2015-12-24T17:54:24.921Z&quot;,
    &quot;duration&quot;: 0.192,
    &quot;queued_duration&quot;: 0.023,
    &quot;artifacts_expire_at&quot;: &quot;2016-01-23T17:54:24.921Z&quot;,
    &quot;tag_list&quot;: [
      &quot;docker runner&quot;, &quot;ubuntu18&quot;
    ],
    &quot;id&quot;: 6,
    &quot;name&quot;: &quot;rspec:other&quot;,
    &quot;pipeline&quot;: {
      &quot;id&quot;: 6,
      &quot;project_id&quot;: 1,
      &quot;ref&quot;: &quot;main&quot;,
      &quot;sha&quot;: &quot;0ff3ae198f8601a285adcf5c0fff204ee6fba5fd&quot;,
      &quot;status&quot;: &quot;pending&quot;
    },
    &quot;ref&quot;: &quot;main&quot;,
    &quot;artifacts&quot;: [],
    &quot;runner&quot;: null,
    &quot;stage&quot;: &quot;test&quot;,
    &quot;status&quot;: &quot;failed&quot;,
    &quot;tag&quot;: false,
    &quot;web_url&quot;: &quot;https://example.com/foo/bar/-/jobs/6&quot;,
    &quot;user&quot;: {
      &quot;id&quot;: 1,
      &quot;name&quot;: &quot;Administrator&quot;,
      &quot;username&quot;: &quot;root&quot;,
      &quot;state&quot;: &quot;active&quot;,
      &quot;avatar_url&quot;: &quot;http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&amp;d=identicon&quot;,
      &quot;web_url&quot;: &quot;http://gitlab.dev/root&quot;,
      &quot;created_at&quot;: &quot;2015-12-21T13:14:24.077Z&quot;,
      &quot;bio&quot;: null,
      &quot;location&quot;: null,
      &quot;public_email&quot;: &quot;&quot;,
      &quot;skype&quot;: &quot;&quot;,
      &quot;linkedin&quot;: &quot;&quot;,
      &quot;twitter&quot;: &quot;&quot;,
      &quot;website_url&quot;: &quot;&quot;,
      &quot;organization&quot;: &quot;&quot;
    }
  }
]
</code></pre>
<p>Once we have the Job ID, we can call the Play Job API:</p>
<pre class="lang-bash prettyprint-override"><code>curl --request POST --header &quot;PRIVATE-TOKEN: &lt;your_access_token&gt;&quot; &quot;https://gitlab.example.com/api/v4/projects/$CI_PROJECT_ID/jobs/$ID_OF_STAGE_B_JOB/play&quot;
</code></pre>
<p>Note: we don't want to use the Predefined Variable <code>$CI_JOB_ID</code> for the Play Job API since that will be the ID of the Stage A job, not the Stage B job.</p>
<p>Here's the <a href="https://docs.gitlab.com/ee/api/jobs.html#list-pipeline-jobs" rel="nofollow noreferrer">Pipeline Jobs API</a>, and here's the <a href="https://docs.gitlab.com/ee/api/jobs.html#play-a-job" rel="nofollow noreferrer">Play Job API</a> for more information and query parameters.</p>

CC BY-SA 4.0
2021-08-26T17:43:22.470Z
2022-03-21T16:20:21.773Z
2022-03-21T16:20:21.773Z
Adam Marshall
<p>I need to pass info from stage A to stage B so I can change stage B dynamically.</p>
<p>Details:</p>
<ul>
<li>I have a stage (A) where I validate a terraform plan and check if
it's going to destroy resources.</li>
<li>I have another stage (B), which applies the terraform plan, which I want to be triggered manually if stage A detected that resources are going to be destroyed.</li>
<li>When nothing is being destroyed the terraform apply stage (B) should run automatically</li>
</ul>
<p>How can I pass this information from stage A to B?</p>
<p>Making an artifact in A and using &quot;rules:exist&quot; in stage B is not possible, <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/215100" rel="nofollow noreferrer">issue</a></p>
<p>And if I understand the <a href="https://docs.gitlab.com/ee/ci/variables/#pass-an-environment-variable-to-another-job" rel="nofollow noreferrer">docs</a> correctly it's not possible to use a &quot;dotenv&quot; var in the &quot;rules:if&quot; section of stage B.</p>
<p>It seems like Gitlab doesn't want you to change the pipeline definition dynamically.
Maybe there is a practical solution that I'm not aware of to solve this usecase?</p>

<terraform><gitlab><pipeline>
Gitlab CI: change pipeline definition dynamically
Pablo
# Gitlab CI: change pipeline definition dynamically

(Asked by: username_0 on 2021-08-17)

I need to pass info from stage A to stage B so I can change stage B dynamically.

Details:

* I have a stage (A) where I validate a terraform plan and check if
  it's going to destroy resources.
* I have another stage (B), which applies the terraform plan, which I want to be triggered manually if stage A detected that resources are going to be destroyed.
* When nothing is being destroyed the terraform apply stage (B) should run automatically

How can I pass this information from stage A to B?

Making an artifact in A and using "rules:exist" in stage B is not possible, [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/215100)

And if I understand the [docs](https://docs.gitlab.com/ee/ci/variables/#pass-an-environment-variable-to-another-job) correctly it's not possible to use a "dotenv" var in the "rules:if" section of stage B.

It seems like Gitlab doesn't want you to change the pipeline definition dynamically.
Maybe there is a practical solution that I'm not aware of to solve this usecase?

**Tags:** terraform, gitlab, pipeline


---

## 1 answers

### Answer by username_1 on 2021-08-26. Score: 3 | Accepted answer

The reason you can't use dotenv variables in the rules clause(s) is that rules are evaluated before the pipeline even starts executing.
If you have a rules clause like
rules:
  - if: $CI_COMMIT_TAG === 'v1.0.0'
  when: never
  - when: always

Gitlab CI will check the following things:

Is the pipeline a Tag Pipeline (from the CI_PIPELINE_SOURCE variable)? If no, CI_COMMIT_TAG won't exist, so the clause doesn't exist, and when:always is the only rule. Add the job to the pipeline.
If the pipeline is a tag pipeline, is the tag === 'v1.0.0'? Then use when:never. Don't add the job to the pipeline.
Tag pipeline and the tag !== 'v1.0.0', use when:never and add the job to the pipeline.

All these things happen before the pipeline is in the created state, so therefore it cannot depend on a variable that is created once the pipeline starts running.
Knowing this, the way I would solve the problem is to mark the job in Stage B as a manual job like it already is, but when you want to run it automatically, call the Gitlab Jobs API from the Stage A job to run the Stage B job.
This will use a couple of API Calls:

First, we need to get the jobs for this particular Pipeline:

curl --header "PRIVATE-TOKEN: " "https://gitlab.example.com/api/v4/projects/$CI_PROJECT_ID/pipelines/$CI_PIPELINE_IID/jobs?scope=manual"

This will return all the manual jobs from the pipeline. If you have more than one, you can retrieve the correct one by looking at the value for stage and name from the response. Here's an example from the docs:
[
  {
    "commit": {
      "author_email": "admin@example.com",
      "author_name": "Administrator",
      "created_at": "2015-12-24T16:51:14.000+01:00",
      "id": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd",
      "message": "Test the CI integration.",
      "short_id": "0ff3ae19",
      "title": "Test the CI integration."
    },
    "coverage": null,
    "allow_failure": false,
    "created_at": "2015-12-24T15:51:21.727Z",
    "started_at": "2015-12-24T17:54:24.729Z",
    "finished_at": "2015-12-24T17:54:24.921Z",
    "duration": 0.192,
    "queued_duration": 0.023,
    "artifacts_expire_at": "2016-01-23T17:54:24.921Z",
    "tag_list": [
      "docker runner", "ubuntu18"
    ],
    "id": 6,
    "name": "rspec:other",
    "pipeline": {
      "id": 6,
      "project_id": 1,
      "ref": "main",
      "sha": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd",
      "status": "pending"
    },
    "ref": "main",
    "artifacts": [],
    "runner": null,
    "stage": "test",
    "status": "failed",
    "tag": false,
    "web_url": "https://example.com/foo/bar/-/jobs/6",
    "user": {
      "id": 1,
      "name": "Administrator",
      "username": "root",
      "state": "active",
      "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
      "web_url": "http://gitlab.dev/root",
      "created_at": "2015-12-21T13:14:24.077Z",
      "bio": null,
      "location": null,
      "public_email": "",
      "skype": "",
      "linkedin": "",
      "twitter": "",
      "website_url": "",
      "organization": ""
    }
  }
]

Once we have the Job ID, we can call the Play Job API:
curl --request POST --header "PRIVATE-TOKEN: " "https://gitlab.example.com/api/v4/projects/$CI_PROJECT_ID/jobs/$ID_OF_STAGE_B_JOB/play"

Note: we don't want to use the Predefined Variable $CI_JOB_ID for the Play Job API since that will be the ID of the Stage A job, not the Stage B job.
Here's the Pipeline Jobs API, and here's the Play Job API for more information and query parameters.


---

