<p>There are two approaches to this, neither of them, in my opinion, are especially good as they don't strictly "NOT Fail" the build:</p>

<ol>
<li><p>Use a plugin such as <a href="https://wiki.jenkins.io/display/JENKINS/Naginator+Plugin" rel="nofollow noreferrer">Naginator</a> which can be configured to restart the build until it passes.  This can be quite frustrating as you need to differentiate between ephemeral/preemptable virtual machines being terminated and a truly failed build.</p></li>
<li><p>Move to an <a href="https://cloudify.co/2015/04/09/jenkins-ci-server-automation-cloud-cd-continuous-integration-delivery.html" rel="nofollow noreferrer">orchestrator model</a> that manages individual jobs and restarts them based upon a set of rules.</p></li>
</ol>

<p>Perhaps a better solution would be to develop a Jenkins Plugin similar to the <a href="https://github.com/jenkinsci/ec2-fleet-plugin" rel="nofollow noreferrer">EC2 Fleet Plugin</a> which is responsible for spinning up and maintaining Spot Instances (i.e. similar to GCEs Preemptable Instances) resubmitting jobs when they fail due to instance reclamation.</p>

CC BY-SA 4.0
2019-10-17T10:52:49.183Z
2019-10-17T10:52:49.183Z
Richard Slater
<p>Exactly this. In our environment, we're using GCE preemptible nodes and the Jenkins GCE plugin. As a consequence of this, it's highly likely (even expected) that a node will go away during the course of a build. The problem is, the Jenkins GCE plugin just marks that node as "offline", but a parallel job "part" that was assigned an executor on that node will just keep churning away there, seemingly forever, apparently in the hope that the node will come back.</p>

<p>The question I have, then, is simple:</p>

<p>Is it possible to force Jenkins to reschedule (<strong><em>NOT</em></strong> fail) a parallel build "part" as soon as the node it's executing on goes offline? If so, how is this done?</p>

<jenkins><jenkins-pipeline>
Force Jenkins to reschedule parallel pipeline step as soon as node goes offline
Sebastian Lenartowicz
# Force Jenkins to reschedule parallel pipeline step as soon as node goes offline

(Asked by: username_0 on 2018-06-27)

Exactly this. In our environment, we're using GCE preemptible nodes and the Jenkins GCE plugin. As a consequence of this, it's highly likely (even expected) that a node will go away during the course of a build. The problem is, the Jenkins GCE plugin just marks that node as "offline", but a parallel job "part" that was assigned an executor on that node will just keep churning away there, seemingly forever, apparently in the hope that the node will come back.

The question I have, then, is simple:

Is it possible to force Jenkins to reschedule (***NOT*** fail) a parallel build "part" as soon as the node it's executing on goes offline? If so, how is this done?

**Tags:** jenkins, jenkins-pipeline


---

## 1 answers

### Answer by username_1 on 2019-10-17. Score: 2

There are two approaches to this, neither of them, in my opinion, are especially good as they don't strictly "NOT Fail" the build:

1. Use a plugin such as [Naginator](https://wiki.jenkins.io/display/JENKINS/Naginator+Plugin) which can be configured to restart the build until it passes. This can be quite frustrating as you need to differentiate between ephemeral/preemptable virtual machines being terminated and a truly failed build.
2. Move to an [orchestrator model](https://cloudify.co/2015/04/09/jenkins-ci-server-automation-cloud-cd-continuous-integration-delivery.html) that manages individual jobs and restarts them based upon a set of rules.

Perhaps a better solution would be to develop a Jenkins Plugin similar to the [EC2 Fleet Plugin](https://github.com/jenkinsci/ec2-fleet-plugin) which is responsible for spinning up and maintaining Spot Instances (i.e. similar to GCEs Preemptable Instances) resubmitting jobs when they fail due to instance reclamation.


---

