<p>According to the policies of the company where I work we have dev, preprod and prod environments (on some service dev can be missing). And the path of new version preprod->tests->validation->prod.</p>

<p>In our case jobs in preprod are heavy and complex enough to be sure we do not need to pray when implement in prod :)</p>

<p><strong>Note</strong>: We use svn to maintain and deliver configuration. We do not make changes inplace.</p>

CC BY-SA 3.0
2018-02-19T20:12:11.823Z
I feel like this doesn't quite answer the question wholly.  This answer describes how you deploy changes, but not how you test the changes through the deploy pipeline.
jayhendren
CC BY-SA 3.0
2017-02-28T17:26:44.170Z
How do you maintain the configuration of the different Jenkins servers? Manually?
Michael Pereira
CC BY-SA 3.0
2017-02-28T17:27:45.983Z
We use svn to maintain and deliver configuration. We do not make changes inplace
Romeo Ninov
CC BY-SA 3.0
2017-02-28T17:24:35.060Z
2017-03-04T23:38:31.550Z
2017-03-04T23:38:31.550Z
Romeo Ninov
<p>We needed a 100% HA Jenkins environment. we often upgrade plugins / Jenkins itself.</p>

<p>This causes a big headache if the build breaks after the upgrade.</p>

<p>The safest way to sort this is really getting a Demo Jenkins setup. Maybe on the same machine using multiple Tomcat application, you can achieve this cheaper.</p>

<p>What we did is create a separate (Demo)VM and replicated the prod set-up on the Demo VM. Before changing / upgrading anything we would take a snapshot of both VMs. Then we would test the upgrades on Demo VM. If it works good, change it on Prod.</p>

<p>I guess you can look up the community (like SE/SO) if anyone has faced any problems with the plugin you are planning for.</p>

CC BY-SA 3.0
2018-03-08T05:44:07.070Z
2018-03-08T05:44:07.070Z
Mihir Deshpande
<p>I'd always manually trigger a re-run or two on at least one recent green (or almost green) label on every relevant project/branch that uses the respective plugin and check that I get the same results. Just to be on the safe side.</p>

<p>Any result discrepancy would need to be investigated to determine it they're caused by the plugin update or not. Maybe a few more re-runs with both the old and new plugins?</p>

CC BY-SA 3.0
2017-03-15T20:08:55.307Z
Sure, no problem.
Dan Cornilescu
CC BY-SA 3.0
2017-03-16T00:04:03.260Z
From past experience my opinions often aren't that popular, so in general I tend to avoid the spotlight if possible :) I'm also unfamiliar with the mod tools. But I don't mind helping, especially if it is needed - I have high hopes for this site.
Dan Cornilescu
CC BY-SA 3.0
2017-03-05T04:31:37.530Z
2017-03-05T04:31:37.530Z
Dan Cornilescu
<p>If you ever been bitten by a plugin upgrade that broke some functionality you must have given some thought about this problem: What should be the Jenkins plugins upgrade policy? How do you test changes before deploying them?</p>

<p>Has anyone gone as far as having a test instance running dummy jobs to test the new versions or do you just pray that upgrading the versions won't break anything?</p>

CC BY-SA 3.0
2017-03-06T19:58:27.327Z
@DanCornilescu my organisation policy as this is for our internal Jenkins server
Michael Pereira
CC BY-SA 3.0
2017-03-06T20:01:10.260Z
@AssafLavie That's highly dependent on the way Jenkins is running: Standalone server, VM, docker container, kubernetes pods (our case). It may not be easy to take a snapshot of the current state to restore it as is.

In our case we can clone the EBS volume holding the Jenkins data but it's a manual and time-consuming process to restore both the container and data volume to a specific state.
Michael Pereira
CC BY-SA 3.0
2017-04-04T14:15:13.743Z
Hi @MichaelPereira if either of the two answers below have solved your question please consider [accepting it](https://meta.stackexchange.com/q/5234/179419) by clicking the check-mark. This indicates to the wider community that you've found a solution and gives some reputation to both the answerer and yourself. There is no obligation to do this.  If you don't feel that your question has been answered please do feel free to engage with the authors in the comments.
Richard Slater
CC BY-SA 3.0
2017-03-05T04:22:48.787Z
Do you mean Jeankins' team policy or your (organisation's) policy?
Dan Cornilescu
CC BY-SA 3.0
2017-03-05T08:24:03.490Z
I would take a snapshot of the Jenkins node before upgrading and just test it out. In my experience Jenkins has never been a mission critical component. If it's "down" for 15 minutes because some plugin upgrade broke it, it's usually not blocking production in any way, so manual intervention is acceptable. Of course, if that's not the case for you (and Jenkins must be 100% HA), it's not the right approach.
Assaf Lavie
<jenkins><jenkins-plugins>
How do you test changes to Jenkins plugins before deploying them?
Michael Pereira
# How do you test changes to Jenkins plugins before deploying them?

(Asked by: username_0 on 2017-02-28)

If you ever been bitten by a plugin upgrade that broke some functionality you must have given some thought about this problem: What should be the Jenkins plugins upgrade policy? How do you test changes before deploying them?

Has anyone gone as far as having a test instance running dummy jobs to test the new versions or do you just pray that upgrading the versions won't break anything?

**Tags:** jenkins, jenkins-plugins

> **Comments:**
> **username_2**: Do you mean Jeankins' team policy or your (organisation's) policy?  
> **username_3**: I would take a snapshot of the Jenkins node before upgrading and just test it out. In my experience Jenkins has never been a mission critical component. If it's "down" for 15 minutes because some plugin upgrade broke it, it's usually not blocking production in any way, so manual intervention is acceptable. Of course, if that's not the case for you (and Jenkins must be 100% HA), it's not the right approach.  
> **username_0**: @DanCornilescu my organisation policy as this is for our internal Jenkins server  
> **username_0**: @AssafLavie That's highly dependent on the way Jenkins is running: Standalone server, VM, docker container, kubernetes pods (our case). It may not be easy to take a snapshot of the current state to restore it as is.  In our case we can clone the EBS volume holding the Jenkins data but it's a manual and time-consuming process to restore both the container and data volume to a specific state.  
> **username_1**: Hi @MichaelPereira if either of the two answers below have solved your question please consider [accepting it](https://meta.stackexchange.com/q/5234/179419) by clicking the check-mark. This indicates to the wider community that you've found a solution and gives some reputation to both the answerer and yourself. There is no obligation to do this.  If you don't feel that your question has been answered please do feel free to engage with the authors in the comments.  

---

## 3 answers

### Answer by username_4 on 2017-02-28. Score: 4

According to the policies of the company where I work we have dev, preprod and prod environments (on some service dev can be missing). And the path of new version preprod->tests->validation->prod.

In our case jobs in preprod are heavy and complex enough to be sure we do not need to pray when implement in prod :)

**Note**: We use svn to maintain and deliver configuration. We do not make changes inplace.

> **Comments:**
> **username_0**: How do you maintain the configuration of the different Jenkins servers? Manually?  
> **username_4**: We use svn to maintain and deliver configuration. We do not make changes inplace  
> **username_5**: I feel like this doesn't quite answer the question wholly.  This answer describes how you deploy changes, but not how you test the changes through the deploy pipeline.  

---

### Answer by username_6 on 2018-03-08. Score: 2

We needed a 100% HA Jenkins environment. we often upgrade plugins / Jenkins itself.

This causes a big headache if the build breaks after the upgrade.

The safest way to sort this is really getting a Demo Jenkins setup. Maybe on the same machine using multiple Tomcat application, you can achieve this cheaper.

What we did is create a separate (Demo)VM and replicated the prod set-up on the Demo VM. Before changing / upgrading anything we would take a snapshot of both VMs. Then we would test the upgrades on Demo VM. If it works good, change it on Prod.

I guess you can look up the community (like SE/SO) if anyone has faced any problems with the plugin you are planning for.


---

### Answer by username_2 on 2017-03-05. Score: 0

I'd always manually trigger a re-run or two on at least one recent green (or almost green) label on every relevant project/branch that uses the respective plugin and check that I get the same results. Just to be on the safe side.

Any result discrepancy would need to be investigated to determine it they're caused by the plugin update or not. Maybe a few more re-runs with both the old and new plugins?

> **Comments:**
> **username_2**: Sure, no problem.  
> **username_2**: From past experience my opinions often aren't that popular, so in general I tend to avoid the spotlight if possible :) I'm also unfamiliar with the mod tools. But I don't mind helping, especially if it is needed - I have high hopes for this site.  

---

