<p>There is a natural tension here between:</p>

<p>(a) <strong>fixing something that is wrong automatically</strong>: because for example, a CVE has been created with a high enough severity to need to do something about it.</p>

<p>(b) <strong>breaking the system</strong>: because the automation introduced a breaking change or a defect.</p>

<p>I have a tendency towards systems being under constant development, even if no features are being added to them they need to kept current to avoid exposure to vulnerabilities.   The rate of change to the 3rd party components of the majority of systems I cover is so high that's its actually worth rebuilding everything and redeploying it daily.</p>

<p>This introduces a couple of requirements:</p>

<ol>
<li>It is highly automated,</li>
<li>99% of user journies are covered by end-to-end tests so each build has a high degree of confidence that it will work,</li>
<li>We don't roll out to all users, we employ <a href="https://martinfowler.com/bliki/CanaryRelease.html" rel="nofollow noreferrer">canary release</a> and a rollback mechanism,</li>
<li>We use version <em>constraints</em> in package manifests, etc... i.e. the <code>version of React must be &gt;= 16.7.1 and &lt; 17.0.0</code></li>
<li>We manage by exception, i.e. if the end-to-end tests fail or the canary release fails humans get involved.</li>
</ol>

<p>This, in turn, makes systems like <a href="https://github.com/aquasecurity/trivy" rel="nofollow noreferrer">trivy</a>, <a href="https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project" rel="nofollow noreferrer">OWASP ZAP</a> and <a href="https://www.sonarqube.org/" rel="nofollow noreferrer">SonarQube</a> more about generating metrics and stopping vulnerabilities from going live than triggering automated remediation.</p>

<p>We do then have a set of rules like:</p>

<ol>
<li>The live application must not have an unmitigated CVE with a CVSS score of 7.0 or higher.</li>
<li>The list of CVEs with a score of 4.0 or higher will be reviewed on a weekly basis to assess if remediation should be undertaken (i.e. change to a different library, introduce a new safeguard, remove a defunct library, fix the CVE at source, etc.)</li>
</ol>

<p>This then gives my SRE teams the operational boundaries to work within, they may in practice:</p>

<ol>
<li>just re-run the deployment ad-hoc to fix it <strong>now</strong> rather than within 24h,</li>
<li>override a default, for example, if there is a known vulnerability deploy something with a less critical but still higher than CVSS 7.0 known vulnerability,</li>
<li>Introduce a new safeguard, i.e. a Web Application Firewall Rule to mitigate,</li>
<li>Swap out the dependency for something less vulnerable,</li>
<li>Disable that functionality entirely until a fix can be introduced.</li>
</ol>

CC BY-SA 4.0
2019-10-14T10:58:59.187Z
2019-10-14T10:58:59.187Z
Richard Slater
<p>Let's say I have a number of tools like Docker bench security, some static security scanners, etc. Now I have let's say free Ansible edition (no expenses are considered).</p>

<p>I have lots of logs, but not much automated, actionable info that triggers scripts. What is a proper way to automate link between static scanners and CM tools like Ansible and Jenkins? </p>

<p>Should it go like:</p>

<ol>
<li>Parse logs for set of errors and then automatically trigger "correcting scripts"?</li>
<li>Just run a playbook once in a while as it checks the state before doing anything?</li>
<li>Use tools/plugins for Ansible?</li>
<li>Something else?</li>
</ol>

CC BY-SA 4.0
2018-08-04T05:14:05.660Z
With respect to [Tools, not policy](https://en.wikipedia.org/wiki/Unix_philosophy) there will be more proper ways depending on the use-case.
Vladimir Botka
CC BY-SA 4.0
2018-10-22T15:13:18.093Z
It sounds like what you are describing here is automated _remediation_ not automated configuration management.
Bruce Becker
<jenkins><ansible><jenkins-pipeline><configuration-management>
How to properly automate CM based on log events?
kol23
# How to properly automate CM based on log events?

(Asked by: username_0 on 2018-06-08)

Let's say I have a number of tools like Docker bench security, some static security scanners, etc. Now I have let's say free Ansible edition (no expenses are considered).

I have lots of logs, but not much automated, actionable info that triggers scripts. What is a proper way to automate link between static scanners and CM tools like Ansible and Jenkins?

Should it go like:

1. Parse logs for set of errors and then automatically trigger "correcting scripts"?
2. Just run a playbook once in a while as it checks the state before doing anything?
3. Use tools/plugins for Ansible?
4. Something else?

**Tags:** jenkins, ansible, jenkins-pipeline, configuration-management

> **Comments:**
> **username_1**: With respect to [Tools, not policy](https://en.wikipedia.org/wiki/Unix_philosophy) there will be more proper ways depending on the use-case.  
> **username_2**: It sounds like what you are describing here is automated _remediation_ not automated configuration management.  

---

## 1 answers

### Answer by username_3 on 2019-10-14. Score: 1

There is a natural tension here between:

(a) **fixing something that is wrong automatically**: because for example, a CVE has been created with a high enough severity to need to do something about it.

(b) **breaking the system**: because the automation introduced a breaking change or a defect.

I have a tendency towards systems being under constant development, even if no features are being added to them they need to kept current to avoid exposure to vulnerabilities. The rate of change to the 3rd party components of the majority of systems I cover is so high that's its actually worth rebuilding everything and redeploying it daily.

This introduces a couple of requirements:

1. It is highly automated,
2. 99% of user journies are covered by end-to-end tests so each build has a high degree of confidence that it will work,
3. We don't roll out to all users, we employ [canary release](https://martinfowler.com/bliki/CanaryRelease.html) and a rollback mechanism,
4. We use version *constraints* in package manifests, etc... i.e. the `version of React must be >= 16.7.1 and < 17.0.0`
5. We manage by exception, i.e. if the end-to-end tests fail or the canary release fails humans get involved.

This, in turn, makes systems like [trivy](https://github.com/aquasecurity/trivy), [OWASP ZAP](https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project) and [SonarQube](https://www.sonarqube.org/) more about generating metrics and stopping vulnerabilities from going live than triggering automated remediation.

We do then have a set of rules like:

1. The live application must not have an unmitigated CVE with a CVSS score of 7.0 or higher.
2. The list of CVEs with a score of 4.0 or higher will be reviewed on a weekly basis to assess if remediation should be undertaken (i.e. change to a different library, introduce a new safeguard, remove a defunct library, fix the CVE at source, etc.)

This then gives my SRE teams the operational boundaries to work within, they may in practice:

1. just re-run the deployment ad-hoc to fix it **now** rather than within 24h,
2. override a default, for example, if there is a known vulnerability deploy something with a less critical but still higher than CVSS 7.0 known vulnerability,
3. Introduce a new safeguard, i.e. a Web Application Firewall Rule to mitigate,
4. Swap out the dependency for something less vulnerable,
5. Disable that functionality entirely until a fix can be introduced.


---

