<p>You could extract the common, shareable portions of the scripts which you'd package as either standalone scripts or libraries (depending on the scripting language you use and/or preferences), placed in a shared location (of course, backed by their own repository). </p>

<p>Then, in the scripts from your code repositories you'd replace the extracted portions with invocations of the scripts (or calls into the libraries) from the shared location.</p>

<p>Some CI/CD tools <em>might</em> be capable of directly invoking the scripts from the shared location, so if there aren't any specific portions in some of the code repo scripts you could drop them completely from those repos.</p>

CC BY-SA 4.0
2019-03-17T22:50:32.317Z
2019-03-17T22:50:32.317Z
Dan Cornilescu
<p>Hey Sergey and Welcome to the DevOps StackExchange.</p>

<p>I agree with @bgdnlp on using git submodules if you are really keen on keeping your bash scripts. </p>

<p>I however would recommend you look at other tooling to facilitate your transition into the world of CI/CD. While your stack is simple keeping a couple bash scripts handy is a sensible way to go about it. As your stack gets more complicated and the need for more testing, secrets management, monitoring and performance testing arises I think you will find that your bash scripts are a little lacking. </p>

<p>Where I work we use Jenkins to handle our builds which allows us to use shared libraries written in Groovy. We right one function and can now use it across many builds.</p>

<p>You could also do it with some newer tools like <a href="http://buildbot.net/" rel="nofollow noreferrer">http://buildbot.net/</a></p>

<p>You could also convert your scripts to go or python and bundle them up in a package and install them each time on your build agent.</p>

CC BY-SA 4.0
2019-03-17T02:14:18.687Z
2019-03-17T02:14:18.687Z
Levi
<p>What I have seen working is to use a custom docker image with all the common tools bundled in during the building process. And it seems that Travis CI supports this as well </p>

<p><a href="https://docs.travis-ci.com/user/docker/" rel="nofollow noreferrer">https://docs.travis-ci.com/user/docker/</a></p>

<p>That way you can always expect that the custom building modules are available during the building process. </p>

CC BY-SA 4.0
2019-04-21T21:01:51.937Z
2019-04-21T21:01:51.937Z
petrchpetr
<p>Let's say you've got several code repositories on GitHub. You want to build, test and deploy them. You picked a continuous integration service (e.g. Travis CI, AppVeyor or something else) and wrote some scripts for building, testing and deploying. </p>

<p>If your projects are very similar, let's say all of them written in Golang, the result of the build is always a Docker image and deployment is always a push to Docker Hub, you don't want to duplicate the scripts across all of your repositories. How would you avoid that?</p>

CC BY-SA 4.0
2019-03-16T10:19:35.307Z
The question seems too broad to me. Different CI tools have different features. Might be better to specify exactly what you're looking for. That said, since you're using git, you might want to look into [git submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules). It's a way of including other repositories into your git repository. That is, assuming the CI tool script lives in the app's repo. You'd put your scripts into one CI repo and use that as a submodule. Combine with symlinks and you might get something working. But, again, more specific information might be useful.
bgdnlp
CC BY-SA 4.0
2019-03-16T10:31:01.893Z
Maybe indeed it's a quite broad question. Then let's, just for narrowing it down, say that I have several Git repositories, my build [scripts](https://github.com/moikot/golang-dep/tree/master/.travis) are written in Bash, they are very similar, not too complex but, at the same time, I don't want to duplicate them in other repositories. Git submodule is a nice idea and maybe it is the way to go, but I was thinking along the way of Gradle. But Gradle looks too heavy for this kind of task.
Sergey Anisimov
<continuous-integration><automation><shell-script>
How to avoid build scripts duplication
Sergey Anisimov
# How to avoid build scripts duplication

(Asked by: username_0 on 2019-03-16)

Let's say you've got several code repositories on GitHub. You want to build, test and deploy them. You picked a continuous integration service (e.g. Travis CI, AppVeyor or something else) and wrote some scripts for building, testing and deploying.

If your projects are very similar, let's say all of them written in Golang, the result of the build is always a Docker image and deployment is always a push to Docker Hub, you don't want to duplicate the scripts across all of your repositories. How would you avoid that?

**Tags:** continuous-integration, automation, shell-script

> **Comments:**
> **username_1**: The question seems too broad to me. Different CI tools have different features. Might be better to specify exactly what you're looking for. That said, since you're using git, you might want to look into [git submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules). It's a way of including other repositories into your git repository. That is, assuming the CI tool script lives in the app's repo. You'd put your scripts into one CI repo and use that as a submodule. Combine with symlinks and you might get something working. But, again, more specific information might be useful.  
> **username_0**: Maybe indeed it's a quite broad question. Then let's, just for narrowing it down, say that I have several Git repositories, my build [scripts](https://github.com/moikot/golang-dep/tree/master/.travis) are written in Bash, they are very similar, not too complex but, at the same time, I don't want to duplicate them in other repositories. Git submodule is a nice idea and maybe it is the way to go, but I was thinking along the way of Gradle. But Gradle looks too heavy for this kind of task.  

---

## 3 answers

### Answer by username_2 on 2019-03-17. Score: 1

You could extract the common, shareable portions of the scripts which you'd package as either standalone scripts or libraries (depending on the scripting language you use and/or preferences), placed in a shared location (of course, backed by their own repository).

Then, in the scripts from your code repositories you'd replace the extracted portions with invocations of the scripts (or calls into the libraries) from the shared location.

Some CI/CD tools *might* be capable of directly invoking the scripts from the shared location, so if there aren't any specific portions in some of the code repo scripts you could drop them completely from those repos.


---

### Answer by username_3 on 2019-03-17. Score: 0

Hey Sergey and Welcome to the DevOps StackExchange.

I agree with @username_1 on using git submodules if you are really keen on keeping your bash scripts.

I however would recommend you look at other tooling to facilitate your transition into the world of CI/CD. While your stack is simple keeping a couple bash scripts handy is a sensible way to go about it. As your stack gets more complicated and the need for more testing, secrets management, monitoring and performance testing arises I think you will find that your bash scripts are a little lacking.

Where I work we use Jenkins to handle our builds which allows us to use shared libraries written in Groovy. We right one function and can now use it across many builds.

You could also do it with some newer tools like <http://buildbot.net/>

You could also convert your scripts to go or python and bundle them up in a package and install them each time on your build agent.


---

### Answer by username_4 on 2019-04-21. Score: 0

What I have seen working is to use a custom docker image with all the common tools bundled in during the building process. And it seems that Travis CI supports this as well

<https://docs.travis-ci.com/user/docker/>

That way you can always expect that the custom building modules are available during the building process.


---

