<p>Helm is a Kubernetes tool so this answer assumes you are deploying to Kubernetes or OpenShift not Docker Swarm. </p>

<p>If you are using Helm well then I don't think it is particularly important what build pipeline tech you are using to invoke it. I think what is more important is how you manage all your configuration within Git and how you structure the Helm deployments.  </p>

<p>There are two different paradigms you can follow: </p>

<ol>
<li>Build the app container image with the correct environment variables baked within it for each environment. </li>
<li>Build only one app container image that picks up environment variables from within the environment it has been deployed.  </li>
</ol>

<p>If you follow <a href="https://www.12factor.net/config" rel="nofollow noreferrer">12factor.net</a> best practices then you go for Option 2. IMHO it feels far cleaner and less error-prone to move the exact same image between kubernetes namespaces that represent different environments. We store environment variables inside of Kubernetes as <code>secrets</code> or <code>configmaps</code>. We then get to two more questions:</p>

<ol>
<li>How do you load <code>secrets</code> and <code>configmaps</code> as environment variables into your .Net Core runtime images when the container starts up?</li>
<li>How to you manage the <code>secrets</code> and <code>configmaps</code> in each environment? </li>
</ol>

<p>With respect to the first question, Kubernetes has this covered. Any deployment can name a bunch of different secrets that should be <a href="https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets-as-environment-variables" rel="nofollow noreferrer">loaded</a> as environment variables. Ditto for <a href="https://akomljen.com/kubernetes-environment-variables/" rel="nofollow noreferrer">configmaps</a>. You just need each environment to be a different namespace so that the same logical secret can have the same name in each environment. Your database credentials can then be a secret called <code>main-database-credentials</code> that holds different credentials in different environments. </p>

<p>To deal with the second question we use Helm to manage both secrets and configmaps. We have one generic Helm chart that deploys all our different secrets in all our different environment:</p>

<p><code>helm upgrade --install -f $NAME.yaml $NAME secret-1.1.tar.gz</code> </p>

<p>So to upgrade our <code>main-database-credentials</code> secret we would change the settings in <code>main-database-credentials.yaml</code> then tell the chart to recreate the <code>main-database-credentials</code> deployment using that command. The chart simply uses the Helm templating language to build yaml that defines the secret using the name set in the yaml file. Likewise, we have a generic chart that updates <code>configmaps</code> that is invoked with a very simlar command as above. Helm uses a standard Golang templating library so it is very straight-forward to create those generic charts. </p>

<p>At first, it may seem a bit heavyweight to use Helm to update something like a database password secret. Yet helm gives you a deployment history and lets you diff the history of deployments. It also lets you easily roll-back to a previous deployment. That is really powerful as an audit trail of config changes that is really helpful when managing a complex microservices deployment. Since we also use Helm to deploy all our apps it makes perfect sense to use the same power tool to manage the deployment of configuration into our environments. </p>

<p>As we can use a generic helm chart to deploy many secrets into many environments why not use a generic helm chart to deploy all our microservices? We do: </p>

<p><code>helm upgrade --install -f $NAME.yaml $NAME deployer-1.3.tar.gz</code></p>

<p>This means that <code>NAME=my-microservice</code> and <code>NAME=my-admin-webapp</code> or anything else can be deployed using one generic <code>deployer</code> chart. The path and tag of the application container image to deploy is in the yaml file. The container path is different for different apps and the container tag can be different between environments. </p>

<p>That might sound strange as normally Helm packages a particular app so you would expect a one-to-one relationship between charts and apps. Yet if all your microservices are containers following 12factor.net best practices shouldn't they all "look the same" from the perspective of deploying them; it is just the container and the environment variables you deploy into the one generic chart. I see it that the app is the pure container with none of the configurations to run it and the Kubernetes configuration to run it is "application infrastructure". So I have a Helm chart for each type of application infrastructure. I don't want each bit of application infrastructure to be hand-rolled I want to cookie cutter create it using Helm. </p>

<p>Now we have covered all the Helm mechanics we are left with the question "how do we manage all our configuration within git?". We use different repo for different purposes: </p>

<ol>
<li>Each microservice has its own repo. We use CircleCI to build the code into an image that is configured via environment variables. We don't store any Helm or Kubernetes config in this repo. We do have a properties file that names the latest version of the <code>deployer</code> chart this code is compatible with. </li>
<li>We have a GitHub repo that holds the source of the generic Helm charts that are used to deploy all our apps and secrets. These don't actually change very often now they are fairly stable. CircleCI builds and releases the charts as tar.gz onto the GitHub releases page for this repo.  </li>
<li>We have a git repo per environment. This has a folder per deployed microservice within that environment. Each contains the <code>yaml</code> file to pass to the helm deploy encrypted with <code>git secret</code>. It has a folder per deployed <code>secret</code> within that environment. Each contains a <code>yaml</code> file that is <code>git secret</code> encrypted. Ditto for <code>configmaps</code>.  </li>
</ol>

<p>Why a repo per environment? A branch per environment sounds more normal. Only a database credential for the live environment never moves between environments. As we use a separate repo for live we can give it separate permissions. We might let all developers go wild in the <code>dev-environment</code> repo we have a bit more discipline with the <code>live-environment</code> repo. </p>

<p>We then have CircleCI do automated deployments of configuration. This works as Helm data detects changes and doesn't deploy a change to any Kubrernetes object if the latest yaml it generates is already deployed. To change a password in live you change it in the <code>git secret</code> encrypted yaml in the live repo and apply a release tag. CircleCI pattern matches on the tag name to see that its a release tag and it runs the deploy step. That uses a container with the helm client to run a bash script that loops over all app folders, secret folders and configmap folders running Helm for the right chart and correct yaml. Helm will only apply updates where there are changes within git. </p>

CC BY-SA 4.0
2018-11-10T20:21:18.153Z
Thank you. Great info here. The k8s and docker env was built before i started and they arent using it yet.  My real question is do i replace octopus or take advantange of whatvis already There and use tge new festures that allow you to deploy to k8s. This customer really has no vlue what they wsnt. But the seem to believe they shoukd replace octopus alltogether with k8s and docker. I think we use what we have and ad k8s and docker to octopus
Just a guy
CC BY-SA 4.0
2018-11-11T07:37:38.317Z
old microsoft tech wasn't 12factor friendly. octopus was liked by teams i knew as solved repackaging same code "built once" for each env. that isn't a problem with .net core and k8s. you only got a few tentacles for free their so they liked octopus but couldn't use it on microservices as not enough free capacity. yet if it is working for your team today why change it immediately? how config is managed and whether the same container moves through envs that devs can run locally overriding vars is most important part. get it working that way and remove octopus last if its not helping.
simbo1905
CC BY-SA 4.0
2018-11-11T08:08:07.907Z
for "build once, deploy many times" I strongly recommend Helm. it doesn't really matter what is invoking Helm. we use a CircleCI deploy step, in the past, I have had VSTS deploying to k8s with bash script. developers have strong opinions the about "build" solution and SCM but are not bothered about deploy tooling if it "just works". CircleCI works for us as it integrates with GitHub and lets me run a helm container in the deploy step with a simple bash script.
simbo1905
CC BY-SA 4.0
2018-11-11T16:26:06.007Z
Thank you. Its been very difficult finding info. Mostly because people usually choise one or another.as of now i am creating a new octopus server to test the octopus to k8s deployment since this team knows tfs and octopus.  They no nothing about hhelm k8s or docker. Once i hsve a test env. I will do 1 in octopus and one without octopus
Just a guy
CC BY-SA 4.0
2018-11-11T16:57:52.527Z
Im standing up a new env in parallel to what thry currently use. So i can build it however i want. So tfs to dockerfile, build .net in container" image push to local registry. Deploy to k8s via helm
Just a guy
CC BY-SA 4.0
2018-11-11T16:59:35.690Z
so what SCM and build CI? GitHub and ...?
simbo1905
CC BY-SA 4.0
2018-11-11T17:51:41.373Z
As i am new to helm one of the things i am a little confused by is..the nuget pkgs.  I need to have tfs deploy to a dockerfile, run the dotnet build there,  image it then have helm pull the image from our local docker registry? This would eliminate octopus
Just a guy
CC BY-SA 4.0
2018-11-11T18:01:35.247Z
Since all of the repos are in tfs and the mgr uses tfs to create work items and perform his mgnt duties along with security policies. We are upgrading to tfs 2018. Each micro service is in its own git repo within tfs. So from tfs deploy to dockerfile insert env variables from octopus into dockerfile, image, push to docker registry.. After that, as i said im new to helm and i believe you may have answered this above. Helm would deploy the docker image to k8s.. I guess i really need to look at helm more
Just a guy
CC BY-SA 4.0
2018-11-11T18:02:45.317Z
sure tfs makes the image ready to run. that’s build not deploy. to deploy you must run helm client against tiller server running in k8s. you don’t want to run helm client by hand “something” must run it to change k8s deployment config object to reference new image built by tfs. I used vsts (SaaS tfs) to do deployments with bash script run on Linux azure VMs that could call something like helm to do the deploy workflow.
simbo1905
CC BY-SA 4.0
2018-11-11T18:05:08.343Z
So getting env variables out of octopus and into either 1 of thec2 options above. Strip them ftom octopus, put them in docker file or put them in k8s as secrets
Just a guy
CC BY-SA 4.0
2018-11-11T18:11:52.490Z
I cant Thank you enough. You cleared up a lot of my questions. Tomortow i will spend more time learning helm better. But i think this is a great place to start. You may see me add a comment this week if ibrun into some confusion. Agsin thank you
Just a guy
CC BY-SA 4.0
2018-11-11T18:51:01.523Z
default tfs/vso bash task only runs script in repo you build. so you cannot use my answer approach of having deploy scripts in another git repo. the solution is a free contributor bash task in the marketplace that lets you set the bash script code as text property of the task. so script is saved in the deploy workflow. I had it “curl $deploy_git_url | /bin/bash” where the “$deploy_git_url” was set at project level to point to my separate git repo of k8s deploy code. That script can run helm having been passed parameters from deploy pipeline of what is latest built container to deploy.
simbo1905
CC BY-SA 4.0
2018-11-30T06:10:02.647Z
No-one has up voted my answer :*(
simbo1905
CC BY-SA 4.0
2018-12-10T18:53:01.567Z
Wow still no upvote or acceptance of my answer.
simbo1905
CC BY-SA 4.0
2018-11-10T16:15:54.603Z
2018-11-10T17:31:14.150Z
2018-11-10T17:31:14.150Z
simbo1905
<p>My current client uses .net and .net core,  tfs and deploys via octopus. They want to use helm, k8s and docker.  With all of the environments, env variables are already in octopus.</p>

<p>I am thinking to restore the nuget packages in the dockerfile, build the image, push to our local registry. Add our docker registry as a feed, add helm as a feed then creating a lifecycle for dev and prod. Then modify the octopus jobs to point to the local registry to pull the images.</p>

<p>Anyone see any issues here? Should I strip out the env variables from octopus and just get rid of octopus?</p>

<docker><kubernetes><helm><octopus-deploy>
Tfs octopus migration to k8s and docker
Just a guy
# Tfs octopus migration to k8s and docker

(Asked by: username_0 on 2018-11-10)

My current client uses .net and .net core, tfs and deploys via octopus. They want to use helm, k8s and docker. With all of the environments, env variables are already in octopus.

I am thinking to restore the nuget packages in the dockerfile, build the image, push to our local registry. Add our docker registry as a feed, add helm as a feed then creating a lifecycle for dev and prod. Then modify the octopus jobs to point to the local registry to pull the images.

Anyone see any issues here? Should I strip out the env variables from octopus and just get rid of octopus?

**Tags:** docker, kubernetes, helm, octopus-deploy


---

## 1 answers

### Answer by username_1 on 2018-11-10. Score: 2

Helm is a Kubernetes tool so this answer assumes you are deploying to Kubernetes or OpenShift not Docker Swarm.

If you are using Helm well then I don't think it is particularly important what build pipeline tech you are using to invoke it. I think what is more important is how you manage all your configuration within Git and how you structure the Helm deployments.

There are two different paradigms you can follow:

1. Build the app container image with the correct environment variables baked within it for each environment.
2. Build only one app container image that picks up environment variables from within the environment it has been deployed.

If you follow [12factor.net](https://www.12factor.net/config) best practices then you go for Option 2. IMHO it feels far cleaner and less error-prone to move the exact same image between kubernetes namespaces that represent different environments. We store environment variables inside of Kubernetes as `secrets` or `configmaps`. We then get to two more questions:

1. How do you load `secrets` and `configmaps` as environment variables into your .Net Core runtime images when the container starts up?
2. How to you manage the `secrets` and `configmaps` in each environment?

With respect to the first question, Kubernetes has this covered. Any deployment can name a bunch of different secrets that should be [loaded](https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets-as-environment-variables) as environment variables. Ditto for [configmaps](https://akomljen.com/kubernetes-environment-variables/). You just need each environment to be a different namespace so that the same logical secret can have the same name in each environment. Your database credentials can then be a secret called `main-database-credentials` that holds different credentials in different environments.

To deal with the second question we use Helm to manage both secrets and configmaps. We have one generic Helm chart that deploys all our different secrets in all our different environment:

`helm upgrade --install -f $NAME.yaml $NAME secret-1.1.tar.gz`

So to upgrade our `main-database-credentials` secret we would change the settings in `main-database-credentials.yaml` then tell the chart to recreate the `main-database-credentials` deployment using that command. The chart simply uses the Helm templating language to build yaml that defines the secret using the name set in the yaml file. Likewise, we have a generic chart that updates `configmaps` that is invoked with a very simlar command as above. Helm uses a standard Golang templating library so it is very straight-forward to create those generic charts.

At first, it may seem a bit heavyweight to use Helm to update something like a database password secret. Yet helm gives you a deployment history and lets you diff the history of deployments. It also lets you easily roll-back to a previous deployment. That is really powerful as an audit trail of config changes that is really helpful when managing a complex microservices deployment. Since we also use Helm to deploy all our apps it makes perfect sense to use the same power tool to manage the deployment of configuration into our environments.

As we can use a generic helm chart to deploy many secrets into many environments why not use a generic helm chart to deploy all our microservices? We do:

`helm upgrade --install -f $NAME.yaml $NAME deployer-1.3.tar.gz`

This means that `NAME=my-microservice` and `NAME=my-admin-webapp` or anything else can be deployed using one generic `deployer` chart. The path and tag of the application container image to deploy is in the yaml file. The container path is different for different apps and the container tag can be different between environments.

That might sound strange as normally Helm packages a particular app so you would expect a one-to-one relationship between charts and apps. Yet if all your microservices are containers following 12factor.net best practices shouldn't they all "look the same" from the perspective of deploying them; it is just the container and the environment variables you deploy into the one generic chart. I see it that the app is the pure container with none of the configurations to run it and the Kubernetes configuration to run it is "application infrastructure". So I have a Helm chart for each type of application infrastructure. I don't want each bit of application infrastructure to be hand-rolled I want to cookie cutter create it using Helm.

Now we have covered all the Helm mechanics we are left with the question "how do we manage all our configuration within git?". We use different repo for different purposes:

1. Each microservice has its own repo. We use CircleCI to build the code into an image that is configured via environment variables. We don't store any Helm or Kubernetes config in this repo. We do have a properties file that names the latest version of the `deployer` chart this code is compatible with.
2. We have a GitHub repo that holds the source of the generic Helm charts that are used to deploy all our apps and secrets. These don't actually change very often now they are fairly stable. CircleCI builds and releases the charts as tar.gz onto the GitHub releases page for this repo.
3. We have a git repo per environment. This has a folder per deployed microservice within that environment. Each contains the `yaml` file to pass to the helm deploy encrypted with `git secret`. It has a folder per deployed `secret` within that environment. Each contains a `yaml` file that is `git secret` encrypted. Ditto for `configmaps`.

Why a repo per environment? A branch per environment sounds more normal. Only a database credential for the live environment never moves between environments. As we use a separate repo for live we can give it separate permissions. We might let all developers go wild in the `dev-environment` repo we have a bit more discipline with the `live-environment` repo.

We then have CircleCI do automated deployments of configuration. This works as Helm data detects changes and doesn't deploy a change to any Kubrernetes object if the latest yaml it generates is already deployed. To change a password in live you change it in the `git secret` encrypted yaml in the live repo and apply a release tag. CircleCI pattern matches on the tag name to see that its a release tag and it runs the deploy step. That uses a container with the helm client to run a bash script that loops over all app folders, secret folders and configmap folders running Helm for the right chart and correct yaml. Helm will only apply updates where there are changes within git.

> **Comments:**
> **username_0**: Thank you. Great info here. The k8s and docker env was built before i started and they arent using it yet.  My real question is do i replace octopus or take advantange of whatvis already There and use tge new festures that allow you to deploy to k8s. This customer really has no vlue what they wsnt. But the seem to believe they shoukd replace octopus alltogether with k8s and docker. I think we use what we have and ad k8s and docker to octopus  
> **username_1**: old microsoft tech wasn't 12factor friendly. octopus was liked by teams i knew as solved repackaging same code "built once" for each env. that isn't a problem with .net core and k8s. you only got a few tentacles for free their so they liked octopus but couldn't use it on microservices as not enough free capacity. yet if it is working for your team today why change it immediately? how config is managed and whether the same container moves through envs that devs can run locally overriding vars is most important part. get it working that way and remove octopus last if its not helping.  
> **username_1**: for "build once, deploy many times" I strongly recommend Helm. it doesn't really matter what is invoking Helm. we use a CircleCI deploy step, in the past, I have had VSTS deploying to k8s with bash script. developers have strong opinions the about "build" solution and SCM but are not bothered about deploy tooling if it "just works". CircleCI works for us as it integrates with GitHub and lets me run a helm container in the deploy step with a simple bash script.  
> **username_0**: Thank you. Its been very difficult finding info. Mostly because people usually choise one or another.as of now i am creating a new octopus server to test the octopus to k8s deployment since this team knows tfs and octopus.  They no nothing about hhelm k8s or docker. Once i hsve a test env. I will do 1 in octopus and one without octopus  
> **username_0**: Im standing up a new env in parallel to what thry currently use. So i can build it however i want. So tfs to dockerfile, build .net in container" image push to local registry. Deploy to k8s via helm  
> **username_1**: so what SCM and build CI? GitHub and ...?  
> **username_0**: As i am new to helm one of the things i am a little confused by is..the nuget pkgs.  I need to have tfs deploy to a dockerfile, run the dotnet build there,  image it then have helm pull the image from our local docker registry? This would eliminate octopus  
> **username_0**: Since all of the repos are in tfs and the mgr uses tfs to create work items and perform his mgnt duties along with security policies. We are upgrading to tfs 2018. Each micro service is in its own git repo within tfs. So from tfs deploy to dockerfile insert env variables from octopus into dockerfile, image, push to docker registry.. After that, as i said im new to helm and i believe you may have answered this above. Helm would deploy the docker image to k8s.. I guess i really need to look at helm more  
> **username_1**: sure tfs makes the image ready to run. that’s build not deploy. to deploy you must run helm client against tiller server running in k8s. you don’t want to run helm client by hand “something” must run it to change k8s deployment config object to reference new image built by tfs. I used vsts (SaaS tfs) to do deployments with bash script run on Linux azure VMs that could call something like helm to do the deploy workflow.  
> **username_0**: So getting env variables out of octopus and into either 1 of thec2 options above. Strip them ftom octopus, put them in docker file or put them in k8s as secrets  
> **username_0**: I cant Thank you enough. You cleared up a lot of my questions. Tomortow i will spend more time learning helm better. But i think this is a great place to start. You may see me add a comment this week if ibrun into some confusion. Agsin thank you  
> **username_1**: default tfs/vso bash task only runs script in repo you build. so you cannot use my answer approach of having deploy scripts in another git repo. the solution is a free contributor bash task in the marketplace that lets you set the bash script code as text property of the task. so script is saved in the deploy workflow. I had it “curl $deploy_git_url | /bin/bash” where the “$deploy_git_url” was set at project level to point to my separate git repo of k8s deploy code. That script can run helm having been passed parameters from deploy pipeline of what is latest built container to deploy.  
> **username_1**: No-one has up voted my answer :*(  
> **username_1**: Wow still no upvote or acceptance of my answer.  

---

