<p>Packer has taken an active decision not to build in such capabilities. This problem is much better solved by other tools such as jq, etc. We think the best way to leverage Packer is to wrap it if you need more ways to create the template. </p>

<p>It's very easy to create json in any programming language which gives you unlimited expressiveness when creating templates. One great way to manipulate templates is to define them in <em>jinja2</em> and preprocess it with Python.</p>

CC BY-SA 4.0
2018-11-24T10:27:15.817Z
2018-12-07T10:45:00.087Z
2018-12-07T10:45:00.087Z
Rickard von Essen
<p>Utilizing the <a href="https://www.packer.io/docs/from-1.5" rel="nofollow noreferrer">HCL2</a> syntax is definitely the correct approach here. You can use the different building <a href="https://www.packer.io/docs/from-1.5/blocks" rel="nofollow noreferrer">blocks</a> to define different source blocks and then assign sources to the builders and even conditionally execute the provisioners and <a href="https://www.packer.io/docs/from-1.5/blocks/build/post-processors" rel="nofollow noreferrer">post processors</a> on specific sources using <a href="https://www.packer.io/docs/from-1.5/onlyexcept" rel="nofollow noreferrer">only/except</a>.</p>
<p>Using this approach you can easily accomplish what you are looking for. Here is a rough example adapted from the hashicorp documentation with multiple sources and ansible:</p>
<pre><code>source &quot;amazon-ebs&quot; &quot;first-example&quot; {
    ...
}
source &quot;amazon-ebs&quot; &quot;second-example&quot; {
    ...
}
build {
  name = &quot;my_build&quot;
  sources [
    &quot;source.amazon-ebs.first-example&quot;,&quot;source.amazon-ebs.second-example&quot;
  ]

# I run on both sources 
provisioner &quot;ansible&quot; {
        playbook_file = &quot;common.yml&quot;
  }

# Only runs on first example
provisioner &quot;ansible&quot; {
    only = [&quot;source.amazon-ebs.first-example&quot;]
    playbook_file = &quot;playbook-a.yml&quot;
  }

# Never runs on second example
provisioner &quot;ansible&quot; {
    except = [&quot;source.amazon-ebs.second-example&quot;]
    playbook_file = &quot;playbook-b.yml&quot;
  }
}
</code></pre>
<p>Depending on your source blocks and the builder being used, this opens up the ability to template multi-cloud, arch, distribution and purposed images from the same packer build command.</p>

CC BY-SA 4.0
2024-12-10T21:35:22.373Z
One thing that I'd like to figure out when doing this is for the "I run on both sources", how to specify the "user" var for example.  This is often different for each source (eg: rocky/ec2-user/ubuntu)
openCivilisation
CC BY-SA 4.0
2024-12-11T22:15:55.137Z
The easiest way to do this is to create a variable for user, pass it into your source and then define a source specific variable file like rocky.pkvars.hcl for each source and pass it into the packer build command and orchestrate that for each source using your ci system. It also helps to debug and limit the blast radius of failures to separate them this way.
zm31
CC BY-SA 4.0
2024-12-11T22:56:41.770Z
alternatively, you can expand on doing something like this:

variable "username_map" {
  type = map(string)
  default = {
    "rocky" = "rockyuser"
    "rhel"  = "redhatuser"
  }
}

source "null" "rocky" {
  communicator = "none"
}

source "null" "rhel" {
  communicator = "none"
}

build {
  sources = [
    "source.null.rhel",
    "source.null.rocky"
  ]

  provisioner "shell-local" {
    inline = [
      "echo ${var.username_map[source.name]}"
    ]
  }
}

This echos "redhatuser" when running the rhel source and "rockeruser" when using that source
zm31
CC BY-SA 4.0
2024-12-11T23:06:26.017Z
see https://developer.hashicorp.com/packer/docs/templates/hcl_templates/contextual-variables
zm31
CC BY-SA 4.0
2024-12-12T02:08:45.993Z
That documentation is very deceptive - its not actually possible to combine sources with something like an amazon-ebs plugin, probably the only reason why it would be usefull.  You can't event add extra vars to those sources, they aren't allowd.  But what does work is using a map, to lookup values via source.name, thats about as good as it got.
openCivilisation
CC BY-SA 4.0
2021-02-06T19:38:59.417Z
2021-02-06T19:38:59.417Z
zm31
<p>This is what <a href="https://www.packer.io/docs/from-1.5/blocks/source" rel="nofollow noreferrer">source blocks</a> are for in HCL2. I've been using this approach recently (define builder in a source block, then use it from various build blocks, filling in the variables as needed) and believe this to be the new best practice / idiom.</p>

CC BY-SA 4.0
2024-12-11T21:35:19.677Z
As far as I can tell you cannot combine source blocks with an amazon-ebs source block when I tried.
openCivilisation
CC BY-SA 4.0
2021-02-02T09:01:35.077Z
2021-02-02T09:01:35.077Z
timblaktu
<p>So far, I've taken to defining the default <code>builders</code> and <code>provisioners</code> we use, with <a href="https://www.packer.io/docs/templates/user-variables.html" rel="nofollow noreferrer">user variables</a> interpolated in where required, eg:</p>

<pre><code> "type": "amazon-ebs",
 "instance_type": "t2.micro",
 "ami_name": "{{ user `ami-name` }} 
</code></pre>

<p>I've got files called eg. <code>packer-builders.json</code> and <code>packer-provisioners.json</code>, and then all the separate Packer templates in their own directories.</p>

<p>I've then written a quick Bash script that takes the name of the template one wants to build, finds all the <code>packer-*.json</code> files, and uses <code>jq --slurp</code> to merge them in (eg. <code>jq --slurp '.[0] * .[1] * .[2]' $LIST_OF_FILES</code>) with the supplied template at the end. It saves the output and sends that along to <code>packer build</code>.</p>

<p>As a bonus, I'm also using <code>jsmin</code> before hitting <code>jq</code> so that I can include comments in any of the JSON files.</p>

<p>It works well, but I'm not a <em>massive</em> fan of it because it's custom and it adds an additional barrier for entry for others coming across it (much more to consider than just <code>packer build something.json</code>!</p>

<p>So I'm really keen for additional answers here if there's a better way to do this.</p>

CC BY-SA 4.0
2018-06-18T02:25:26.930Z
2018-06-18T02:25:26.930Z
Tim Malone
<p>I'm creating about 10-12 Packer templates which almost all work the same way. Same builder (Amazon EBS), with some small variations in AMI names, and almost the same provisioner (Ansible Remote) - sometimes with additional variables and sometimes multiple playbooks.</p>

<p>But, much of the Packer templates remain the same.</p>

<p>I know I can use variables to get some of the changeable values into one place in the file. But I'm keen to cut down on all of the copy-pasta in the templates.</p>

<p>I could do this with a wrapper shell script utilising <code>jq</code> or something. But each complexity I add gives something else for others to learn. Try as I might, I can't seem to find a 'blessed' idiomatic way of doing this with Packer.</p>

<p>Is there an idiomatic way of doing this - or at least a generally accepted way?</p>

CC BY-SA 4.0
2023-07-04T08:00:21.960Z
Related issue: https://github.com/hashicorp/packer/issues/9154
stackprotector
<packer>
Is there an idiomatic way to create reusable Packer templates?
Tim Malone
# Is there an idiomatic way to create reusable Packer templates?

(Asked by: username_0 on 2018-06-17)

I'm creating about 10-12 Packer templates which almost all work the same way. Same builder (Amazon EBS), with some small variations in AMI names, and almost the same provisioner (Ansible Remote) - sometimes with additional variables and sometimes multiple playbooks.

But, much of the Packer templates remain the same.

I know I can use variables to get some of the changeable values into one place in the file. But I'm keen to cut down on all of the copy-pasta in the templates.

I could do this with a wrapper shell script utilising `jq` or something. But each complexity I add gives something else for others to learn. Try as I might, I can't seem to find a 'blessed' idiomatic way of doing this with Packer.

Is there an idiomatic way of doing this - or at least a generally accepted way?

**Tags:** packer

> **Comments:**
> **username_1**: Related issue: https://github.com/hashicorp/packer/issues/9154  

---

## 4 answers

### Answer by username_2 on 2018-11-24. Score: 3

Packer has taken an active decision not to build in such capabilities. This problem is much better solved by other tools such as jq, etc. We think the best way to leverage Packer is to wrap it if you need more ways to create the template.

It's very easy to create json in any programming language which gives you unlimited expressiveness when creating templates. One great way to manipulate templates is to define them in *jinja2* and preprocess it with Python.


---

### Answer by username_3 on 2021-02-06. Score: 3

Utilizing the HCL2 syntax is definitely the correct approach here. You can use the different building blocks to define different source blocks and then assign sources to the builders and even conditionally execute the provisioners and post processors on specific sources using only/except.
Using this approach you can easily accomplish what you are looking for. Here is a rough example adapted from the hashicorp documentation with multiple sources and ansible:
source "amazon-ebs" "first-example" {
    ...
}
source "amazon-ebs" "second-example" {
    ...
}
build {
  name = "my_build"
  sources [
    "source.amazon-ebs.first-example","source.amazon-ebs.second-example"
  ]

# I run on both sources 
provisioner "ansible" {
        playbook_file = "common.yml"
  }

# Only runs on first example
provisioner "ansible" {
    only = ["source.amazon-ebs.first-example"]
    playbook_file = "playbook-a.yml"
  }

# Never runs on second example
provisioner "ansible" {
    except = ["source.amazon-ebs.second-example"]
    playbook_file = "playbook-b.yml"
  }
}

Depending on your source blocks and the builder being used, this opens up the ability to template multi-cloud, arch, distribution and purposed images from the same packer build command.

> **Comments:**
> **username_4**: One thing that I'd like to figure out when doing this is for the "I run on both sources", how to specify the "user" var for example.  This is often different for each source (eg: rocky/ec2-user/ubuntu)  
> **username_3**: The easiest way to do this is to create a variable for user, pass it into your source and then define a source specific variable file like rocky.pkvars.hcl for each source and pass it into the packer build command and orchestrate that for each source using your ci system. It also helps to debug and limit the blast radius of failures to separate them this way.  
> **username_3**: alternatively, you can expand on doing something like this:  variable "username_map" {   type = map(string)   default = {     "rocky" = "rockyuser"     "rhel"  = "redhatuser"   } }  source "null" "rocky" {   communicator = "none" }  source "null" "rhel" {   communicator = "none" }  build {   sources = [     "source.null.rhel",     "source.null.rocky"   ]    provisioner "shell-local" {     inline = [       "echo ${var.username_map[source.name]}"     ]   } }  This echos "redhatuser" when running the rhel source and "rockeruser" when using that source  
> **username_3**: see https://developer.hashicorp.com/packer/docs/templates/hcl_templates/contextual-variables  
> **username_4**: That documentation is very deceptive - its not actually possible to combine sources with something like an amazon-ebs plugin, probably the only reason why it would be usefull.  You can't event add extra vars to those sources, they aren't allowd.  But what does work is using a map, to lookup values via source.name, thats about as good as it got.  

---

### Answer by username_5 on 2021-02-02. Score: 3

This is what [source blocks](https://www.packer.io/docs/from-1.5/blocks/source) are for in HCL2. I've been using this approach recently (define builder in a source block, then use it from various build blocks, filling in the variables as needed) and believe this to be the new best practice / idiom.

> **Comments:**
> **username_4**: As far as I can tell you cannot combine source blocks with an amazon-ebs source block when I tried.  

---

### Answer by username_0 on 2018-06-18. Score: 3

So far, I've taken to defining the default builders and provisioners we use, with user variables interpolated in where required, eg:
 "type": "amazon-ebs",
 "instance_type": "t2.micro",
 "ami_name": "{{ user `ami-name` }} 

I've got files called eg. packer-builders.json and packer-provisioners.json, and then all the separate Packer templates in their own directories.
I've then written a quick Bash script that takes the name of the template one wants to build, finds all the packer-*.json files, and uses jq --slurp to merge them in (eg. jq --slurp '.[0] * .[1] * .[2]' $LIST_OF_FILES) with the supplied template at the end. It saves the output and sends that along to packer build.
As a bonus, I'm also using jsmin before hitting jq so that I can include comments in any of the JSON files.
It works well, but I'm not a massive fan of it because it's custom and it adds an additional barrier for entry for others coming across it (much more to consider than just packer build something.json!
So I'm really keen for additional answers here if there's a better way to do this.


---

