<p>So what happens in first case is, as Dan's already said, there's no shell used and as such nothing to expand the <code>~</code>. Quoting the <a href="https://www.terraform.io/docs/providers/external/data_source.html#program" rel="noreferrer">documentation about program</a>:</p>

<blockquote>
  <p>Terraform does not execute the program through a shell</p>
</blockquote>

<p>On the second case, bash receive in stdin something like this:</p>

<pre><code>{ "-c": "ssh-keygen -y -f ~/.ssh/id_rsa" }
</code></pre>

<p>And this looks like a command block for bash, but -c is not a valid command.</p>

<p>What could work could be this kind of program (assuming no specific input):</p>

<pre><code>jq -n --arg pubkey "$(ssh-keygen -y -f ~/.ssh/id_rsa)" '{"pubkey":$pubkey}'
</code></pre>

<p>So something like this should work to get the key in <code>local_key["pubkey"]</code> if I understand the documentation properly:</p>

<pre><code>data "external" "local_key" {
  program = [
    "bash", "-c jq -n --arg pubkey \"$(ssh-keygen -y -f ~/.ssh/id_rsa)\" '{\"pubkey\":$pubkey}'"
  ]
}
</code></pre>

<p>There's a need to use bash for a one liner to take advantage of command subsitution. You can also do a .sh script like:</p>

<pre><code>#!/bin/sh
jq -n --arg pubkey "$(ssh-keygen -y -f ~/.ssh/id_rsa)" '{\"pubkey\":$pubkey}'
</code></pre>

<p>And call this script in the <code>program</code> parameter.</p>

CC BY-SA 3.0
2018-02-22T09:02:19.363Z
2018-02-22T09:02:19.363Z
Tensibai
<p>Note: this answer is based solely on documentation, I didn't try it.</p>

<p>From <a href="https://www.terraform.io/docs/providers/external/data_source.html" rel="nofollow noreferrer">External Data Source</a> (emphasis mine):</p>

<blockquote>
  <p>The following arguments are supported:</p>
  
  <ul>
  <li><a href="https://www.terraform.io/docs/providers/external/data_source.html#program" rel="nofollow noreferrer">program</a> - (Required) A list of strings, whose first element is the program to run and whose subsequent elements are optional command
  line arguments to the program. <strong>Terraform does not execute the
  program through a shell</strong>, so it is not necessary to escape shell
  metacharacters nor add quotes around arguments containing spaces.</li>
  </ul>
</blockquote>

<p>Expanding <code>~</code> to the home directory is a capability of the shell, not of <code>ssh-keygen</code> and since in the 1st example <code>ssh-keygen</code> is invoked, it literally attempts to open a file named <code>~/.ssh/id_rsa</code>. Which fails because a <code>~</code> directory doesn't exist (I'm not sure if that's what you mean by <code>presumably happens because ~ expansion doesn't</code>).</p>

<p>In the 2nd example you're passing the <code>ssh-keygen</code> command as a query, not as arguments to <code>bash</code>. Probably what's executed is not what you're expecting.</p>

<p>I'd try:</p>

<pre><code>data "external" "local_key" {
  program = [
    "bash", "-c", "ssh-keygen", "-y", "-f ~/.ssh/id_rsa"
  ]
}
</code></pre>

<p>But I'm not sure if <code>bash</code> itself (or <code>ssh-keygen</code> for that matter) follows the External Program Protocol required  (also on the above referenced page). You may need to write your own script to wrap the cmd you desire while also providing the data according to the protocol. Invoke that script as the program instead of <code>bash</code> or <code>ssh-keygen</code>.</p>

<p>Finally, the entire external data source page doesn't appear to suggest that <code>export</code> would be designed for interactive programs and <code>ssh-keygen -y -f ~/.ssh/id_rsa</code> asks for the user passphrase interactively. YMMV.</p>

CC BY-SA 3.0
2018-02-22T08:26:48.883Z
It is not compatible, terraform give input in json and waits for a json output to be used in the query block. But I don't get the question goal...
Tensibai
CC BY-SA 3.0
2018-02-22T08:29:07.707Z
Yep, that's what I suspect as well - see my 2nd to last paragraph.
Dan Cornilescu
CC BY-SA 3.0
2018-02-22T08:34:44.850Z
It was a confirmation of your suspicion :) the terraform documentation is clear on this point:  "The program must then produce a valid JSON object on stdout, ..."
Tensibai
CC BY-SA 3.0
2018-02-22T08:17:56.200Z
2018-02-22T08:17:56.200Z
Dan Cornilescu
<p>Have you considered other options? e.g. A discussion my peers and I engaged recently was "time and place". We use mostly Ansible for deploying servers. Yet some of our team wanted to use Terraform and others argued Cloud Formation. Our discussion concluded why not let Terraform do what it does best and leave the rest to Ansible? In that within the <a href="https://www.terraform.io/docs/providers/template/index.html" rel="nofollow noreferrer">Terraform user_data</a> section you can call your Ansible Playbooks.  My '<strong>limited</strong>' understanding of Terraform is that its purpose is to deploy services to a limited extent and not be a full on automation drop-in replacement for services like Ansible, Chef, Puppet, etc. As a side note, Ansible has a module to handle the ssh keys generation and plenty more.</p>

CC BY-SA 3.0
2018-02-22T16:50:52.733Z
I wasn't the person who down-voted you, but I think I can see why you were. Your points are valid, Terraform should be used in conjunction with a configuration management (CM) tool like Ansible. Terraform is not a replacement. However, the OP's question was about a specific function of the Terraform program. There's nothing from the OP that tells us _why_ they are running this ssh-keygen or how its used in their pipeline. Its entirely possible that running this with `terraform` _actually is_ the best place for their use case.
BoomShadow
CC BY-SA 3.0
2018-02-23T07:11:27.887Z
I use terraform for orchestration and ansible configuration management, this particular issue is just about having my dropping in the initial SSH key in an automated fashion so ansible can connect. :P
Kit Sunde
CC BY-SA 3.0
2018-03-01T17:04:02.027Z
@KitSunde; In my use case of Ansible we have it create the SSH key(s) for all servers and store them in out management server. Is it possible to use Ansible to create this and possibly pass that via file back to Terraform? Apologies for the inexperienced question.
Steven K7FAQ
CC BY-SA 3.0
2018-02-22T16:17:52.567Z
2018-02-22T16:17:52.567Z
Steven K7FAQ
<p>The best way to achieve what you're trying to do (generate an openssh public key from an <code>id_rsa</code> private key) is using the <a href="https://www.terraform.io/docs/providers/tls/d/public_key.html" rel="nofollow noreferrer"><code>tls_public_key</code></a> data provider built-in to terraform. Using the <code>external</code> provider should be considered an option of last resort because it introduces dependencies on the local OS which may not be portable. It's like an escape hatch. </p>

<p>You can use it like this:</p>

<pre><code>data "tls_public_key" "example" {
  private_key_pem = "${file("~/.ssh/id_rsa")}"
}
</code></pre>

<p>Then to get the public key suitable for openssh, you can access</p>

<pre><code>"${data.tls_public_key.example.public_key_openssh}" 
</code></pre>

<p>Lastly, if you are open to generating a new SSH key pair, you can use the <a href="https://github.com/cloudposse/terraform-tls-ssh-key-pair" rel="nofollow noreferrer"><code>terraform-tls-ssh-key-pair</code></a> module which will write the keys to a new file and exposes a <code>public_key</code> output with the openssh public key.</p>

CC BY-SA 4.0
2021-01-20T18:49:26.703Z
Just keep in mind that Terraform does not support DSA and ED25519 certs. DSA probably will never be supported but ED25519 seems is on the roadmap.
Dzintars
CC BY-SA 4.0
2018-07-27T04:30:30.983Z
2018-07-27T05:06:31.547Z
2018-07-27T05:06:31.547Z
Erik Osterman
<p>I seem to be confused about how <a href="https://www.terraform.io/docs/providers/external/index.html" rel="noreferrer">external</a> works. I tried:</p>

<pre><code>data "external" "local_key" {
  program = [
    "ssh-keygen", "-y", "-f ~/.ssh/id_rsa"
  ]
}
</code></pre>

<p>This gives me:</p>

<blockquote>
  <p>failed to execute "ssh-keygen":  ~/.ssh/id_rsa: No such file or directory</p>
</blockquote>

<p>Which presumably happens because <code>~</code> expansion doesn't. <code>ssh-keygen -y -f ~/.ssh/id_rsa</code> works normally. So instead I tried invoking bash like:</p>

<pre><code>data "external" "local_key" {
  program = [
    "bash"
  ]

  query {
    "-c" = "ssh-keygen -y -f ~/.ssh/id_rsa"
  }
}
</code></pre>

<p>I'm still getting the same issue. I realize I can just pass the output as a var to terraform from the outside, but I'm still curious what the solution is.</p>

CC BY-SA 3.0
2018-02-22T05:36:16.207Z
Why insisting in using ~ where you can use $HOME?
Tensibai
CC BY-SA 3.0
2018-02-22T06:29:12.297Z
@Tensibai Good point, it's just habit. :P I still don't understand why calling bash won't end up doing the expansion though.
Kit Sunde
CC BY-SA 3.0
2018-02-22T07:14:05.887Z
Maybe because there's no or invalid login when Terraform execute the command? But external is supposed to be an interaction with the remote machine, not running a command on the remote machine from what I understand from the documentation.
Tensibai
CC BY-SA 3.0
2018-02-22T07:16:32.560Z
I'm not sure to really understand what you're trying to achieve in fact.
Tensibai
CC BY-SA 3.0
2018-02-22T08:28:32.487Z
@Tensibai External runs local programs, it doesn't imply interaction with anything remote. It's an external data source relative to terraforms execution environment, not relative to the machine terraform runs on. "external is a special provider that exists to provide an interface between Terraform and external programs." I'm just getting the public key for the local private key, the public key doesn't exist on the machine I'm running terraform on.
Kit Sunde
CC BY-SA 3.0
2018-02-22T08:33:19.867Z
So that's just a XY problem, getting the key in a difficult manner instead of extracting it once for all in a file and using this file as usual...
Tensibai
CC BY-SA 3.0
2018-02-22T08:39:14.773Z
@Tensibai What is a XY problem? Like I mentioned in my question, I can solve the issue just fine by passing it as a variable from the outside. I wanted to understand *why* I couldn't solve it in the manner I first tried.
Kit Sunde
CC BY-SA 3.0
2018-02-22T08:43:38.820Z
The why is simply because the external protocol waits a json output to be used later: "The program must then produce a valid JSON object on stdout, which will be used to populate the result attribute exported to the rest of the Terraform configuration. " (quoted from the [documentation](https://www.terraform.io/docs/providers/external/data_source.html)). ssh-keygen doesn't output a json object
Tensibai
CC BY-SA 3.0
2018-02-22T08:46:41.517Z
I mean that running `ssh-keygen -y -f ~/.ssh/id_rsa > ~/.ssh/id_rsa.pub` outside of terraform and using this .pub file in terraform would be far easier IMHO.
Tensibai
CC BY-SA 3.0
2018-02-22T08:48:44.827Z
And your second try doesn't work because the query block content will be passed as a json file to your program, bash doesn't read arguments from json, so as the documentation show you should create a script to aprse the input json, run the command and then format the output to be json format.
Tensibai
<terraform>
How can I get terraforms extern to execute `ssh-keygen -y -f ~/.ssh/id_rsa`?
Kit Sunde
# How can I get terraforms extern to execute `ssh-keygen -y -f ~/.ssh/id_rsa`?

(Asked by: username_0 on 2018-02-22)

I seem to be confused about how external works. I tried:
data "external" "local_key" {
  program = [
    "ssh-keygen", "-y", "-f ~/.ssh/id_rsa"
  ]
}

This gives me:

failed to execute "ssh-keygen":  ~/.ssh/id_rsa: No such file or directory

Which presumably happens because ~ expansion doesn't. ssh-keygen -y -f ~/.ssh/id_rsa works normally. So instead I tried invoking bash like:
data "external" "local_key" {
  program = [
    "bash"
  ]

  query {
    "-c" = "ssh-keygen -y -f ~/.ssh/id_rsa"
  }
}

I'm still getting the same issue. I realize I can just pass the output as a var to terraform from the outside, but I'm still curious what the solution is.

**Tags:** terraform

> **Comments:**
> **username_1**: Why insisting in using ~ where you can use $HOME?  
> **username_0**: @username_1 Good point, it's just habit. :P I still don't understand why calling bash won't end up doing the expansion though.  
> **username_1**: Maybe because there's no or invalid login when Terraform execute the command? But external is supposed to be an interaction with the remote machine, not running a command on the remote machine from what I understand from the documentation.  
> **username_1**: I'm not sure to really understand what you're trying to achieve in fact.  
> **username_0**: @username_1 External runs local programs, it doesn't imply interaction with anything remote. It's an external data source relative to terraforms execution environment, not relative to the machine terraform runs on. "external is a special provider that exists to provide an interface between Terraform and external programs." I'm just getting the public key for the local private key, the public key doesn't exist on the machine I'm running terraform on.  
> **username_1**: So that's just a XY problem, getting the key in a difficult manner instead of extracting it once for all in a file and using this file as usual...  
> **username_0**: @username_1 What is a XY problem? Like I mentioned in my question, I can solve the issue just fine by passing it as a variable from the outside. I wanted to understand *why* I couldn't solve it in the manner I first tried.  
> **username_1**: The why is simply because the external protocol waits a json output to be used later: "The program must then produce a valid JSON object on stdout, which will be used to populate the result attribute exported to the rest of the Terraform configuration. " (quoted from the [documentation](https://www.terraform.io/docs/providers/external/data_source.html)). ssh-keygen doesn't output a json object  
> **username_1**: I mean that running `ssh-keygen -y -f ~/.ssh/id_rsa > ~/.ssh/id_rsa.pub` outside of terraform and using this .pub file in terraform would be far easier IMHO.  
> **username_1**: And your second try doesn't work because the query block content will be passed as a json file to your program, bash doesn't read arguments from json, so as the documentation show you should create a script to aprse the input json, run the command and then format the output to be json format.  

---

## 4 answers

### Answer by username_5 on 2018-07-27. Score: 4 | Accepted answer

The best way to achieve what you're trying to do (generate an openssh public key from an id_rsa private key) is using the tls_public_key data provider built-in to terraform. Using the external provider should be considered an option of last resort because it introduces dependencies on the local OS which may not be portable. It's like an escape hatch. 
You can use it like this:
data "tls_public_key" "example" {
  private_key_pem = "${file("~/.ssh/id_rsa")}"
}

Then to get the public key suitable for openssh, you can access
"${data.tls_public_key.example.public_key_openssh}" 

Lastly, if you are open to generating a new SSH key pair, you can use the terraform-tls-ssh-key-pair module which will write the keys to a new file and exposes a public_key output with the openssh public key.

> **Comments:**
> **username_6**: Just keep in mind that Terraform does not support DSA and ED25519 certs. DSA probably will never be supported but ED25519 seems is on the roadmap.  

---

### Answer by username_1 on 2018-02-22. Score: 6

So what happens in first case is, as Dan's already said, there's no shell used and as such nothing to expand the ~. Quoting the documentation about program:

Terraform does not execute the program through a shell

On the second case, bash receive in stdin something like this:
{ "-c": "ssh-keygen -y -f ~/.ssh/id_rsa" }

And this looks like a command block for bash, but -c is not a valid command.
What could work could be this kind of program (assuming no specific input):
jq -n --arg pubkey "$(ssh-keygen -y -f ~/.ssh/id_rsa)" '{"pubkey":$pubkey}'

So something like this should work to get the key in local_key["pubkey"] if I understand the documentation properly:
data "external" "local_key" {
  program = [
    "bash", "-c jq -n --arg pubkey \"$(ssh-keygen -y -f ~/.ssh/id_rsa)\" '{\"pubkey\":$pubkey}'"
  ]
}

There's a need to use bash for a one liner to take advantage of command subsitution. You can also do a .sh script like:
#!/bin/sh
jq -n --arg pubkey "$(ssh-keygen -y -f ~/.ssh/id_rsa)" '{\"pubkey\":$pubkey}'

And call this script in the program parameter.


---

### Answer by username_2 on 2018-02-22. Score: 1

Note: this answer is based solely on documentation, I didn't try it.
From External Data Source (emphasis mine):

The following arguments are supported:

program - (Required) A list of strings, whose first element is the program to run and whose subsequent elements are optional command
  line arguments to the program. Terraform does not execute the
  program through a shell, so it is not necessary to escape shell
  metacharacters nor add quotes around arguments containing spaces.


Expanding ~ to the home directory is a capability of the shell, not of ssh-keygen and since in the 1st example ssh-keygen is invoked, it literally attempts to open a file named ~/.ssh/id_rsa. Which fails because a ~ directory doesn't exist (I'm not sure if that's what you mean by presumably happens because ~ expansion doesn't).
In the 2nd example you're passing the ssh-keygen command as a query, not as arguments to bash. Probably what's executed is not what you're expecting.
I'd try:
data "external" "local_key" {
  program = [
    "bash", "-c", "ssh-keygen", "-y", "-f ~/.ssh/id_rsa"
  ]
}

But I'm not sure if bash itself (or ssh-keygen for that matter) follows the External Program Protocol required  (also on the above referenced page). You may need to write your own script to wrap the cmd you desire while also providing the data according to the protocol. Invoke that script as the program instead of bash or ssh-keygen.
Finally, the entire external data source page doesn't appear to suggest that export would be designed for interactive programs and ssh-keygen -y -f ~/.ssh/id_rsa asks for the user passphrase interactively. YMMV.

> **Comments:**
> **username_1**: It is not compatible, terraform give input in json and waits for a json output to be used in the query block. But I don't get the question goal...  
> **username_2**: Yep, that's what I suspect as well - see my 2nd to last paragraph.  
> **username_1**: It was a confirmation of your suspicion :) the terraform documentation is clear on this point:  "The program must then produce a valid JSON object on stdout, ..."  

---

### Answer by username_3 on 2018-02-22. Score: 0

Have you considered other options? e.g. A discussion my peers and I engaged recently was "time and place". We use mostly Ansible for deploying servers. Yet some of our team wanted to use Terraform and others argued Cloud Formation. Our discussion concluded why not let Terraform do what it does best and leave the rest to Ansible? In that within the [Terraform user\_data](https://www.terraform.io/docs/providers/template/index.html) section you can call your Ansible Playbooks. My '**limited**' understanding of Terraform is that its purpose is to deploy services to a limited extent and not be a full on automation drop-in replacement for services like Ansible, Chef, Puppet, etc. As a side note, Ansible has a module to handle the ssh keys generation and plenty more.

> **Comments:**
> **username_4**: I wasn't the person who down-voted you, but I think I can see why you were. Your points are valid, Terraform should be used in conjunction with a configuration management (CM) tool like Ansible. Terraform is not a replacement. However, the OP's question was about a specific function of the Terraform program. There's nothing from the OP that tells us _why_ they are running this ssh-keygen or how its used in their pipeline. Its entirely possible that running this with `terraform` _actually is_ the best place for their use case.  
> **username_0**: I use terraform for orchestration and ansible configuration management, this particular issue is just about having my dropping in the initial SSH key in an automated fashion so ansible can connect. :P  
> **username_3**: @KitSunde; In my use case of Ansible we have it create the SSH key(s) for all servers and store them in out management server. Is it possible to use Ansible to create this and possibly pass that via file back to Terraform? Apologies for the inexperienced question.  

---

