<p>This is not a good way to go about this, terraform is a declarative language and you should always follow this principle.</p>
<p>Either terraform is managing a resource or terraform is querying an existing resource in order to provide information to manage a resource. If you allow scripting, very soon people will be implementing OOP, interaces, generics and it loses the whole focus.</p>
<p>With that out of the way, yes, you can do this.
What you need is the policy document, in that case, you can use some non standard strategies like an <a href="https://registry.terraform.io/providers/hashicorp/external/latest/docs/data-sources/data_source" rel="nofollow noreferrer">external data source</a></p>
<p>the external data source will return a string with instructions for what to do next. If the policy exists, it returns &quot;yes&quot; if it doesn't it returns &quot;nope&quot;</p>
<pre><code>data &quot;external&quot; &quot;iam-policy-exists&quot; {
  program = [&quot;bash&quot;, &quot;${path.module}/iam-policy-exists.sh&quot;]

  query = {
    policy = &quot;read-only-s3-${terraform.workspace}&quot;
  }
}
</code></pre>
<p>then, the bash script that will check if the iam policy exists</p>
<p>iam-policy-exists.sh</p>
<pre><code>#! /bin/bash

aws iam get-policy $1 #tries to fetch the policy

if $? == &quot;0&quot;; then #$? returns the result of the last command, 0 is success
    echo '{&quot;exists&quot;:&quot;yes&quot;}'
else
    echo '{&quot;exists&quot;:&quot;nope&quot;}'
fi
</code></pre>
<p>then you have a data query with a count</p>
<pre><code>data &quot;aws_iam_policy&quot; &quot;existing&quot; {
  #this count == 1 if the policy exists
  count = data.external.iam-policy-exists.exists == &quot;yes&quot; ? 1 : 0
  name = &quot;read-only-s3-${terraform.workspace}&quot;
}

resource &quot;aws_iam_policy&quot; &quot;read_only_policy&quot; {
  #this will be 1 if the policy doesn't exists
  count = data.external.iam-policy-exists.exists == &quot;nope&quot; ? 1 : 0
  name        = &quot;read-only-s3-${terraform.workspace}&quot;
  path        = &quot;/&quot;

  policy = data.aws_iam_policy_document.merged.json
}
</code></pre>
<p>So far, this is what you wanted, to either query or create an IAM policy.</p>
<p>Now, if you want to create a second merged policy you can get the resulting json</p>
<pre><code>locals {
    iam_policy_json = data.external.iam-policy-exists.exists == &quot;yes&quot; ? data.aws_iam_policy.existing[0].json : resource.aws_iam_policy.read_only_policy[0].json
}
</code></pre>
<p>and now, you have the json policy on local.iam_policy_json</p>
<p>Disclaimer, see how ugly things get when you try to do the procedural way?
I'm not trying to compel you to do this differently, but it would be a good idea to rethink your approach to this.</p>

CC BY-SA 4.0
2021-11-26T22:07:11.213Z
External data source is cool -- I have used null resource before, but hadn't found external -- it's nice and lightweight.

Completely agree with maintaining declarative nature of Terraform, which is really why I asked the question -- if I can't do it declaratively, it's a pretty strong indication I am doing something wrong.  That said, using external is really just a way to overcome the limitations of the implementation -- great power / great responsibility!

Anyway, thanks!
Tom Harrison
CC BY-SA 4.0
2021-11-23T20:03:16.510Z
2021-11-23T20:03:16.510Z
Magus
<p>Several of our terraform root modules need add to an existing policy that provides read-only permissions for S3 buckets -- each module has its own bucket.  I cannot figure out how to add to the existing policy, and merge new values (the new bucket) into it.</p>
<p>At first, we had just attached a new policy (three actually, one for development, staging, production) to the role.  But we can only attach a maximum of 20 policies.  So instead, I used the AWS CLI and just listed all the production buckets in one policy, staging, and dev likewise.  Cool so far.</p>
<p>Now I want to accomplish the same thing with terraform.</p>
<p>I have created a submodule that the root modules can all call, passing their bucket in.</p>
<p>I can define the new policy as</p>
<pre><code>data &quot;aws_iam_policy_document&quot; &quot;read_only&quot; {
  statement {
    actions = [
      &quot;s3:ListBucket&quot;,
      &quot;s3:GetObject&quot;,
      &quot;s3:GetBucketLocation&quot;,
    ]
    effect = &quot;Allow&quot;
    resources = [
      &quot;${var.bucket_arn}/*&quot;,
      var.bucket_arn,
    ]
  }
}
</code></pre>
<p>I can look up an <em>existing</em> policy with</p>
<pre><code>data &quot;aws_iam_policy&quot; &quot;existing&quot; {
  name = &quot;read-only-s3-${terraform.workspace}&quot;
}
</code></pre>
<p>and potentially merge that with the new policy using</p>
<pre><code>data &quot;aws_iam_policy_document&quot; &quot;merged&quot; {
  source_json = [
    data.aws_iam_policy_document.read_only.json,
    data.aws_iam_policy.existing.json
  ]
}
</code></pre>
<p>and then create the resource</p>
<pre><code>resource &quot;aws_iam_policy&quot; &quot;read_only_policy&quot; {
  name        = &quot;read-only-s3-${terraform.workspace}&quot;
  path        = &quot;/&quot;

  policy = data.aws_iam_policy_document.merged.json
}
</code></pre>
<p>But this fails if there's no <code>data.aws_iam_policy.existing</code> because AWS cannot find it so terraform fails with an error.</p>
<p>There's a great deal of documentation about how to merge policies <a href="https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document" rel="nofollow noreferrer">in the policy document reference</a>, but there's very little indication of how to determine if there's an existing policy.</p>
<p>I could recreate the entire policy if I could find the existing one and merge it with the new one, or even if I just knew the names of the buckets in the existing policy.  But the existing policies are created in other root modules.  And there's no way to look up buckets other than by their name (no <code>tags</code> or <code>filter</code> on  <code>data &quot;aws_s3_bucket&quot;</code>).</p>
<p>Is there a way to determine that there's an existing policy and get without throwing an error?  Then I could get its policy document and use the <code>source_json</code> approach.</p>
<p>Is there a way that I can merge a new policy document onto whatever exists (or not)?</p>
<p>Open to other approaches.  Really would prefer to keep this in IAM and not go to bucket policies because they are so opaque and decentralized.</p>

<amazon-web-services><terraform><aws-iam>
Terraform: add to existing AWS policy, or create policy if needed
Tom Harrison
# Terraform: add to existing AWS policy, or create policy if needed

(Asked by: username_0 on 2021-11-23)

Several of our terraform root modules need add to an existing policy that provides read-only permissions for S3 buckets -- each module has its own bucket.  I cannot figure out how to add to the existing policy, and merge new values (the new bucket) into it.
At first, we had just attached a new policy (three actually, one for development, staging, production) to the role.  But we can only attach a maximum of 20 policies.  So instead, I used the AWS CLI and just listed all the production buckets in one policy, staging, and dev likewise.  Cool so far.
Now I want to accomplish the same thing with terraform.
I have created a submodule that the root modules can all call, passing their bucket in.
I can define the new policy as
data "aws_iam_policy_document" "read_only" {
  statement {
    actions = [
      "s3:ListBucket",
      "s3:GetObject",
      "s3:GetBucketLocation",
    ]
    effect = "Allow"
    resources = [
      "${var.bucket_arn}/*",
      var.bucket_arn,
    ]
  }
}

I can look up an existing policy with
data "aws_iam_policy" "existing" {
  name = "read-only-s3-${terraform.workspace}"
}

and potentially merge that with the new policy using
data "aws_iam_policy_document" "merged" {
  source_json = [
    data.aws_iam_policy_document.read_only.json,
    data.aws_iam_policy.existing.json
  ]
}

and then create the resource
resource "aws_iam_policy" "read_only_policy" {
  name        = "read-only-s3-${terraform.workspace}"
  path        = "/"

  policy = data.aws_iam_policy_document.merged.json
}

But this fails if there's no data.aws_iam_policy.existing because AWS cannot find it so terraform fails with an error.
There's a great deal of documentation about how to merge policies in the policy document reference, but there's very little indication of how to determine if there's an existing policy.
I could recreate the entire policy if I could find the existing one and merge it with the new one, or even if I just knew the names of the buckets in the existing policy.  But the existing policies are created in other root modules.  And there's no way to look up buckets other than by their name (no tags or filter on  data "aws_s3_bucket").
Is there a way to determine that there's an existing policy and get without throwing an error?  Then I could get its policy document and use the source_json approach.
Is there a way that I can merge a new policy document onto whatever exists (or not)?
Open to other approaches.  Really would prefer to keep this in IAM and not go to bucket policies because they are so opaque and decentralized.

**Tags:** amazon-web-services, terraform, aws-iam


---

## 1 answers

### Answer by username_1 on 2021-11-23. Score: 3 | Accepted answer

This is not a good way to go about this, terraform is a declarative language and you should always follow this principle.
Either terraform is managing a resource or terraform is querying an existing resource in order to provide information to manage a resource. If you allow scripting, very soon people will be implementing OOP, interaces, generics and it loses the whole focus.
With that out of the way, yes, you can do this.
What you need is the policy document, in that case, you can use some non standard strategies like an external data source
the external data source will return a string with instructions for what to do next. If the policy exists, it returns "yes" if it doesn't it returns "nope"
data "external" "iam-policy-exists" {
  program = ["bash", "${path.module}/iam-policy-exists.sh"]

  query = {
    policy = "read-only-s3-${terraform.workspace}"
  }
}

then, the bash script that will check if the iam policy exists
iam-policy-exists.sh
#! /bin/bash

aws iam get-policy $1 #tries to fetch the policy

if $? == "0"; then #$? returns the result of the last command, 0 is success
    echo '{"exists":"yes"}'
else
    echo '{"exists":"nope"}'
fi

then you have a data query with a count
data "aws_iam_policy" "existing" {
  #this count == 1 if the policy exists
  count = data.external.iam-policy-exists.exists == "yes" ? 1 : 0
  name = "read-only-s3-${terraform.workspace}"
}

resource "aws_iam_policy" "read_only_policy" {
  #this will be 1 if the policy doesn't exists
  count = data.external.iam-policy-exists.exists == "nope" ? 1 : 0
  name        = "read-only-s3-${terraform.workspace}"
  path        = "/"

  policy = data.aws_iam_policy_document.merged.json
}

So far, this is what you wanted, to either query or create an IAM policy.
Now, if you want to create a second merged policy you can get the resulting json
locals {
    iam_policy_json = data.external.iam-policy-exists.exists == "yes" ? data.aws_iam_policy.existing[0].json : resource.aws_iam_policy.read_only_policy[0].json
}

and now, you have the json policy on local.iam_policy_json
Disclaimer, see how ugly things get when you try to do the procedural way?
I'm not trying to compel you to do this differently, but it would be a good idea to rethink your approach to this.

> **Comments:**
> **username_0**: External data source is cool -- I have used null resource before, but hadn't found external -- it's nice and lightweight.  Completely agree with maintaining declarative nature of Terraform, which is really why I asked the question -- if I can't do it declaratively, it's a pretty strong indication I am doing something wrong.  That said, using external is really just a way to overcome the limitations of the implementation -- great power / great responsibility!  Anyway, thanks!  

---

