#terraform - August 2019

August 2019
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago(edited)
Oh wow, hope that works!
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
report back...
Andy
Andyover 7 years ago(edited)
Hi everyone!
I am facing issues while trying to provision each ec2 instance with the below TF connection block:
connection {
    type     = "winrm"
    host     = "${aws_instance.ec2instance.*.private_ip[count.index]}"
    user     = "${var.username}"
    password = "${var.admin_password}"
    timeout  = "${var.timeout_tf}"
  }

The issue is with host = "${aws_instance.ec2instance.*.private_ip[count.index]}" line. I have tried all the below modifications however still get the same error message Error: Cycle: aws_instance.ec2instance[1], aws_instance.ec2instance[0]
1. host = "${aws_instance.ec2instance.*.private_ip}"
2. host = "${ element(aws_instance.ec2instance.*.private_ip, count.index) }"
3. host = "${aws_instance.ec2instance.*.private_ip[count.index]}"

Any pointers will be greatly appreciated.
Andy
Andyover 7 years ago(edited)
Additional information:
I have the count set to 2.
From the above point#1, If I put in host = "${aws_instance.ec2instance.1.private_ip}", It does not give me the error message however doesnt provision both the instances. Just provisions the instance with count.index=1.
Adrian
Adrianover 7 years ago
any workaround for
The "count" value depends on resource attributes that cannot be determined
Adrian
Adrianover 7 years ago
I want to use output of MSK bootstrap servers to create R53 CNAME entries
loren
lorenover 7 years ago(edited)
don't reference an output/attribute in count? can only reference vars and locals (and the locals must be fully deterministic in advance, i.e. cannot themselves rely on outputs/attributes)
Adrian
Adrianover 7 years ago
@loren so is it possible to get output from e.g. resource "aws_msk_cluster" and use as count in another resource e.g. "aws_route53_record"
loren
lorenover 7 years ago
not as the count, no, not as far as i know, you'll always get count cannot be determined errors
loren
lorenover 7 years ago(edited)
you can use the output in an attribute on another resource, but not in count
loren
lorenover 7 years ago(edited)
and you can set the count some other way where the length is fully deterministic from a var or local without relying on an attr of a resource/data source
loren
lorenover 7 years ago(edited)
say you pass in a var that determines the number of nodes in your cluster... you can use that var to count aws_route53_record, and then reference the attrs of aws_msk_cluster in aws_route53_record attrs
Adrian
Adrianover 7 years ago
ye I saw workaround with bash echo
Adrian
Adrianover 7 years ago
# Verify that the count matches the list
resource "null_resource" "verify_list_count" {
  provisioner "local-exec" {
    command = <<SH
if [ ${var.topic_arns_count} -ne ${length(var.topic_arns)} ]; then
  echo "var.topic_arns_count must match the actual length of var.topic_arns";
  exit 1;
fi
SH
  }
Adrian
Adrianover 7 years ago
smth like this 🙂
loren
lorenover 7 years ago
here's the issue with all the fun details, https://github.com/hashicorp/terraform/issues/12570
loren
lorenover 7 years ago
situation is somewhat improved in tf 0.12, so may run into the error less frequently, but it's still a problem
Adrian
Adrianover 7 years ago
thnx for answer
Adrian
Adrianover 7 years ago
I'm stupid, already there number_of_broker_nodes
ivano
ivanoover 7 years ago
hello @here anyone knows and can recommend good libvirt provider other then this one here > https://github.com/dmacvicar/terraform-provider-libvirt
sirhopcount
sirhopcountabout 7 years ago
Does anybody know how to reference an instance created via a google_compute_instance_group_manager? I'm creating a route via google_compute_route and I need to have the name of the instance created via the group so I can set next_hop_instance.
C
Cloud Posseabout 7 years ago
:zoom: Join us for "Office Hours" every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Aug 14, 2019 11:30AM.

👉️ Register for Webinar

:slack: #office-hours (our channel)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
we've just published our first EMR module (by @Andriy Knysh (Cloud Posse)) https://registry.terraform.io/modules/cloudposse/emr-cluster/aws/0.1.0
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
We've been inconsistent about recording them
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
if we do, they'll be posted to #office-hours
Andy
Andyabout 7 years ago
NP. Thanks!
Andy
Andyabout 7 years ago(edited)
I am facing an issue with provisioning multiple Windows EC2 instances using Terraform.
This is an overview of my TF template (Oversimplified with syntax ignored):
Template Overview
Resource EC2 {
count = variable

connection 
type     = "winrm"
host     = "${self.private_ip}"
user     = "${var.username}"
password = "${var.admin_password}"
timeout  = "${var.timeout_tf}"

provisioner remote-exec inline
    1. powershell.exe rename-computer -Machine is rebooted once this is run.
provisioner remote-exec inline
    2. powershell copy platform code from s3 bucket
    3. powershell.exe run DomainAdd.ps1 - Machine is rebooted once this is run.
provisioner remote-exec inline
    4. powershell.exe run PreDeploy.ps1 (DSC script)
}

- If I set the count of the instance to 1. All the above #1, #2, #3 and #4 provisioning steps work fine.
Issue:
If I set the count of the instance to anything other than 1 (e.g. 2), Terraform successfully runs #1, #2 and #3 on both the instances and runs #4 on ONLY ONE of the instances.

Observations:
- After running #3 on both the instances, the remote-exec is able to eastablish the connection with both the instances successfully however runs #4 on only one of the instances.
- Even after running #4 on one instance, it keeps on showing the following output unless I force TF to stop.
aws_instance.ec2instance[1]: Still creating... [7m0s elapsed]
aws_instance.ec2instance[0]: Still creating... [7m0s elapsed]
aws_instance.ec2instance[1]: Still creating... [7m10s elapsed]
aws_instance.ec2instance[0]: Still creating... [7m10s elapsed]
aws_instance.ec2instance[0]: Still creating... [7m20s elapsed]
aws_instance.ec2instance[1]: Still creating... [7m20s elapsed]
aws_instance.ec2instance[1]: Still creating... [7m30s elapsed]
aws_instance.ec2instance[0]: Still creating... [7m30s elapsed]
aws_instance.ec2instance[0]: Still creating... [7m40s elapsed]
aws_instance.ec2instance[1]: Still creating... [7m40s elapsed]
aws_instance.ec2instance[1]: Still creating... [7m50s elapsed]


Why is Terraform behaving inconsistently when the instance count is set to anything other than 1? Is there something I might be missing? Any suggestions/pointers will be greatly appreciated! TF_LOG are not showing anything useful.
Hetal S
Hetal Sabout 7 years ago
Need help in accessing values as a list element inside map values in terraform
Hetal S
Hetal Sabout 7 years ago
variable "controller_name" {
        type    = "list"
        default = [{
        z1      = ["EKS-controller1"]
        z2      = []
        z3      = ["EKS-controller1","EKS-controller2"]
        z4      = []
        }]
}
Hetal S
Hetal Sabout 7 years ago
this is how the variable looks like
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
@mmarseglia we converted the module to TF 0.12 https://github.com/cloudposse/terraform-aws-ecr/releases/tag/0.7.0
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
can you try with TF 0.12?
mmarseglia
mmarsegliaabout 7 years ago
i'm not sure all the modules I'm using have been converted. they weren't, last I checked.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
mmarseglia
mmarsegliaabout 7 years ago
i built a manifest using elasticbeanstalk app w/ a multidocker container.
mmarseglia
mmarsegliaabout 7 years ago
i am using 7 different modules. i think the elasticbeakstalk ones haven't been converted yet?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago(edited)
no, benstalk has not been converted yet
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
i mean that we converted https://github.com/cloudposse/terraform-aws-ecr/releases/tag/0.7.0 to TF 0.12 a few weeks ago and it did not throw any policy errors
mmarseglia
mmarsegliaabout 7 years ago
yes, i would like to use the new one. 🙂 you have done great work converting them in a short time.
mmarseglia
mmarsegliaabout 7 years ago
i'll figure out a way around this in the short term and look to upgrade that module to 0.7.0
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
@mmarseglia try to delete the statement = [] from https://github.com/cloudposse/terraform-aws-ecr/blob/0.6.1/main.tf#L124
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
(and we tested only 0.12 version recently)
Brandon Wilson
Brandon Wilsonabout 7 years ago
If someone from Cloudposse gets a chance to review a PR: https://github.com/cloudposse/terraform-aws-datadog-integration/pull/11
Milos Backonja
Milos Backonjaabout 7 years ago(edited)
Guys, I am looking how can I tag resources dynamically, and without repeating same block of code with one change.

# Resource 1
tags = merge(
    var.tags,
    map(
      "Name", format("dev-bastion-0%s.${var.domain}",count.index+1),
      "type", "bastion"
    )
  )

# Resource 2

tags = merge(
    var.tags,
    map(
      "Name", format("dev-app-0%s.${var.domain}",count.index+1),
      "type", "app"
    )
  )

any suggestions are welcome, thanks. Okay I see now that I could just abstract that with module 😄
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
erik
erikabout 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
I wonder what the backstory is here...
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
a) why did they launch with HCL support
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
b) why did they drop HCL support
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
strange that they even started with HCL support
loren
lorenabout 7 years ago
maybe the devs working on that feature were also supporting the terraform github provider, and thought, wouldn't it be neat if...? 😉
Rich Allen
Rich Allenabout 7 years ago(edited)
Hi all, this is probably a very dumb and novice questions, but I'm having a hard time understanding what I'm doing wrong here. As far as I can tell, when I try to attach a policy document using a role from the CP role repository, the base module is expecting a string?

data "aws_iam_policy_document" "s3_full_access" {
    statement {
      sid       = "FullAccess"
      effect    = "Allow"
      resources = [
        "arn:aws:s3:::${module.static-app.s3_bucket}",
        "arn:aws:s3:::${module.static-app.s3_bucket}/*"
      ]

      actions = [
        "s3:PutObject",
        "s3:PutObjectAcl",
        "s3:GetObject",
        "s3:DeleteObject",
        "s3:ListBucket",
        "s3:ListBucketVersions",
        "s3:ListBucketMultipartUploads",
        "s3:GetBucketLocation",
        "s3:AbortMultipartUpload",
      ]
    }
  }

  module "s3-write-role" {
    source     = "git::<https://github.com/cloudposse/terraform-aws-iam-role.git?ref=0.4.0>"

    enabled    = "true"
    namespace = "${var.namespace}"
    stage     = "${var.stage}"
    name      = "${var.name}"

    policy_description = "Allow S3 FullAccess"
    role_description   = "IAM role with permissions to perform actions on S3 resources"

    policy_documents = ["${data.aws_iam_policy_document.s3_full_access.json}"]
  }

I'm wondering if I'm missing something ovbious here, or not understanding how to use this module?
Rich Allen
Rich Allenabout 7 years ago
Follow up error

Error: Incorrect attribute value type

  on .terraform/modules/s3-write-role.aggregated_assume_policy/main.tf line 23, in data "aws_iam_policy_document" "zero":
  23:   override_json = "${element(local.policies, 0)}"

Inappropriate value for attribute "override_json": string required.
Joan Hermida
Joan Hermidaabout 7 years ago
You are missing something there
Joan Hermida
Joan Hermidaabout 7 years ago
Why error says _aggregated_assume_policy_ ?
Joan Hermida
Joan Hermidaabout 7 years ago
Oh, your source is a git tag
Rich Allen
Rich Allenabout 7 years ago
I'm sorry, I'm not understanding what the source of confusion is? Could you possibly rephrase?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
(I personally did not test the latest changes to the module, so can’t just say what’s the exact issue is)
Rich Allen
Rich Allenabout 7 years ago
That is the example I'm working from. I've looked through it several times. And can't see a difference. From what I can tell, my only real difference is I don't have an outputs.tf
Rich Allen
Rich Allenabout 7 years ago
is is required I have that file, so that the module exposes the outputs to the consumer?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
it does not work with one item in the list
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
since it checks only for >1
Rich Allen
Rich Allenabout 7 years ago
well so, I can provide and update, and I'm using 2
Rich Allen
Rich Allenabout 7 years ago
so the base + full admin, as the example suggests
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
policy_documents = [“${data.aws_iam_policy_document.s3_full_access.json}“]
Rich Allen
Rich Allenabout 7 years ago
the previous was to simplify, I will post an updated stanza
Rich Allen
Rich Allenabout 7 years ago
I'm now using 2 as the example suggests

provider "aws" {
  region = "${var.region}"
}

module "static-app" {
  source                   = "git::<https://github.com/cloudposse/terraform-aws-cloudfront-s3-cdn.git?ref=0.10.0>"
  namespace                = "${var.namespace}"
  stage                    = "${var.stage}"
  name                     = "${var.name}"
  aliases                  = "${var.aliases}"
  parent_zone_name         = "${var.parent_zone_name}"
  default_root_object      = "${var.default_root_object}"
  acm_certificate_arn      = "${var.acm_certificate_arn}"
  cors_allowed_headers     = ["GET", "HEAD"]
  cors_allowed_methods     = ["GET", "HEAD"]
  cors_allowed_origins     = ["*"]
}

data "aws_iam_policy_document" "s3_full_access" {
  statement {
    sid       = "FullAccess"
    effect    = "Allow"
    resources = ["arn:aws:s3:::${module.static-app.s3_bucket}/*"]

    actions = [
      "s3:PutObject",
      "s3:PutObjectAcl",
      "s3:GetObject",
      "s3:DeleteObject",
      "s3:ListBucket",
      "s3:ListBucketMultipartUploads",
      "s3:GetBucketLocation",
      "s3:AbortMultipartUpload",
    ]
  }
}

data "aws_iam_policy_document" "base" {
  statement {
    sid = "BaseS3Access"

    actions = [
      "s3:ListBucket",
      "s3:ListBucketVersions",
    ]

    resources = ["*"]
    effect    = "Allow"
  }
}

  module "s3-write-role" {
    source     = "git::<https://github.com/cloudposse/terraform-aws-iam-role.git?ref=0.4.0>"

    enabled    = "true"
    namespace = "${var.namespace}"
    stage     = "${var.stage}"
    name      = "${var.name}"

    policy_description = "Allow S3 FullAccess"
    role_description   = "IAM role with permissions to perform actions on S3 resources"

    policy_documents = [
      "${data.aws_iam_policy_document.base.json}",
      "${data.aws_iam_policy_document.s3_full_access.json}"
      ]
  }
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
working for you?
Rich Allen
Rich Allenabout 7 years ago
no this is the result of that update
Rich Allen
Rich Allenabout 7 years ago(edited)

17:04 $ terraform plan

Error: Incorrect attribute value type

  on .terraform/modules/s3-write-role.aggregated_assume_policy/main.tf line 23, in data "aws_iam_policy_document" "zero":
  23:   override_json = "${element(local.policies, 0)}"

Inappropriate value for attribute "override_json": string required.


Error: Incorrect attribute value type

  on .terraform/modules/s3-write-role.aggregated_policy/main.tf line 23, in data "aws_iam_policy_document" "zero":
  23:   override_json = "${element(local.policies, 0)}"

Inappropriate value for attribute "override_json": string required.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
what TF version are you using?
Rich Allen
Rich Allenabout 7 years ago
17:04 $ terraform -v
Terraform v0.12.2
+ provider.aws v2.22.0
+ provider.local v1.3.0
+ provider.null v2.1.2
+ provider.template v2.1.2
Joan Hermida
Joan Hermidaabout 7 years ago
Ok, I'm seeing something nasty
Rich Allen
Rich Allenabout 7 years ago
I hope it's not something that dumb like I forgot brew update
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
this module has not been updated to TF 0.12 yet https://github.com/cloudposse/terraform-aws-iam-role
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
that’s why the errors
Joan Hermida
Joan Hermidaabout 7 years ago
Joan Hermida
Joan Hermidaabout 7 years ago
Main file
Rich Allen
Rich Allenabout 7 years ago
so I'm a noobie here, @Andriy Knysh (Cloud Posse) where is that documented? I'm not seeing it but that could be true.
Joan Hermida
Joan Hermidaabout 7 years ago
At line 23, you will see the error
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
both modules were not converted to TF 0.12 yet
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
that’s why TF 0.12 throws the errors
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
we’ll get to it soon
Rich Allen
Rich Allenabout 7 years ago
@Joan Hermida I'm not sure I understand the context there
Joan Hermida
Joan Hermidaabout 7 years ago
Downgrade to the latest 0.11 version
Rich Allen
Rich Allenabout 7 years ago
@Andriy Knysh (Cloud Posse) could you just point out where I can see that? To avoid version mismatches in the future?
Joan Hermida
Joan Hermidaabout 7 years ago
Joan Hermida
Joan Hermidaabout 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
so for CloudPosse modules, the ones that were converted to TF 0.12 have hcl2 tag in the repo, e.g. https://github.com/cloudposse/terraform-aws-rds
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
and you can tell TF 0.11 from TF 0.12 by the syntax
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
TF 0.12 does not use any string interpolations https://github.com/cloudposse/terraform-aws-rds/blob/master/outputs.tf
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago(edited)
and uses real first-class types like bool, number, list(string), map(string) instead if strings like “string” and “list” https://github.com/cloudposse/terraform-aws-rds/blob/master/variables.tf#L19
Rich Allen
Rich Allenabout 7 years ago
ty, that's very helpful, I'm getting a few new errors related to the downgrade but I'll fix these up. Much appreciate the help.
Joan Hermida
Joan Hermidaabout 7 years ago
👍️
PePe Amengual
PePe Amengualabout 7 years ago(edited)
Hello, I’m using : “git::https://github.com/cloudposse/terraform-aws-alb-target-group-cloudwatch-sns-alarms.git?ref=tags/0.6.1” and I notice when using the newer alb module that target_group_name = “${module.alb.target_group_name}” and target_group_arn_suffix = “${module.alb.target_group_arn_suffix}” are not valid outputs anymore so it can’t be use with this cloudwatch-sns module, are you guys deprecating the use of the cloudwatch-sns-alarms or recommend something else ?
PePe Amengual
PePe Amengualabout 7 years ago
and now I think I found a bug , I’m getting this :
Error: Error running plan: 1 error occurred:
	* module.alb_ingress.local.target_group_arn: local.target_group_arn: Resource 'aws_lb_target_group.default' not found for variable 'aws_lb_target_group.default.arn'
PePe Amengual
PePe Amengualabout 7 years ago(edited)
PePe Amengual
PePe Amengualabout 7 years ago
locals {
  target_group_enabled = "${var.target_group_arn == "" ? "true" : "false"}"
  target_group_arn     = "${local.target_group_enabled == "true" ? aws_lb_target_group.default.arn : var.target_group_arn}"
}
PePe Amengual
PePe Amengualabout 7 years ago
If I pass the ARN or the module output still fails
PePe Amengual
PePe Amengualabout 7 years ago
unless I’m doing something really wrong
PePe Amengual
PePe Amengualabout 7 years ago(edited)
I made a copy of the module , removed the data resource and it does work, so I’m guessing this lies : https://github.com/cloudposse/terraform-aws-alb-ingress/blob/0.7.0/main.tf#L6
PePe Amengual
PePe Amengualabout 7 years ago
don’t really need to be there ? why making a lookup of something I’m already passing and creating that data resource that is not used in that tf ?
PePe Amengual
PePe Amengualabout 7 years ago
I guess is to check that the target group exists before continuing but in my case I used -target module.alb to make sure I had everything before continuing but somehow still fails
Leonard Wood
Leonard Woodabout 7 years ago
Hi everyone - new to the channel but was hoping to find a solution to a problem I am running into dealing with output variables from modules
Leonard Wood
Leonard Woodabout 7 years ago
I use the aws vpc terraform module, and there is a specific output that that gets created as a list, specifically the database subnet output. I am trying to reference this output as an input variable for an rds module
Leonard Wood
Leonard Woodabout 7 years ago
does anyone know how to properly reference an output list generated by a module, as an input variable for another module?
Leonard Wood
Leonard Woodabout 7 years ago
any guidance or direction would be sincerely appreciated
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
outputs types are the same regardless whether it’s a list or a string
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
and another example where the list output from the subnet module goes into the elasticsearch module https://github.com/cloudposse/terraform-aws-elasticsearch/blob/master/examples/complete/main.tf#L33
Leonard Wood
Leonard Woodabout 7 years ago
bingo!
Leonard Wood
Leonard Woodabout 7 years ago
thank you @Andriy Knysh (Cloud Posse)
PePe Amengual
PePe Amengualabout 7 years ago
any ideas on this ?
PePe Amengual
PePe Amengualabout 7 years ago
I know the posts from yesterday usually get buried….
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
what module are you using? can you share the code
PePe Amengual
PePe Amengualabout 7 years ago
mmm that did not work, my complete post is just a higher
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
PePe Amengual
PePe Amengualabout 7 years ago(edited)
that is exactly where I got the example :
module "alb_ingress" {
  #source            = "git::<https://github.com/cloudposse/terraform-aws-alb-ingress.git?ref=tags/0.7.0>"
  source            = "../terraform-aws-alb-ingress"
  name              = "${var.name}"
  namespace         = "${var.namespace}"
  stage             = "${var.stage}"
  attributes        = "${var.attributes}"
  vpc_id            = "${var.vpc_id}"
  port              = "${var.container_port}"
  health_check_path = "${var.health_check_path}"
  target_group_arn  = "${module.alb.default_target_group_arn}"

  # Without authentication, both HTTP and HTTPS endpoints are supported
  unauthenticated_listener_arns       = ["${module.alb.listener_arns}"]
  unauthenticated_listener_arns_count = 1

  # All paths are unauthenticated
  unauthenticated_paths    = ["/*"]
  unauthenticated_priority = "100"
}
PePe Amengual
PePe Amengualabout 7 years ago
locals {
  target_group_enabled = "${var.target_group_arn == "" ? "true" : "false"}"
  target_group_arn     = "${local.target_group_enabled == "true" ? aws_lb_target_group.default.arn : var.target_group_arn}"
}
PePe Amengual
PePe Amengualabout 7 years ago
when this evaluation happens , for some reason this :
data "aws_lb_target_group" "default" {
  arn = "${local.target_group_arn}"
}
PePe Amengual
PePe Amengualabout 7 years ago
can’t find the ALB
PePe Amengual
PePe Amengualabout 7 years ago
and I’m 100% sure the value is correct
PePe Amengual
PePe Amengualabout 7 years ago
I mean the arn
PePe Amengual
PePe Amengualabout 7 years ago
but that data resource is the one that fails but is not anywhere else int he code
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Did you provision the ALB?
PePe Amengual
PePe Amengualabout 7 years ago
yes first
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Look at the root modules ECS folder
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Did you try to run terraform apply second time?
PePe Amengual
PePe Amengualabout 7 years ago
yes
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Sometimes there race conditions
PePe Amengual
PePe Amengualabout 7 years ago
I used target module.alb
PePe Amengual
PePe Amengualabout 7 years ago
then I run the rest
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Where some resources are not created yet
PePe Amengual
PePe Amengualabout 7 years ago
I understand
PePe Amengual
PePe Amengualabout 7 years ago
I run target module.alb twice
PePe Amengual
PePe Amengualabout 7 years ago
then run the rest
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Try running the rest twice
PePe Amengual
PePe Amengualabout 7 years ago
I did too
PePe Amengual
PePe Amengualabout 7 years ago
I’m destroying everything again right now
PePe Amengual
PePe Amengualabout 7 years ago
the thing is : even if I set
target_group_arn  = "${module.alb.default_target_group_arn}"
PePe Amengual
PePe Amengualabout 7 years ago
to the target group arn itself
PePe Amengual
PePe Amengualabout 7 years ago
it does not work
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
I ran this project https://github.com/cloudposse/terraform-root-modules/tree/master/aws/ecs about 35 times, but never saw it could not find the target group
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
I saw in many cases that the ALB was not ready yet (it’s slow), so it could not attach the target group
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
after running the whole thing second time, the ALB is already ready, and it finishes provisioning the rest
PePe Amengual
PePe Amengualabout 7 years ago
yes, ALBs are really slow
PePe Amengual
PePe Amengualabout 7 years ago
I’m running this thing again just to make sure I did’t do anything stupid
PePe Amengual
PePe Amengualabout 7 years ago
just tried from zero and run the alb target 3 times, waited few minutes and tun module.alb_ingress and I get the same error
PePe Amengual
PePe Amengualabout 7 years ago
null_resource.default: Refreshing state... (ID: 938128705396491941)
aws_security_group.default: Refreshing state... (ID: sg-0dca458073974ca29)
aws_lb_target_group.default: Refreshing state... (ID: arn:aws:elasticloadbalancing:us-east-1:...o-droneio-alb-default/c5d32c308b13dee7)
data.aws_elb_service_account.default: Refreshing state...
data.aws_iam_policy_document.default: Refreshing state...
aws_s3_bucket.default: Refreshing state... (ID: staging-demo-droneio-alb-alb-access-logs)
aws_lb.default: Refreshing state... (ID: arn:aws:elasticloadbalancing:us-east-1:...ging-demo-droneio-alb/61f67535cafe2201)
aws_lb_listener.http: Refreshing state... (ID: arn:aws:elasticloadbalancing:us-east-1:...-alb/61f67535cafe2201/8e7ac22fe675d697)

Error: Error running plan: 1 error occurred:
	* module.alb_ingress.local.target_group_arn: local.target_group_arn: Resource 'aws_lb_target_group.default' not found for variable 'aws_lb_target_group.default.arn'
`
PePe Amengual
PePe Amengualabout 7 years ago
so I can consistently reproduce the issue
PePe Amengual
PePe Amengualabout 7 years ago(edited)
I can run this :
terraform apply -target data.aws_lb_target_group.default                                                                                                                              
aws_lb_target_group.default: Refreshing state... (ID: arn:aws:elasticloadbalancing:us-east-1:...o-droneio-alb-default/c5d32c308b13dee7)
PePe Amengual
PePe Amengualabout 7 years ago
with same code is failing in the module and it does not fail outside of it
PePe Amengual
PePe Amengualabout 7 years ago
so, something is happening here :
locals {
  target_group_enabled = "${var.target_group_arn == "" ? "true" : "false"}"
  target_group_arn     = "${local.target_group_enabled == "true" ? aws_lb_target_group.default.arn : var.target_group_arn}"
}
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
what TF version are you using?
PePe Amengual
PePe Amengualabout 7 years ago
Terraform v0.11.14
+ provider.aws v2.23.0
+ provider.local v1.3.0
+ provider.null v2.1.2
+ provider.random v2.2.0
+ provider.template v2.1.2
+ provider.tls v2.0.1

Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
i think you are running into some race conditions
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
PePe Amengual
PePe Amengualabout 7 years ago
but
resource "aws_lb_target_group" "default" {
  count       = "${local.target_group_enabled == "true" ? 1 : 0}"
PePe Amengual
PePe Amengualabout 7 years ago
will only be created if target_group_arn is = “”
PePe Amengual
PePe Amengualabout 7 years ago
base on the locals evaluation
PePe Amengual
PePe Amengualabout 7 years ago
so it should not even be trying to create this resource ?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
so yes, you either provide one, or the module will create it
PePe Amengual
PePe Amengualabout 7 years ago(edited)
in my case I’m providing one
PePe Amengual
PePe Amengualabout 7 years ago
so you think somehow the evaluation is failing ?
PePe Amengual
PePe Amengualabout 7 years ago
even if I pass the raw arn value still fails
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
ok yes, there is a bug in that flow. we always created target group in the module (did not test when you provide one)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
the bug is…
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
target_group_arn = "${local.target_group_enabled == "true" ? join("", aws_lb_target_group.default.*.arn) : var.target_group_arn}"
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
want to open a PR? 🙂
PePe Amengual
PePe Amengualabout 7 years ago
for sure
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
thanks for finding it
PePe Amengual
PePe Amengualabout 7 years ago
this thing has been killing me, I though I was doing something wrong
PePe Amengual
PePe Amengualabout 7 years ago
I know I have to run few commands before I create the PR
PePe Amengual
PePe Amengualabout 7 years ago
are those documented somewhere ?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
you mean to rebuild README?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
make init
make readme/deps
make readme
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
you need that if you change any variable or outputs or README.yaml (not in this case where you just fix the code)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
but run it anyway
PePe Amengual
PePe Amengualabout 7 years ago
I see ok
Mike Nock
Mike Nockabout 7 years ago
Has anyone here used terraform as the CD portion of the CI/CD pipeline? Currently, I'm deploying docker images to ECR with Gitlab, but I'm running into an issue where I need to somehow tell terraform to update it's workspaces once a new image comes out. Was curious if anyone else had run into this / figured this out
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
@Mike Nock you mean that terraform is somehow constantly running and waiting for commands? Or it’s a manual process?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
if manual, you can use https://www.terraform.io/docs/providers/aws/d/ecr_image.html to retrieve information about images
Mike Nock
Mike Nockabout 7 years ago
Yea, currently the pipeline is Gitlab > Docker > ECR for CI, and then manually deploying the images by going into TFE and updating the workspace so teraform sees the new image (the CI pipeline retags the image with :production and removes that tag from the old image when deploying to ECR, and terraform is set to only use the image with that tag).
I'd like to automate that. So, when the developers push a new feature to master, it build, deposits the image, and then either sends an API call to terraform to update (prefer not doing it this way), or preferably someway of having terraform monitor the tags, and deploy once the tag is removed?
Mike Nock
Mike Nockabout 7 years ago
Also, for backstory, the reason I prefer not doing the api calls to update, is we are doing self-service terraform where client environments are being created regularly and dynamically, so I wouldn't have a list of all the workspaces to send individual API calls to, and you can't send 1 api call to update all workspaces, same as you can't send 1 api call with a map of variables for the workspace but have to list out each one by one in separate calls.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
that’s interesting (but we at CloudPosse did not use TFE)
Mike Nock
Mike Nockabout 7 years ago
Understandable, just figured it was worth asking if anyone else had run into it. Thanks!
Hetal S
Hetal Sabout 7 years ago(edited)

1 error occurred:
        * module.s3cdn-dev.aws_route53_record.cert_validation: At column 19, line 1: list "local.dvo" does not have any elements so cannot determine type. in:

${lookup(local.dvo[count.index], "resource_record_value")}
PePe Amengual
PePe Amengualabout 7 years ago(edited)
PePe Amengual
PePe Amengualabout 7 years ago
I’m guessing that is only available in providers and should not be used with other terraform code except rds ?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
those are attributes on the provider which you can use to disable some checks if you want to run it faster
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
has nothing to do with RDS
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
used in some tests
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
but not necessary at all
PePe Amengual
PePe Amengualabout 7 years ago
RDS takes so long I was thinking on adding them to my big TF but I was not sure if they could break something
Milos Backonja
Milos Backonjaabout 7 years ago
Do you have module which covers ACM Certificate for CloudFront if Route 53 HostedZone is not in us-east-1?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Milos Backonja
Milos Backonjaabout 7 years ago
Thanks @Andriy Knysh (Cloud Posse) Of course, its provider thing, I may miss something, but I don't see how this reference address my use case
Milos Backonja
Milos Backonjaabout 7 years ago
If I see well it presume that Hosted Zone is in same Region
Milos Backonja
Milos Backonjaabout 7 years ago
or i take that back 😄
Milos Backonja
Milos Backonjaabout 7 years ago
sorry
Milos Backonja
Milos Backonjaabout 7 years ago
Hosted Zones are global
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Yes hosted zones are global
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
You create a certificate in us-east-1 using different provider
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Then reference the cert ARN when you create a distribution
Milos Backonja
Milos Backonjaabout 7 years ago
@Andriy Knysh (Cloud Posse) Thank, I did that, but i messed up something else which made me problem 😆
Milos Backonja
Milos Backonjaabout 7 years ago
provisioning atm, everything looks good so far
Julio Tain Sueiras
Julio Tain Sueirasabout 7 years ago
regarding the HCL deprecation on github action
Julio Tain Sueiras
Julio Tain Sueirasabout 7 years ago
is very likely due to the recent news that github action will have built-in CI/CD
Julio Tain Sueiras
Julio Tain Sueirasabout 7 years ago
and given github is owned by microsoft
Julio Tain Sueiras
Julio Tain Sueirasabout 7 years ago
is very likely the backend is azuredevops
Julio Tain Sueiras
Julio Tain Sueirasabout 7 years ago
and until next tuesday, automation for azuredevops pipeline is in yaml
Julio Tain Sueiras
Julio Tain Sueirasabout 7 years ago
@Andriy Knysh (Cloud Posse) after much time, upcoming tuesday is the decided to release the azuredevops terraform provider
Julio Tain Sueiras
Julio Tain Sueirasabout 7 years ago
decided day*
Leonard Wood
Leonard Woodabout 7 years ago
Are there any known bugs with the ec2 module, release 0.11?
Leonard Wood
Leonard Woodabout 7 years ago
deployed an ec2 instance using the ec2 module, and had associate_public_ip_address set to true. Changed it to false from true, which then prompted a redeploy, now it fails with the error:
value = coalesce(
aws_eip.default is empty tuple
aws_instance.default is tuple with 1 element
Call to function "coalesce" failed: no non-null, non-empty-string arguments.
Leonard Wood
Leonard Woodabout 7 years ago
I cant destroy or apply any updates to my entire deployment 😞
Leonard Wood
Leonard Woodabout 7 years ago
looks like there is a pull request for this specific issue: https://github.com/cloudposse/terraform-aws-ec2-instance/pull/45
Leonard Wood
Leonard Woodabout 7 years ago
does anyone know if there is a temp workaround for this?
Maarten van der Hoef
Maarten van der Hoefabout 7 years ago
@LeoGmad you can fork the branch of the PR and use that one ?
Leonard Wood
Leonard Woodabout 7 years ago
I will try, thank you @Maarten van der Hoef
Leonard Wood
Leonard Woodabout 7 years ago
I was able to successful fork the PR but issue still persists. I would be interested if anyone has been successful with this PR or finding a workaround
Leonard Wood
Leonard Woodabout 7 years ago
the only way to resolve is to set all instances "associate_public_ip_address = true"
Leonard Wood
Leonard Woodabout 7 years ago
which I guess isnt a big deal as long as they're deployed behind a NAT or limited ACL
Maarten van der Hoef
Maarten van der Hoefabout 7 years ago
@Leonard Wood ok let me run the example with associate_public_ip_address set to false, and see what can be done.
Maarten van der Hoef
Maarten van der Hoefabout 7 years ago
@Leonard Wood see if it works with the new pr
Leonard Wood
Leonard Woodabout 7 years ago
I tried with the new PR but no luck. I am deploying 2 ec2 instances, one set to true and one to set false - and thats when the issue occurs. All ec2 instance deployments have to be set to 'true' for it to deploy.
Maarten van der Hoef
Maarten van der Hoefabout 7 years ago
@Leonard Wood make sure to clean your cache. I'm running 'examples/complete' with instance_enabled = falseand that works
C
Cloud Posseabout 7 years ago
:zoom: Join us for "Office Hours" every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Aug 21, 2019 11:30AM.

👉️ Register for Webinar

:slack: #office-hours (our channel)
Vitaliy Lobachev
Vitaliy Lobachevabout 7 years ago
Hello! I want to share with you resource:for_each and dynamic:block:for_each construction for terraform 12.6 .
Hope this helps you work with large arrays of resources.
https://github.com/devops-best-practices/terraform-best-practice/blob/master/s3.tf
Sharanya
Sharanyaabout 7 years ago
Hey people, looking for terraform template on vpc peering ( syntax 0.12) any help plz
Vikram Yerneni
Vikram Yerneniabout 7 years ago
Hola fellas....
Vikram Yerneni
Vikram Yerneniabout 7 years ago
Quick question here...
We have a Terraform RDS module (typical base build format) for the build deployments to use to setup RDS instances in our AWS Setups. Now I am tying to enable alerting (SNS topics with Cloudwatch) within the existing RDS module but not sure how to enable the alerting within an existing RDS module. I found out eh cloud posse GitHub repo (https://github.com/cloudposse/terraform-aws-rds-cloudwatch-sns-alarms) will give the ability to create the sns topics (please do correct me if I am wrong here), but what I need is to enable the alerting within the RDS module so that the users will create the required sns topics based on their needs.
Anyone worked on this kind of typical setups before? Any input helps me for my cause here
Vikram Yerneni
Vikram Yerneniabout 7 years ago
or even anyone been through this kind of requirement before?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
you can use the alarms from https://github.com/cloudposse/terraform-aws-rds-cloudwatch-sns-alarms/blob/master/alarms.tf (update them and add new ones if needed)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
then you can create an SNS topic in diff module (or manually, or however you need it), and subscribe the RDS instance to the topic https://github.com/cloudposse/terraform-aws-rds-cloudwatch-sns-alarms/blob/master/main.tf#L20
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
(the module currently always creates an SNS topic)
PePe Amengual
PePe Amengualabout 7 years ago(edited)
and since we are talking about SNS alarms :
PePe Amengual
PePe Amengualabout 7 years ago(edited)
I’m using : “git::https://github.com/cloudposse/terraform-aws-alb-target-group-cloudwatch-sns-alarms.git?ref=tags/0.6.1” and I notice when using the newer alb module that target_group_name = “${module.alb.target_group_name}” and target_group_arn_suffix = “${module.alb.target_group_arn_suffix}” are not valid outputs anymore so it can’t be use with this cloudwatch-sns module, are you guys deprecating the use of the cloudwatch-sns-alarms or recommend something else ?
Vikram Yerneni
Vikram Yerneniabout 7 years ago
Sure @Andriy Knysh (Cloud Posse)... Let me give it a try and will update here in the group with the progress..
sarkis
sarkisabout 7 years ago(edited)
Vikram Yerneni
Vikram Yerneniabout 7 years ago
Also @Andriy Knysh (Cloud Posse), just a quick fyi.. we dont use the IAM policy in the main.tf for the SNS topics creation as we use the user aces at higher levels in our build deployments. Is there other way to comment that part out of the module?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
if you don’t use aws_sns_topic, then you don’t need the policy as well
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
comment out of the module: you can fork it and comment out aws_sns_topic and the policy, or you can open a PR and add new var sns_topic_enabled (set to true by default for backwards compatibility). Then use count = "${var.sns_topic_enabled == "true" ? 1 : 0}" for aws_sns_topic and aws_db_event_subscription and aws_sns_topic_policy`
Vikram Yerneni
Vikram Yerneniabout 7 years ago
Without using the aws_sns_topic, how can I subscribe the RDS instance to the topic https://github.com/cloudposse/terraform-aws-rds-cloudwatch-sns-alarms/blob/master/main.tf#L20 (from your above statement dude)?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
you need an SNS topic to subscribe the instance to a topic
Vikram Yerneni
Vikram Yerneniabout 7 years ago
ok ok
Vikram Yerneni
Vikram Yerneniabout 7 years ago
let me comment out the section from the module and will try it out
Maarten van der Hoef
Maarten van der Hoefabout 7 years ago(edited)
Hi Everyone, I'm having the issue that when I attach a private EIP to an instance in a private subnet,, that associate_public_ip_address gets set to true. This with a subnet with public ip mapping to false. Maybe someone else stumpled upon the same issue ?
joshmyers
joshmyersabout 7 years ago
@Maarten van der Hoef what module is this?
Maarten van der Hoef
Maarten van der Hoefabout 7 years ago
"terraform-aws-modules/ec2-instance/aws" but it's irrelevant, it's not a module problem
joshmyers
joshmyersabout 7 years ago
No but I just want to have a look
joshmyers
joshmyersabout 7 years ago
and what is the actual problem? You don’t want associate_public_ip_address: true ?
Maarten van der Hoef
Maarten van der Hoefabout 7 years ago

# Grafana
module "grafana" {
  source  = "terraform-aws-modules/ec2-instance/aws"
  version = "~> 2.0"

  name           = "grafana-xlt"
  instance_count = 1

  ami                    = var.grafana_ami
  instance_type          = "m4.xlarge"
  key_name               = ""
  monitoring             = true
  vpc_security_group_ids = [module.ec2_sg.this_security_group_id]
  subnet_id              = module.vpc.private_subnets[0]
  #private_ip = "10.0.1.200"
   associate_public_ip_address = false

  user_data = "{\"auth\": [ {\"name\": \"admin\", \"pass\": \"${var.password}\"}]}"

  tags = {
    Name   = "grafana"
    Terraform   = "true"
    Environment = "dev"
  }
}

#resource "aws_eip" "grafana" {
#  vpc = true
#  associate_with_private_ip = "10.0.1.200"
#}
#
#resource "aws_eip_association" "grafana" {
#  instance_id                  = module.grafana.id[0]
#  allocation_id = aws_eip.grafana.id
#}
joshmyers
joshmyersabout 7 years ago(edited)
OK, makes more sense seeing that. saying that, I haven’t come across this
joshmyers
joshmyersabout 7 years ago(edited)
Does the instance then get a routable public IP that you don’t want it to have?
Maarten van der Hoef
Maarten van der Hoefabout 7 years ago
yep
joshmyers
joshmyersabout 7 years ago
Can you deny outside world access via SG rule?
Maarten van der Hoef
Maarten van der Hoefabout 7 years ago
It's a human error I think. I thought the EIP would be private, but it's actually a public EIP .. associated with a private address
sarkis
sarkisabout 7 years ago(edited)
ah i don't think you can have a private EIP - you'd need ENI for that ... assuming I understood correctly what you want (a private static ip?)
joshmyers
joshmyersabout 7 years ago
Don’t think that actually works does it?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
EIPs are always public
Hugo Lesta
Hugo Lestaabout 7 years ago
Hello there, do you have any fargate terraform module to analyze and implement in a production environment?
Julio Tain Sueiras
Julio Tain Sueirasabout 7 years ago
Szymon
Szymonabout 7 years ago
Hi, is there any possibility to setup azurerm app service deployment from bitbucket automatically?
scm_type block doesn't work, as described in this issue: https://github.com/terraform-providers/terraform-provider-azurerm/issues/3696

Do you see any alternatives?
Chris Montes
Chris Montesabout 7 years ago(edited)
Hey guys, anyone know how to get terraform-aws-elastic-beanstalk-environment to attach security groups directly to the launch config this module creates? Currently, whatever SGs you list under var.security_groups, they all get added as security group rules of a new SG that this module creates, instead of actually associating the SGs themselves directly to the Launch Config... https://github.com/cloudposse/terraform-aws-elastic-beanstalk-environment/blob/master/main.tf#L318
tweetyix
tweetyixabout 7 years ago
Hi, I have a general Terraform question. Is anybody using it for deployment of and to HyperV? I found a community provider however I'm a little bit hesitant in using it. Which other gitops based tool would be available for HyperV deployments?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
@tweetyix saw @chrism talking about hyperv in #geodesic last month
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
---
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
Our public weekly #office-hours is starting now! Join https://zoom.us/j/508587304
Alejandro Rivera
Alejandro Riveraabout 7 years ago(edited)
Hi, I’m trying to create a aws_vpc_peering_connection between 2 accounts, cross region using assume-role and specifying a aws provider for the aws_vpc_peering_connection_accepter with corresponding region. This fails because the requester tries to look for the accepter vpc in the same region and fails to find the vpc. I’ve used this code to successfully deploy cross account, but on the same region.
Using TF version 0.11.13
Patrick Beam
Patrick Beamabout 7 years ago
I’ve been tinkering with this module today. https://github.com/cloudposse/terraform-aws-ec2-instance-group Is there a way to get instances spread across mutiple AZ’s?
Callum Robertson
Callum Robertsonabout 7 years ago
@Patrick Beam you could provide a list of availability zones for that region e.g. ["a", "b", "c"] and then in your availability_zone input use this:
Callum Robertson
Callum Robertsonabout 7 years ago
availability_zone = "${element(var.availability_zones, count.index)}"
Callum Robertson
Callum Robertsonabout 7 years ago
You would just have to change that variable to a type = list(string)
Callum Robertson
Callum Robertsonabout 7 years ago
hope that helps
PePe Amengual
PePe Amengualabout 7 years ago
Anyone with experience in
resource "aws_ssm_document
and after destroy the document does not seems to be deleted
Sharanya
Sharanyaabout 7 years ago
Having this error on Tf Plan - Error: Missing resource instance key

on .terraform\modules\vpc_peering_cross_account\accepter.tf line 96, in locals:
96: accepter_aws_route_table_ids = "${distinct(sort(data.aws_route_tables.accepter.ids))}"
S
SweetOpsabout 7 years ago
Are you using some of our terraform-modules in your projects? Maybe you could leave us a testimonial! It means a lot to us to hear from people like you.
Bharat
Bharatabout 7 years ago
How do i set order precedence in terraform? I have main.tf where it destroys a IAM role and deploy’s it again on terraform-apply. I am hitting a error aws_iam_role.service_role: Error creating IAM Role DEFAULT-TestingService-ecs-service-role: EntityAlreadyExists: Role with name DEFAULT-TestingService-ecs-service-role already exists

This is after destroying the role, pretty sure that AWS needs some time to update the cache.

I want to first destroy the services and then create it. Is this possible?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
hmmm…. terraform is used to create resource definitions - how to create them
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
terraform knows from its state file what was created
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago(edited)
how are you doing destroy from TF files?
PePe Amengual
PePe Amengualabout 7 years ago
Hi, do you guys have a preference on using the aws KMS managed key or creating a CMK ?
PePe Amengual
PePe Amengualabout 7 years ago
the reason I ask that is that after removing Kms_key_id from :
module "ssm_tls_ssh_key_pair" {
  source               = "git::<https://github.com/cloudposse/terraform-aws-ssm-tls-ssh-key-pair.git?ref=0.2.0>"
  namespace            = "${var.namespace}"
  stage                = "${var.stage}"
  name                 = "${var.name}"
  attributes           = "${var.attributes}"
  
  ssm_path_prefix      = "${var.stage}/${var.name}/infrastructure/ssh_keys"
  ssh_key_algorithm    = "RSA"
  ssh_private_key_name = "${module.default_label.id}-private"
  ssh_public_key_name  = "${module.default_label.id}-public"
  #kms_key_id           = "${module.kms_key.key_id}"
}
PePe Amengual
PePe Amengualabout 7 years ago
I got
Error refreshing state: 1 error occurred:
	* module.ssm_tls_ssh_key_pair.data.aws_kms_key.kms_key: 1 error occurred:
	* module.ssm_tls_ssh_key_pair.data.aws_kms_key.kms_key: data.aws_kms_key.kms_key: error while describing key [alias/test-demo-chamber]: NotFoundException: Alias arn:aws:kms:us-east-1:046894643055:alias/test-demo-chamber is not found.
	status code: 400, request id: 7e387954-2256-4ef2-a40e-b48269259e9c
Maeghan Porter
Maeghan Porterabout 7 years ago(edited)
Hey all, I updated the release of terraform-aws-dynamic-subnets that I’m pulling down and now I’m getting this error:
Error downloading modules: Error loading modules: module dynamic_subnets: Error parsing .terraform/modules/e972fa1c1c4c2e3a44d52f7491016697/label.tf: At 3:25: Unknown token: 3:25 IDENT var.attributes
Maeghan Porter
Maeghan Porterabout 7 years ago
Any idea what’s going on?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Maeghan Porter
Maeghan Porterabout 7 years ago
oh ok, I’m on TF 0.11.13 and trying to use release 0.16.0 of the module. That must be why then.
Maeghan Porter
Maeghan Porterabout 7 years ago
I’ll try version 0.12.0
Maeghan Porter
Maeghan Porterabout 7 years ago
thanks!
Joe Presley
Joe Presleyabout 7 years ago
I’m looking to move terraform code for a customer from regular terraform to terraform enterprise. I googled but couldn’t find if there are any quirks or things I should know about TFE. My goal is to create a terraform module that can be used for both the open source version as well as TFE.
iG
iGabout 7 years ago
I have an existing aws ecs cluster (made by hand) and would like to update it by using terraform.
Is there any documentation for updating existing aws services (alb, ecr, ecs)?
praveen
praveenabout 7 years ago
Hi, I am using terraform workspace to create azure windows server's in more than one environment using one terraform code. For passing hostname and IP address am using the following lookup's in local
praveen
praveenabout 7 years ago
locals {
location = "${terraform.workspace}"
image_id = "${module.image.image_id}"
environment = "${module.locals.environment}"

subnets = {
eastus2_prod = "${data.terraform_remote_state.shared_networking.eastus2_api_tier_subnet.id}"
centralus_prod = "${data.terraform_remote_state.shared_networking.centralus_api_tier_subnet.id}"
}

lb_ips = {
eastus2_prod = "10.244.160.164"
centralus_prod = "10.245.160.164"
}

system = {
eastus2_prod = [
{
hostname = "wqilpeap101"
ip = "10.244.160.165"
},
{
hostname = "wqilpeap102"
ip = "10.244.160.166"
},
]

centralus_prod = [
{
hostname = "wqilpcap101"
ip = "10.245.160.165"
},
{
hostname = "wqilpcap102"
ip = "10.245.160.166"
},
]

}

subnet_id = "${lookup(local.subnets, format("%s_%s",local.location, var.environment))}"
lb_ip = "${lookup(local.lb_ips, format("%s_%s",local.location, var.environment))}"
systems = "${lookup(local.system, format("%s_%s",local.location, var.environment))}"
}
praveen
praveenabout 7 years ago
it fails with following error message
praveen
praveenabout 7 years ago
Error: Error asking for user input: 1 error occurred:
* local.systems: local.systems: lookup: lookup() may only be used with flat maps, this map contains elements of type list in:

${lookup(local.system, format("%s_%s",local.location, var.environment))}
praveen
praveenabout 7 years ago
can you help me fix this
Maarten van der Hoef
Maarten van der Hoefabout 7 years ago
@praveen you can try:
subnet_id = "${local.subnets[ format("%s_%s",local.location, var.environment)]}"
praveen
praveenabout 7 years ago
subnet is working fine
praveen
praveenabout 7 years ago
issue is with systems
praveen
praveenabout 7 years ago
should I try it for systems?
Maarten van der Hoef
Maarten van der Hoefabout 7 years ago
yes
praveen
praveenabout 7 years ago
trying now
Alex Siegman
Alex Siegmanabout 7 years ago(edited)
So, I'm trying to be a bit clever. I have a need to conditionally add statements to an IAM policy document

I did try to do a join on the data.aws_iam_policy_document.<stuff>.json property of multiple data sources, and mixed data sources and already rendered json documents coming in as variables in strings.

I think the root problem with this approach is that both things will render a FULL json document, so it will confuse stuff, such as:

{
  my policy doc
}
{
  my next policy doc
}


Is there a way to conditionally add statement blocks to a data.aws_iam_policy_document ?

The root cause here is that the cloudposse/terraform-aws-s3-module has some built in document handling to set an "encrypted-only" policy, so if I need to do something like add a separate cross-account access principal policy, I can't, because an s3 bucket can only have one bucket policy attached.

I've forked it and attempted my above described fix here: https://github.com/asiegman/terraform-aws-s3-bucket/blob/moar-bucket-policy-0.11/main.tf#L94

But alas, that didn't work due to the multiple json documents being joined to form invalid json.

I can always not use cloudposse's module and just build my own resources, but if I could add a clever bit to add arbitrary statements in for stuff like this, I was going to deliver it back to the community
Maarten van der Hoef
Maarten van der Hoefabout 7 years ago(edited)
@Alex Siegman in 0.12 you can do this with "dynamic" loops. What you could do with 0.11 is using source_json with another aws_iam_policy_document. I don't really like it so much but it's a funny hack: https://github.com/doingcloudright/terraform-aws-ecr-cross-account/blob/ab55861e4de158d3bf490976c16a2bebb6661c28/main.tf#L43
Alex Siegman
Alex Siegmanabout 7 years ago
Oh interesting. What happens if source json is just a blank string then~? I'll play with it, great lead. Thanks!
Maarten van der Hoef
Maarten van der Hoefabout 7 years ago
Alex Siegman
Alex Siegmanabout 7 years ago
heck, i can probably just use that, i already have multiple valid documents, i just need to aggregate all their statements in to one
davidvasandani
davidvasandaniabout 7 years ago
Why did the release cadence change for this module? https://github.com/cloudposse/terraform-aws-vpc/releases
Norbert
Norbertabout 7 years ago
Hi, I do a bit of TF (0.12) + AWS. I have created a few modules: VPC, Subnets, IGW, etc. Each module outputs.tf some variables, I use S3 backend and I can access exported variables from different modules. All smooth and easy but when I create application load balancer (aws_lb) I can NOT make terraform to output any variable. My output.tf (root module dir) looks like this:

output "alb_id" {
value = aws_lb.alb.id
}
output "alb-security-group_id" {
value = aws_security_group.alb-security-group.id
}
output "alb-target-group_arn" {
value = aws_lb_target_group.alb-target-group.arn
}

The ALB, security group and target group gets created, I can see it in the console but the output is empty, no errors during terraform apply. Why?
Barani
Baraniabout 7 years ago
Hi, Anybody using local persistent volume for any kubernetes_stateful_set. I am trying use affinity with ndoe_selector_term but failing. Kindly guide me if anyone using in this way.
C
Cloud Posseabout 7 years ago
:zoom: Join us for "Office Hours" every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Aug 28, 2019 11:30AM.

👉️ Register for Webinar

:slack: #office-hours (our channel)
Sharanya
Sharanyaabout 7 years ago
anybody working with terraform integration with Jenkins cicd pipeline
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
we use atlantis and Codefresh to deploy terraform, but here are a few articles on how to do it from Jenkins
Brij S
Brij Sabout 7 years ago
Hello, I was curious to see if anyone has ever tried to create a cross account aws codepipeline with terraform. If not, It would be great to get some feedback on my current approach
mpmsimo
mpmsimoabout 7 years ago
Does anyone use software to detect state drift on a recurring basis?

I'd like to start performing infrastructure wide Terraform state drift identification, and work towards making sure I see that excellent "No changes." messaging after issuing a terraform plan across the board more frequently.
loren
lorenabout 7 years ago
We just have our CI run terraform plan -detailed-exitcode, and alert on job failures...
mpmsimo
mpmsimoabout 7 years ago(edited)
I was looking at the detailed-exitcode option, seems good as I can get an explicit list of terraform projects that are having issues. I'd like to go a step further so I can reduce operator work around identification and fixing of the state drift manually.
mpmsimo
mpmsimoabout 7 years ago
Only saw a few project on GitHub, none of them maintained.
loren
lorenabout 7 years ago(edited)
Cheap and easy and works was my thinking, can always optimize later
loren
lorenabout 7 years ago
Could save off the plan, and analyze it separately. Maybe whitelist some resources/diffs to run apply automatically
mpmsimo
mpmsimoabout 7 years ago
The Foqual bot was able to find some information regading GitHub Actions to perform plans, could be a good starting point for me.
mpmsimo
mpmsimoabout 7 years ago
Yeah, very much MVP for this drift detector - iterate later 🙂
mpmsimo
mpmsimoabout 7 years ago
Thanks for the ideas, I'll think about this some more and see how to approach this
PePe Amengual
PePe Amengualabout 7 years ago(edited)
Hi, should this module : https://github.com/cloudposse/terraform-aws-cloudwatch-logs when a CMK key gets passed to create a policy to be able to use that key ?
Callum Robertson
Callum Robertsonabout 7 years ago(edited)
Hi All, I'm creating a bucket policy data resource (https://www.terraform.io/docs/providers/aws/r/s3_bucket_policy.html)

I'm running into an issue where I'm trying to reference the 'json' attribute of the data source in a policy account resource, can anyone help me with the below?
Callum Robertson
Callum Robertsonabout 7 years ago
Callum Robertson
Callum Robertsonabout 7 years ago
I'm getting the error that it's an empty tuple, not sure what I'm doing wrong here..
Maarten van der Hoef
Maarten van der Hoefabout 7 years ago
@Callum Robertson what about you do at line16 the same as line 2, so the count's are in sync
Maarten van der Hoef
Maarten van der Hoefabout 7 years ago
or what is the idea there, that you only want to apply the policy when var.upload_bucket_objects is set to true, correct ? What happens now is that you refer to a policy you are not creating hence it fails.
Sharanya
Sharanyaabout 7 years ago
encountered the following error when running terraform apply?

[3:57 PM]
"policy" contains an invalid JSON: invalid character 'a' looking for beginning of value
Callum Robertson
Callum Robertsonabout 7 years ago
Thanks @Maarten van der Hoef, I think it's the case of starting at a problem for to long
Callum Robertson
Callum Robertsonabout 7 years ago
I fixed it with the following if anyone is interested
Callum Robertson
Callum Robertsonabout 7 years ago
Dan
Danabout 7 years ago
Can't recall but when you create an AWS SG rule with terraform and you do something like count index vs creating separate rules, which one doesn't delete the whole sg and create a new one? Instead of creating a new sg every time I'd like to just keep adding/removing ports if needed.

from_port = "80"
vs
from_port = var.allowed_ports[count.index]

I know in your tf you do the second one. let me know! Thank you!
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
@Dan when you create a separate aws_security_group w/o rules, you can add as many aws_security_group_rule as you need w/o recreating the whole SG
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
sarkis
sarkisabout 7 years ago
Curious, are you all using or looked into dependabot for terraform module dependencies - https://dependabot.com/terraform/
Addison Higham
Addison Highamabout 7 years ago
for the cloudposse modules, I got all these working with 0.12:
https://github.com/cloudposse/terraform-aws-ec2-autoscale-group/pull/14
https://github.com/cloudposse/terraform-aws-eks-workers/pull/21
https://github.com/cloudposse/terraform-aws-eks-cluster/pull/20

I forgot to update the version for CI to 0.12, will try and push that out
Addison Higham
Addison Highamabout 7 years ago
I am using them and they are working
Addison Higham
Addison Highamabout 7 years ago
but could use some help to get the rest of the work done (porting examples and adding the new CI stuff)
Addison Higham
Addison Highamabout 7 years ago
except... looking around, I have no idea how to get it 0.12 to run as part of the things
Sharanya
Sharanyaabout 7 years ago
Error: The role "arn:aws:iam::474562475633:role/gc-invoicedataimport-function-role" cannot be assumed.
There are a number of possible causes of this - the most common are:

* The credentials used in order to assume the role are invalid

* The credentials do not have appropriate permission to assume the role

* The role ARN is not valid
Chad Hartman
Chad Hartmanabout 7 years ago(edited)
Do you need to use version Terraform 0.11 when bootstrapping with the reference-architectures?
gk
gkabout 7 years ago
Hey folks. Is there a way to use a merge or use a splat-type operator in a terraform child block (not identifier values) ? i.e.

data "aws_ami" "potato" {
  filter {
    ...local.default_filters
  }
}
gk
gkabout 7 years ago
I'm effectively trying to filter merge(local.thing, {})
cabrinha
cabrinhaabout 7 years ago
So, when you're creating an ECS service, you either have the choice between using an ALB or not.
cabrinha
cabrinhaabout 7 years ago
If you're using an ALB, you need to pass in additional objects for the load_balancer parameter, example here: https://www.terraform.io/docs/providers/aws/r/ecs_service.html#example-usage
cabrinha
cabrinhaabout 7 years ago
How would you structure a module to make this object parameterized and optional?
cabrinha
cabrinhaabout 7 years ago
just create a ton of these different ecs_service resources based on how they're configured.
Sharanya
Sharanyaabout 7 years ago
The variable "lambda_settings" is required, so Terraform cannot proceed
without a defined value for it. " - any idea about this
C
Cloud Posseabout 7 years ago
:zoom: Join us for "Office Hours" every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Sep 04, 2019 11:30AM.

👉️ Register for Webinar

:slack: #office-hours (our channel)
Brij S
Brij Sabout 7 years ago
Hey all! New to terraform, I'm trying to do some acm cert validation using : https://www.terraform.io/docs/providers/aws/r/acm_certificate_validation.html

In the link, they show the following r53 record for cert validation being created:
resource "aws_route53_record" "cert_validation" {
  name    = "${aws_acm_certificate.cert.domain_validation_options.0.resource_record_name}"
  type    = "${aws_acm_certificate.cert.domain_validation_options.0.resource_record_type}"
  zone_id = "${data.aws_route53_zone.zone.id}"
  records = ["${aws_acm_certificate.cert.domain_validation_options.0.resource_record_value}"]
  ttl     = 60
}


Specifically, aws_acm_certificate.cert.domain_validation_options.0.resource_record_name , I create my acm certs(Specifically 2) like this:
resource "aws_acm_certificate" "cert" {
  provider          = "aws.acm"
  domain_name       = "*.${var.domain}.${element(var.certs, count.index)}.${var.aws_env == "prod" ? "com." : "test."}"
  validation_method = "DNS"

  tags = "${local.required_tags}"

  lifecycle {
    create_before_destroy = true
  }
  count = "${length(var.certs)}"
}


How would I be able to reference each cert for validation?
mrwacky
mrwackyabout 7 years ago
This looks correct
Brij S
Brij Sabout 7 years ago
what looks correct?
Brij S
Brij Sabout 7 years ago
I dont know how to reference each cert for validation
sarkis
sarkisabout 7 years ago(edited)
@Brij S you are missing a resource - let me find it
sarkis
sarkisabout 7 years ago
ah, I see what you mean ... you will need to somehow loop over all of your certs and specify a aws_acm_certificate_validation resource for each one...
Brij S
Brij Sabout 7 years ago
yeah.. thats the tricky part 😛
Brij S
Brij Sabout 7 years ago
not sure how to facilitate that
sarkis
sarkisabout 7 years ago(edited)
@Brij S are you using terraform 0.12+?
Brij S
Brij Sabout 7 years ago
no, TF 0.11
sarkis
sarkisabout 7 years ago
it might be impossible to do this, since the count for resource "aws_route53_record" "cert_validation" will not be able to be <computed>, i.e. can't dynamically set to the cert count iirc
PePe Amengual
PePe Amengualabout 7 years ago
What are you guys toughs on Terraform enterprise ?
PePe Amengual
PePe Amengualabout 7 years ago
is it worth it ?
Brij S
Brij Sabout 7 years ago
@sarkis right! that makes sense, any thoughts on how to go forward with this then
sarkis
sarkisabout 7 years ago
@Brij S i'd say if you want to continue to go with this dynamically, TF 0.12 upgrade may be the only option... otherwise only way I can think of solving this in 0.11 is to have static cert_validations - hopefully someone can prove me wrong here 🙂
Brij S
Brij Sabout 7 years ago
ok, if I was on TF 0.12, how would you go about it
loren
lorenabout 7 years ago
you just count over the variable input again, length(var.certs)
Brij S
Brij Sabout 7 years ago
but how would i reference it?
Brij S
Brij Sabout 7 years ago
aws_acm_certificate.cert.domain_validation_options.0.resource_record_name refers to..one cert?
loren
lorenabout 7 years ago
you want to create a route53 record for each certificate?
Brij S
Brij Sabout 7 years ago
yep
Brij S
Brij Sabout 7 years ago
because for me aws_acm_certificate.cert contains two certs (corresponding to two zones)
loren
lorenabout 7 years ago

resource "aws_route53_record" "cert_validation" {
  count = "${length(var.certs)}"

  name    = "${aws_acm_certificate.cert.domain_validation_options.*.resource_record_name[count.index]}"
  type    = "${aws_acm_certificate.cert.domain_validation_options.*.resource_record_type[count.index]}"
  zone_id = "${data.aws_route53_zone.zone.id}"
  records = ["${aws_acm_certificate.cert.domain_validation_options.*.resource_record_value[count.index]}"]
  ttl     = 60
}
Brij S
Brij Sabout 7 years ago
@loren this is awesome, would the following be the same too then?
resource "aws_acm_certificate_validation" "cert" {
  certificate_arn         = "${aws_acm_certificate.cert.arn}"
  validation_record_fqdns = ["${aws_route53_record.cert_validation.fqdn}"]
}
loren
lorenabout 7 years ago
something like that, can also use element(...) interpolation
loren
lorenabout 7 years ago
need the wildcard to reference all the resources...

resource "aws_acm_certificate_validation" "cert" {
  certificate_arn         = "${aws_acm_certificate.cert.arn}"
  validation_record_fqdns = ["${aws_route53_record.cert_validation.*.fqdn}"]
}
loren
lorenabout 7 years ago
there are also more syntax options in terraform 0.12
Brij S
Brij Sabout 7 years ago
"${aws_acm_certificate.cert.arn}" would be "${aws_acm_certificate.cert.*.arn[count.index]}" ?
loren
lorenabout 7 years ago
validation_record_fqdns is a list, so i think you want a single resource there, not multiple with count. just pass the list of all fqdns to the parameter, rather than a single one (using count.index)
loren
lorenabout 7 years ago
if you do want a aws_acm_certificate_validation resource per cert though, then yes, same setup
Brij S
Brij Sabout 7 years ago
@loren mind if I DM you for more questions? Just want to confirm some things to make sure I understand
Phuc
Phucabout 7 years ago
Hi Guys
Phuc
Phucabout 7 years ago
I have some issue when running terraform validdate on this modules terraform-null-label
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
hi @Phuc
Phuc
Phucabout 7 years ago
Does anyone expenrienced this before ?

  terraform-null-label git:(0.11) ✗ terraform validate

Error: local.generated_tags: local.generated_tags: zipmap: count of keys (1) does not match count of values (0) in:

${zipmap(
    compact(list("Name", local.namespace != "" ? "Namespace" : "", local.environment != "" ? "Environment" : "", local.stage != "" ? "Stage" : "")),
    compact(list(local.id, local.namespace, local.environment, local.stage))
    )}
Phuc
Phucabout 7 years ago
I didn't adjust anything
Phuc
Phucabout 7 years ago
just running simple command to validate at first
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
what TF version?
Phuc
Phucabout 7 years ago
for 0.11 and below
Phuc
Phucabout 7 years ago
I clone the repo on branch 0.11/master
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
how do you use it? show the example
Phuc
Phucabout 7 years ago
actually I didn't used it yet
Phuc
Phucabout 7 years ago
I just try to validate the code first to see if there is error
Phuc
Phucabout 7 years ago
just simple terraform init then terraform validate
Phuc
Phucabout 7 years ago
and that error coming up
Kiran
Kiranabout 7 years ago
@Andriy Knysh (Cloud Posse) Is there any possibility to support TF12 for https://github.com/cloudposse/terraform-aws-ec2-bastion-server module ? I can't find any issue regarding this?
Kiran
Kiranabout 7 years ago
Can I know estimation time for this PR to be merged - https://github.com/cloudposse/terraform-aws-multi-az-subnets/pull/16 ?
Also TravisCI is failing because it checks this TF12 upgrade PR in TF11 binary.
Brij S
Brij Sabout 7 years ago
Hello all, I'm trying to do acm cert validation(multiple at a time) and I'm running into some issues.

resource "aws_route53_record" "cert_validation" {
  count   = length(var.certs)
  name    = aws_acm_certificate.cert[count.index].domain_validation_options.0.resource_record_name
  type    = aws_acm_certificate.cert[count.index].domain_validation_options.0.resource_record_type
  zone_id = aws_route53_zone.zones[count.index].id
  records = ["${aws_acm_certificate.cert[count.index].domain_validation_options.0.resource_record_value}"]
  ttl     = 60
}


the above code..kinda works? when I apply this, I get the following error:
Error: Invalid index

  on ../modules/bootstrap/acm_validation.tf line 4, in resource "aws_route53_record" "cert_validation":
   4:   type    = aws_acm_certificate.cert[count.index].domain_validation_options.0.resource_record_type
    |----------------
    | aws_acm_certificate.cert is tuple with 2 elements
    | count.index is 0

The given key does not identify an element in this collection value.


Error: Invalid index

  on ../modules/bootstrap/acm_validation.tf line 6, in resource "aws_route53_record" "cert_validation":
   6:   records = ["${aws_acm_certificate.cert[count.index].domain_validation_options.0.resource_record_value}"]
    |----------------
    | aws_acm_certificate.cert is tuple with 2 elements
    | count.index is 0

The given key does not identify an element in this collection value.


Error: Invalid index

  on ../modules/bootstrap/acm_validation.tf line 6, in resource "aws_route53_record" "cert_validation":
   6:   records = ["${aws_acm_certificate.cert[count.index].domain_validation_options.0.resource_record_value}"]
    |----------------
    | aws_acm_certificate.cert is tuple with 2 elements
    | count.index is 0

The given key does not identify an element in this collection value.


But when I do destory, it seems its creating at least 1..

module.nonprod.aws_route53_record.cert_validation[1]: Still destroying... [id=Z3FZBH8XNPPPYT__3aedbf37656ebde46d6db19a4f38212c.test-api.dev._CNAME, 30s elapsed]
module.prod.aws_route53_record.cert_validation[1]: Still destroying... [id=Z2IAS5UODUXPHA__73016316fbab15ae3d5db4d2b9b240c8.test-api.com._CNAME, 30s elapsed]
Brij S
Brij Sabout 7 years ago
any ideas?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
what version of TF? what’s type of var.certs? If it’s a list of string, how many items are in it?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
#office-hours starting now! join us here https://zoom.us/s/508587304
Brij S
Brij Sabout 7 years ago
how would this line be turned into 'tf 12 syntax`
"*.${var.domain}-${element(var.certs, count.index)}.${var.aws_env == "prod" ? "com." : "dev."}"
Chris Montes
Chris Montesabout 7 years ago
Joshua Snider
Joshua Sniderabout 7 years ago
Hi everyone! We are currently using the terraform-null-label module for labels in Terraform but we are running into an issue when updating our code to v0.12
We pass "context" between modules so we have a variable defined in the module called "tags_context" and type: map
This was upgraded to type: map(string) by Terraform but then the plan doesn't work
What type should the context variable be in v0.12?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
@Joshua Snider see #announcements
Joshua Snider
Joshua Sniderabout 7 years ago
nvm, it was answered in #announcements 🙂
Sharanya
Sharanyaabout 7 years ago
Did anyone Come across NPM memory Issues ?
Daniel
Danielabout 7 years ago
Hey folks! I see that there’s a Cloud Posse container definition module, but I’m wondering if there’s an easy way to make a container_definition a reusable variable (or similar) that still supports interpolation. Use case: I have an ECS service that I run as both Fargate and EC2 using two different modules and don’t want to duplicate the container definition to keep it dry.
PePe Amengual
PePe Amengualabout 7 years ago
the container definition for Fargate and ECS are slightly different
PePe Amengual
PePe Amengualabout 7 years ago
in Fargate the Task have to set memory and CPU and the container definition too
PePe Amengual
PePe Amengualabout 7 years ago
but in ECS EC2 that is not required
PePe Amengual
PePe Amengualabout 7 years ago
and there is some other differences on the network setup etc
PePe Amengual
PePe Amengualabout 7 years ago
I think is sane to have them separated
Daniel
Danielabout 7 years ago
Hmmm, with 15+ envvars/secrets and 4 environments it feels very anti-dry to repeat it so many times.
PePe Amengual
PePe Amengualabout 7 years ago
you have them in variables ?
PePe Amengual
PePe Amengualabout 7 years ago
what is so not DRY about that ?
PePe Amengual
PePe Amengualabout 7 years ago
if you had them hardcoded I will agree
PePe Amengual
PePe Amengualabout 7 years ago
and you have 4 environments ?
PePe Amengual
PePe Amengualabout 7 years ago
so if you separate those for ENVs in it’s own TF
PePe Amengual
PePe Amengualabout 7 years ago
would you call it DRY ?
Daniel
Danielabout 7 years ago
In my structure I have a directory per env for an application
PePe Amengual
PePe Amengualabout 7 years ago
we have some cluster with like 20 different task defs
Daniel
Danielabout 7 years ago
Perhaps I should switch to using tfvars?
PePe Amengual
PePe Amengualabout 7 years ago
but in reality they should be it’s own thing
PePe Amengual
PePe Amengualabout 7 years ago
I use tfvars
Daniel
Danielabout 7 years ago
Hmmm
Daniel
Danielabout 7 years ago
I think if I reassessed how I structured this it would help my DRY concerns
Daniel
Danielabout 7 years ago
Thanks for the feedback
PePe Amengual
PePe Amengualabout 7 years ago
we populate the tfvar from SSM parameter store
PePe Amengual
PePe Amengualabout 7 years ago
when necesary
PePe Amengual
PePe Amengualabout 7 years ago
or jenkins does it
PePe Amengual
PePe Amengualabout 7 years ago
from other secret/parameter store
Daniel
Danielabout 7 years ago
Does it automatically create the tfvars based on what is in parameter store? I have my secrets name spaced with app/env/secret_name and was thinking I could write something to automatically grab all the params set for an app and put together the secrets block
PePe Amengual
PePe Amengualabout 7 years ago
you can use chamber for that
PePe Amengual
PePe Amengualabout 7 years ago
PePe Amengual
PePe Amengualabout 7 years ago
it can export to tfvar
PePe Amengual
PePe Amengualabout 7 years ago
so you can do something like
chamber write test-ec2-helloworld ecs_parameter_secret password333
PePe Amengual
PePe Amengualabout 7 years ago
where
test-ec2-helloworld
is your app/service
Daniel
Danielabout 7 years ago
Very interesting
PePe Amengual
PePe Amengualabout 7 years ago
chamber export test-ec2-hello -f tfvars                                                               1350ms  Wed 28 Aug 15:42:11 2019
ecs_parameter_secret = "password1"
ecs_parameter_string = "NOTSECRET"
PePe Amengual
PePe Amengualabout 7 years ago
or you can use chamber as ENTRYPOINT in your containers
PePe Amengual
PePe Amengualabout 7 years ago
but I don’t know how usefull is that since now you can use SM or SSM parameter store directly in the task def
Brij S
Brij Sabout 7 years ago(edited)
I'm trying to use a local variable as the bucket name:

resource "aws_s3_bucket" "remote_state" {
    bucket = local.bucket_name
    force_destroy = var.force_destroy
    acl = "private"
    
    versioning {
      enabled = var.versioning_enabled
    }

    tags = local.required_tags
}


local var is :
locals {
  bucket_name = "account-${var.aws_env}-${project_domain}-${var.aws_region}"
}


but when I try to run terraform plan I get the following error:

50:   bucket_name = "account-${var.aws_env}-${project_domain}-${var.aws_region}"

A reference to a resource type must be followed by at least one attribute
access, specifying the resource name.
Brij S
Brij Sabout 7 years ago
what does that mean?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Missing var in project_domain
Brij S
Brij Sabout 7 years ago
missing var?
Brij S
Brij Sabout 7 years ago
I have project_domain declared as a var
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
var.project_domain
Brij S
Brij Sabout 7 years ago
omg!
Brij S
Brij Sabout 7 years ago
@Andriy Knysh (Cloud Posse) cant believe I didnt see that! Thanks!
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Use a good editor with TF error detection, like JetBrains IDEA with TF plugin, or VS Code
Brij S
Brij Sabout 7 years ago
I use vscode with the terraform plugin
Brij S
Brij Sabout 7 years ago
ive been told its 'alright' though
PePe Amengual
PePe Amengualabout 7 years ago
PePe Amengual
PePe Amengualabout 7 years ago
does not work on TF 0.12
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Maybe, only examples/complete tested in 0.12 and have automatic tests for them
PePe Amengual
PePe Amengualabout 7 years ago
I had to do
assume_role_policy = "${data.aws_iam_policy_document.enhanced_monitoring.json}"
to make it work
PePe Amengual
PePe Amengualabout 7 years ago
which I found it weird
PePe Amengual
PePe Amengualabout 7 years ago
I’m new to 0.12 and I posted this in the 0.11 channel
PePe Amengual
PePe Amengualabout 7 years ago
🙄
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Weird indeed
Brij S
Brij Sabout 7 years ago
@Andriy Knysh (Cloud Posse) one more question if you dont mind!
I have a module declaration as follows:

module "s3-remote-state-bucket" {
  source = "../../modules/remote_state"

  versioning_enabled = var.versioning_enabled
  force_destroy = var.force_destroy

  aws_env = "nonprod"
  aws_region = var.aws_region
  aws_account_id_nonprod = var.aws_account_id_nonprod
  aws_account_id_prod = var.aws_account_id_prod

  providers = {
    aws = "aws.nonprod"
  }                                                         
}


in my variables.tf file in the same folder I have:

variable "aws_env" {
  description = "aws account environment"
  type = string
}


Note: no default value. but when I run apply I get asked for the aws_env:

 terraform apply
var.aws_env
  aws account environment

  Enter a value:


I dont want to use a default variable for this, this worked as intended with tf11 - anything new with tf12 maybe?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Don’t believe 0.11 didn’t ask you for a missing value :)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Nothing changed in 0.12
Brij S
Brij Sabout 7 years ago
im providing it in the module declaration though, strange
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
But maybe the var value was provided in tfvar file, or on command line, or in ENV var
Brij S
Brij Sabout 7 years ago
huh, so if I remove it from the vars file and provide the value it works as intended
Brij S
Brij Sabout 7 years ago
wild..
Phuc
Phucabout 7 years ago
hi Guys, I have some issue when running test on module creating s3
Phuc
Phucabout 7 years ago
Here is the code in my main file
Phuc
Phucabout 7 years ago

module "s3_bucket" {
  source                              = "git:xxxxx/terraform-modules/aws-s3.git?ref=terraform_0.11"
  enabled                             = "true"              
  user_enabled                        = "false"         
  allowed_bucket_actions              = []                                                             
  policy                              = ""              
  force_destroy                       = "false"        
  versioning_enabled                  = "true"          
  allow_encrypted_uploads_only        = "false"         
  sse_algorithm                       = "AES256"        
  kms_master_key_arn                  = ""             
  namespace                           = "test"
  name                                = "frontend"
  stage                               = ""
  attributes                          = []
  delimiter                           = "-"              
  tags  = {
    "BusinessUnit" = "XYZ",
    "Snapshot"     = "true"
  }
}
Phuc
Phucabout 7 years ago
and here is in variables.tf

variable "namespace" {
  type        = "string"
  #default     = ""
  description = "Namespace (e.g. `eg` or `cp`)"
}

variable "stage" {
  type        = "string"
  #default     = ""
  description = "Stage (e.g. `prod`, `dev`, `staging`)"
}

variable "name" {
  type        = "string"
  #default     = ""
  description = "Name  (e.g. `app` or `db`)"
}

variable "delimiter" {
  type        = "string"
  default     = "-"
  description = "Delimiter to be used between `namespace`, `stage`, `name` and `attributes`"
}

variable "attributes" {
  type        = "list"
  default     = []
  description = "Additional attributes (e.g. `1`)"
}

variable "tags" {
  type        = "map"
  default     = {}
  description = "Additional tags (e.g. `{ BusinessUnit = \"XYZ\" }`"
}

variable "acl" {
  type        = "string"
  default     = "private"
  description = "The canned ACL to apply. We recommend `private` to avoid exposing sensitive information"
}

variable "policy" {
  type        = "string"
  default     = ""
  description = "A valid bucket policy JSON document. Note that if the policy document is not specific enough (but still valid), Terraform may view the policy as constantly changing in a terraform plan. In this case, please make sure you use the verbose/specific version of the policy."
}

variable "region" {
  type        = "string"
  default     = ""
  description = "If specified, the AWS region this bucket should reside in. Otherwise, the region used by the callee."
}

variable "force_destroy" {
  type        = "string"
  default     = "false"
  description = "A boolean string that indicates all objects should be deleted from the bucket so that the bucket can be destroyed without error. These objects are not recoverable."
}

variable "versioning_enabled" {
  type        = "string"
  default     = "false"
  description = "A state of versioning. Versioning is a means of keeping multiple variants of an object in the same bucket."
}

variable "sse_algorithm" {
  type        = "string"
  default     = "AES256"
  description = "The server-side encryption algorithm to use. Valid values are `AES256` and `aws:kms`"
}

variable "kms_master_key_arn" {
  type        = "string"
  default     = ""
  description = "The AWS KMS master key ARN used for the `SSE-KMS` encryption. This can only be used when you set the value of `sse_algorithm` as `aws:kms`. The default aws/s3 AWS KMS master key is used if this element is absent while the `sse_algorithm` is `aws:kms`"
}

variable "enabled" {
  type        = "string"
  description = "Set to `false` to prevent the module from creating any resources"
  default     = "true"
}

variable "user_enabled" {
  type        = "string"
  default     = "false"
  description = "Set to `true` to create an S3 user with permission to access the bucket"
}

variable "allowed_bucket_actions" {
  type        = "list"
  default     = ["s3:PutObject", "s3:PutObjectAcl", "s3:GetObject", "s3:DeleteObject", "s3:ListBucket", "s3:ListBucketMultipartUploads", "s3:GetBucketLocation", "s3:AbortMultipartUpload"]
  description = "List of actions the user is permitted to perform on the S3 bucket"
}

variable "allow_encrypted_uploads_only" {
  type        = "string"
  default     = "false"
  description = "Set to `true` to prevent uploads of unencrypted objects to S3 bucket"
}
`
Phuc
Phucabout 7 years ago
the issue is, if I run terraform validate to see the resource which will be created. I will come up with this error :

  Test_s3_module terraform validate       

Error: Required variable not set: namespace



Error: Required variable not set: stage



Error: Required variable not set: name

but I already declared the value for each of that value in main.tf
this error wont show up if I put default value under the variables file like this:

variable "namespace" {
  type        = "string"
  default     = ""     <------ this line
  description = "Namespace (e.g. `eg` or `cp`)"
}

variable "stage" {
  type        = "string"
  default     = ""    <------ this line
  description = "Stage (e.g. `prod`, `dev`, `staging`)"
}

variable "name" {
  type        = "string"
  default     = ""    <------ this line
  description = "Name  (e.g. `app` or `db`)"
}
Phuc
Phucabout 7 years ago
I'm testing on TF v0.11
Phuc
Phucabout 7 years ago
Updated, I figure out the meaning of variables, The issue solved
Brij S
Brij Sabout 7 years ago
Todd Lyons
Todd Lyonsabout 7 years ago
I’m a bit flummoxed. I have a 00_remote_state.tf file that I’ve used all over the place that configures s3 for remote state for various vanilla terraform projects. I’m trying to use it now in a new project. terraform init downloads the latest aws plugin and says it succeeds, but it’s not creating the key (dns-nonprod/terraform.tfstate) that I’ve told it to up in the S3 bucket. I swear this used to work. Shouldn’t terraform init create the key up in s3? Running any terraform plan or terraform apply errors with state not found:

data.terraform_remote_state.ops_s3: Refreshing state...
data.aws_route53_zone.qa_example_net: Refreshing state...
data.aws_route53_zone.qa2_example_net: Refreshing state...
data.aws_route53_zone.dev_example_net: Refreshing state...

Error: Unable to find remote state

  on 00_remote_state.tf line 1, in data "terraform_remote_state" "ops_s3":
   1: data "terraform_remote_state" "ops_s3" {
Todd Lyons
Todd Lyonsabout 7 years ago
Does this behavior sound familiar to anybody?
Todd Lyons
Todd Lyonsabout 7 years ago
I have tried forcing older aws plugin version that worked fine previously. 2.19 and 2.14 (2.25 is latest). No change in behavior. I can paste the file here if requested. It’s just not making any sense to me.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago(edited)
1. terraform versions are the same or different?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
2. Does the user you are using to provision have the permissions to access the remote state?
johncblandii
johncblandiiabout 7 years ago
question: I am trying to build a module wrapping a resource and don’t want to provide every single argument for the resources blocks.


resource "type" "name" {
  some_block {
    blah = true
  }
}


I want to instead say:


resource "type" "name" {
  some_block = var.some_block
}


I did see possibly using a loop, but I’m not seeing a definitive answer or direction.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago(edited)
dynamic blocks (TF 0.12 only)
johncblandii
johncblandiiabout 7 years ago
perfect
giphy
giphyabout 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
if the var is not a list of objects (and you don’t want i to be a list), then you could do something like this:
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
dynamic "bootstrap_action" {
    for_each = [var.bootstrap_action]
    content {
      path = bootstrap_action.value.path
      name = bootstrap_action.value.name
      args = bootstrap_action.value.args
    }
  }
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
if you want conditionally add the block depending on some bool expression:
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
dynamic "bootstrap_action" {
    for_each = var.add_block ? [var.bootstrap_action] : []
    content {
      path = bootstrap_action.value.path
      name = bootstrap_action.value.name
      args = bootstrap_action.value.args
    }
  }
johncblandii
johncblandiiabout 7 years ago
nice
johncblandii
johncblandiiabout 7 years ago
so we still need to define the content of each individual one, but we do not need to worry w/ defining all vars
johncblandii
johncblandiiabout 7 years ago
that’s lovely
johncblandii
johncblandiiabout 7 years ago
and this works with blocks of blocks?
johncblandii
johncblandiiabout 7 years ago
can you for_each in a content block?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
yes
johncblandii
johncblandiiabout 7 years ago
good deal
johncblandii
johncblandiiabout 7 years ago(edited)
spot check (if you don’t mind):


resource "aws_msk_cluster" "this" {
  cluster_name           = var.cluster_name
  kafka_version          = var.kafka_version
  number_of_broker_nodes = var.number_of_broker_nodes
  tags                   = var.tags

  dynamic "client_authentication" {
    for_each = var.client_authentication
    content {
      dynamic "tls" {
        for_each client_authentication.tls

        content {
          certificate_authority_arns = client_authentication.tls.value.certificate_authority_arns
        }
      }
    }
  }
}


https://www.terraform.io/docs/providers/aws/r/msk_cluster.html
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
var.client_authentication should be list(object)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
tls inside it should be list(string)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
and add this:
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago(edited)
dynamic "tls" {
        for_each = toset(client_authentication.value.tls)
        iterator = item

        content {
          certificate_authority_arns = item.value.certificate_authority_arns
        }
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago(edited)
TF 0.12 has difficulties with list(string), needs it to be set(string)
johncblandii
johncblandiiabout 7 years ago
so i take it we need to flesh out the object in the variable declaration as well?


Error: Invalid type specification

  on ../variables.tf line 18, in variable "client_authentication":
  18:   type        = list(object)

The object type constructor requires one argument specifying the attribute
types and values as a map.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
yea, need to specify the var correctly:
Todd Lyons
Todd Lyonsabout 7 years ago
@Andriy Knysh (Cloud Posse) I thought I had found it.
In my s3.conf used for terraform init -backend-config=s3.conf, I had put dns-nonprod/terraform.tfstate.
And in my 00_remote_state.tf, I had put dns_nonprod/terraform.tfstate.
Todd Lyons
Todd Lyonsabout 7 years ago
I fixed the second one, and now it’s giving me the same error. This is perplexing, still working through it. Thanks for the sanity questions previously.
Todd Lyons
Todd Lyonsabout 7 years ago
In my s3.conf, I have:

bucket = "foo"
key = "dns-nonprod/terraform.tfstate"
region = "us-east-1"
Todd Lyons
Todd Lyonsabout 7 years ago
Is it possible to make the terraform_remote_state configuration read that in for the config = { ... } section? I hate having two different sources of truth (one for terraform init - the s3.conf, and one for everything else)
Robert
Robertabout 7 years ago
@Robert has joined the channel
Robert
Robertabout 7 years ago(edited)
@Todd Lyons with the consul backend you can do something like this:


terraform {
    backend "consul" {
        address = "consul.vault:8500"
        scheme  = "http"
        path    = ""
    }
}



export BACKEND_KEY=project/environment/name
terraform init -backend-config="path=$BACKEND_KEY" "$TERRAFORM_DIR"
Robert
Robertabout 7 years ago
Maybe you could do the same with the key.
Robert
Robertabout 7 years ago
🙂
Todd Lyons
Todd Lyonsabout 7 years ago
It sounds like I can only do it with something remote, not a local file. I tried reading it in using the local_file resource and then all manner of tomap() and split() and such things. Some things flat out errored, some things acted like they were going to work but then complained about the attempt I was making.
Todd Lyons
Todd Lyonsabout 7 years ago
I also found the original issue I was having. It turns out that the key in S3 isn’t actually created until the first apply is run. So when I was doing a terraform plan, the data lookup wasn’t finding it. I wonder how the heck it ever worked.
Todd Lyons
Todd Lyonsabout 7 years ago
Hmmm, maybe now that I got past that original issue, I can retry some things.
oscar
oscarabout 7 years ago
What is your goal again? And the issue
Todd Lyons
Todd Lyonsabout 7 years ago
I want to configure my s3 bucket name, key, and region in one place, that can be used both by terraform init and by terraform plan|apply|refresh|output, etc. I think I just figured it out (now that the previous issue was working on is resolved).
Todd Lyons
Todd Lyonsabout 7 years ago
I have a file, s3.conf, that has:

$ cat s3.conf
bucket = "foo"
key    = "dns-nonprod/terraform.tfstate"
region = "us-east-1"
Todd Lyons
Todd Lyonsabout 7 years ago
I initialize like this: terraform init -backend-config=s3.conf
Todd Lyons
Todd Lyonsabout 7 years ago
And my remote state is configured like this:

data "local_file" "s3" {
  filename = "${path.module}/s3.conf"
}

terraform {
  backend "s3" {
    config = tomap(data.local_file.s3.content)
  }
}

# Default provider works for the various pieces of the terraform initialization
provider "aws" {
  region  = "us-east-1"
}
Todd Lyons
Todd Lyonsabout 7 years ago
So far, it seems to be working.
Todd Lyons
Todd Lyonsabout 7 years ago
The simple solution above was just muddied by my initial error: attempting to set a data “terraform_remote_state” for an S3 key dns-nonprod/terraform.tfstate that didn’t exist yet, because I hadn’t yet run an apply (couldn’t run an apply because init failed, because that file didn’t exist yet because I hadn’t yet run an apply, circular dependencies FTW).
Todd Lyons
Todd Lyonsabout 7 years ago
I don’t know why, but I could swear that terraform init used to create that S3 key with a minimal tfstate file. I must have been wrong. This has all been with the latest aws module and terraform 0.12.4.
Todd Lyons
Todd Lyonsabout 7 years ago
Sorry for spamming the channel. I’ll exercise restraint from now on.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
i would put the backend files into a separate folder, provision it first (w/o specifying remote backend obviously b/c it does not exist yet), then add the remote backend config to the code, then terraform init will ask you to import the old (local) backend config into the remote one
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
after that, don’t touch the tf-backend folder
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
for all other modules, use diff folders
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
although outdated, the doc above will give you an idea on what needs to be done to provision the remote backend without having a remote backend to provision it in the first place
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
here is the project structure that we usually use https://github.com/cloudposse/testing.cloudposse.co/tree/master/conf
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
tfstate-backend is in separate folder and gets provisioned separately and only once
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
the script to provision the backend locally and then enable remote backend https://github.com/cloudposse/terraform-root-modules/blob/master/aws/tfstate-backend/scripts/init.sh
Todd Lyons
Todd Lyonsabout 7 years ago
I’ll surely study them. If I’m having trouble, I’ll bug my co-worker Jon and see if he can fill in the gaps.
Todd Lyons
Todd Lyonsabout 7 years ago
Thank you for all you’ve shown me.
johncblandii
johncblandiiabout 7 years ago
I’ve seen some chatter on the pre-commit-terraform working with tf 0.12, but it seems to have a problem with a simple something = var.something declaration