#terraform - February 2020

February 2020
Ola
Olaalmost 7 years ago
Thanks @Erik Osterman (Cloud Posse) this is helpful.
Prasanna Pawar
Prasanna Pawaralmost 7 years ago(edited)
@here how to do vpc peering using multiple natgateway with terraform ?
C
Cloud Possealmost 7 years ago
:zoom: Join us for "Office Hours" every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Feb 12, 2020 11:30AM.

👉️ Register for Webinar

:slack: #office-hours (our channel)
Brij S
Brij Sover 6 years ago
can outputs not have conditional count like resources do?
on ../outputs.tf line 7, in output "distribution_cross_account_role_arn":
   7:   count       = var.aws_env == "prod" ? 1 : 0

An argument named "count" is not expected here.
Adrian
Adrianover 6 years ago
what for? if you use count with "prod" in resource you will have output for prod
Brij S
Brij Sover 6 years ago
well a resource is only created if aws_env == prod, otherwise not
Brij S
Brij Sover 6 years ago
so in tht output, it only needs to output if the aws_env is prod, otherwise the resource wouldnt exist in the first place
Adrian
Adrianover 6 years ago
so you will have output if aws_env ==prod otherwise it will be empty
Brij S
Brij Sover 6 years ago
exactly
Brij S
Brij Sover 6 years ago
since that resource wouldnt exist if aws_env != prod
Adrian
Adrianover 6 years ago
e. g.
output "slack_channel" {
  value = var.enabled ? var.slack_channel : "UNSET"
}
Adrian
Adrianover 6 years ago
put some fancy text instead of "UNSET", "No output for this env" :P
Adrian
Adrianover 6 years ago
or "Valid only for prod"
Brij S
Brij Sover 6 years ago
so value = var.aws_env == "prod" ? aws_iam_role…… : "UNSET" ?
Adrian
Adrianover 6 years ago
locals {
  aws_env = "prod"
}

output "test" {
   value = local.aws_env == "prod" ? "This is prod" : "UNSET"
}

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

test = This is prod
Adrian
Adrianover 6 years ago
so yes 🙂
Rhawnk
Rhawnkover 6 years ago
Does anyone know if terraform .12.x allows for_each to loop through regions. Im attempting to create global dynamo tables in aws and figured I could save keystrokes if i use a for_each and pass the value into provider.
resource "aws_dynamodb_table" "table" {
    
  for_each = toset(var.table_regions)
  
  provider = aws.each.key
Richy de la cuadra
Richy de la cuadraover 6 years ago
Richy de la cuadra
Richy de la cuadraover 6 years ago
i did it with a lots of love
Rich Allen
Rich Allenover 6 years ago(edited)
Hi, probably a dumb question but I would like to check what I'm trying to build/fix is possible. I'm using the following module: https://github.com/cloudposse/terraform-aws-cloudfront-s3-cdn

We have a root aws account which manages the hosted zone example.com , I am trying to create a static site in a child organization at mysite.example.com. I've gone ahead and created a certificate from certificate manager in the child account. The root account has validated the certificate via DNS and I have verified the the child account has the certificate validated.

I have also set a route53 CNAME entry in the root account mysite.example.com -> ourCFDISTROID.cloudfront.net

I'm currently receiving an ERR_SSL_VERSION_OR_CIPHER_MISMATCH error. Is what I'm trying to do going to work in aws? I've hit a wall an am not sure how to proceed.
johncblandii
johncblandiiover 6 years ago
Hey folks, I’m starting to push out some videos around different devops/engineering topics. I’d love some feedback and even suggestions/requests for topics.

I’ll add links to the first few in this thread.
Igor
Igorover 6 years ago
Is there any way to work around the following errors in TF:
The "count" value depends on resource attributes that cannot be determined
until apply, so Terraform cannot predict how many instances will be created.

or
The "for_each" value depends on resource attributes that cannot be determined
until apply, so Terraform cannot predict how many instances will be created.

If TF knows the resourceA count, why wouldn't I be able to then use length(resourceA) on resourceB count...
Gabe
Gabeover 6 years ago
Hey everyone, does anyone have advice on how to best manage terraform with micro services? Do you use a monorepo with for all of the terraform? Put the terraform with the service? Why did you decide that and how has it worked out?
Dhrumil Patel
Dhrumil Patelover 6 years ago(edited)
Hello Guys I am new in terraform and stuck in problem to create elastic beanstalk application using terraform can you help me here ? Here is my code :
Dhrumil Patel
Dhrumil Patelover 6 years ago
resource "aws_elastic_beanstalk_application" "default" {
name = var.application_name
description = var.application_description
}
resource "aws_elastic_beanstalk_application_version" "default" {
name = "${var.application_name}-v1"
application = aws_elastic_beanstalk_application.default.name
description = var.application_description
bucket = var.bucket_id
key = var.object_id
}
resource "aws_elastic_beanstalk_environment" "default" {
depends_on = [aws_elastic_beanstalk_application_version.default]
name = "${var.application_name}-env"
application = aws_elastic_beanstalk_application.default.name
solution_stack_name = "64bit Amazon Linux 2018.03 v2.9.5 running Python 3.6"
version_label = "${var.application_name}-v1"
dynamic "setting"{
for_each = {"ImageId" = var.ami, "InstanceType" = var.instance_type}
content{
namespace = "aws:autoscaling:launchconfiguration"
name = setting.key
value = setting.value
}
}
}
grv
grvover 6 years ago
error message?
Dhrumil Patel
Dhrumil Patelover 6 years ago
here error message :
Dhrumil Patel
Dhrumil Patelover 6 years ago(edited)
Error: Error waiting for Elastic Beanstalk Environment (...) to become ready: 2 errors occurred:
* 2020-02-07 09:25:38.663 +0000 UTC (...) : Stack named '..' aborted operation. Current state: 'CREATE_FAILED' Reason: The following resource(s) failed to create: [AWSEBInstanceLaunchWaitCondition].
* 2020-02-07 09:25:38.781 +0000 UTC (..) : LaunchWaitCondition failed. The expected number of EC2 instances were not initialized within the given time. Rebuild the environment. If this persists, contact support.
Dhrumil Patel
Dhrumil Patelover 6 years ago
I think elastic Beanstalk environment can't communicate with instances.
Dhrumil Patel
Dhrumil Patelover 6 years ago
here creation log :
Dhrumil Patel
Dhrumil Patelover 6 years ago(edited)
2020-02-07 22:47:21 UTC+0530 INFO Launched environment: TestApp-007-env. However, there were issues during launch. See event log for details.
2020-02-07 22:47:19 UTC+0530 ERROR LaunchWaitCondition failed. The expected number of EC2 instances were not initialized within the given time. Rebuild the environment. If this persists, contact support.
2020-02-07 22:47:19 UTC+0530 ERROR Stack named '..' aborted operation. Current state: 'CREATE_FAILED' Reason: The following resource(s) failed to create: [AWSEBInstanceLaunchWaitCondition].
2020-02-07 22:30:32 UTC+0530 INFO Created CloudWatch alarm named: ..
2020-02-07 22:30:32 UTC+0530 INFO Created CloudWatch alarm named: ..
2020-02-07 22:30:16 UTC+0530 INFO Created Auto Scaling group policy named: ..
2020-02-07 22:30:16 UTC+0530 INFO Created Auto Scaling group policy named: ..
2020-02-07 22:30:16 UTC+0530 INFO Waiting for EC2 instances to launch. This may take a few minutes.
2020-02-07 22:30:16 UTC+0530 INFO Created Auto Scaling group named: ..
2020-02-07 22:29:55 UTC+0530 INFO Adding instance.. to your environment.
2020-02-07 22:29:55 UTC+0530 INFO Added EC2 instance .. to Auto Scaling Group..
2020-02-07 22:29:12 UTC+0530 INFO Created Auto Scaling launch configuration named: ..
2020-02-07 22:29:12 UTC+0530 INFO Created security group named: ..
2020-02-07 22:29:12 UTC+0530 INFO Created load balancer named: ..
2020-02-07 22:28:56 UTC+0530 INFO Created security group named: ...
2020-02-07 22:28:34 UTC+0530 INFO Using ... as Amazon S3 storage bucket for environment data.
2020-02-07 22:28:33 UTC+0530 INFO createEnvironment is starting.
Pierre-Yves
Pierre-Yvesover 6 years ago
Hello,
I am using terraform remote state and have to move one resource to its parent folder .
I am trying to use terraform state mv to avoid recreation the resource.

# download the state file
terraform state pull > local_state.out

# change the state file
terraform state mv -state=local_state.out module.network.module.vpn1.azurerm_subnet.vpn1 module.network.azurerm_subnet.vpn1

Move "module.network.module.vpn1.azurerm_subnet.vpn1" to "module.network.azurerm_subnet.vpn1"
Successfully moved 1 object(s).

but then we I do the terraform plan -state=local_state.out
Terraform still wants to delete the resource I have moved


do you have any hint on how to achieve this move ?
aaratn
aaratnover 6 years ago
@Pierre-Yves you will need to upload the state again to remote backend
aaratn
aaratnover 6 years ago
and try to do plan
aaratn
aaratnover 6 years ago
it should fix the issue
Igor
Igorover 6 years ago
Any suggested readings or words of wisdom for someone looking to get automated testing going for TF? We're looking at terratest at the moment for the tool.
C
Cloud Posseover 6 years ago
:zoom: Join us for "Office Hours" every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Feb 19, 2020 11:30AM.

👉️ Register for Webinar

:slack: #office-hours (our channel)
Matt Gowie
Matt Gowieover 6 years ago
Anyone know of a way to use data.aws_ssm_parameter to pull a number of parameters given a path? I am trying to find a way to avoid supplying all the param names to my application through vars.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 6 years ago
if any of your PRs for cloudposse repos are blocked in review, hit up our pal @Maxim Mironenko (Cloud Posse) to get help and speed up the review =)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 6 years ago
btai
btaiover 6 years ago
what kills efficiency the most working with terraform…. aws resource limits
Chris Fowles
Chris Fowlesover 6 years ago
well if that's your worst efficiency blocker i'd say you're doing aye ok
Chris Fowles
Chris Fowlesover 6 years ago
😄
johncblandii
johncblandiiover 6 years ago
I didn’t see this posted yet, but TF Cloud is adding run triggers; in short, a way to build CI pipelines.

https://www.hashicorp.com/blog/creating-infrastructure-pipelines-with-terraform-cloud-run-triggers
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 6 years ago
That's great
Chris Fowles
Chris Fowlesover 6 years ago
that looks extremely useful
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 6 years ago
@Chris Fowles: @johncblandii does a live demo in our office hours today
https://cloudposse.wistia.com/medias/g6p0zu4txy
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 6 years ago
@johncblandii you mentioned you had another video you recorded specifically demo'ing this functionality
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 6 years ago
is that on youtube?
johncblandii
johncblandiiover 6 years ago
YouTube is processing the 4K right now. Hopefully it’ll be done soon
johncblandii
johncblandiiover 6 years ago
i’ll post when it is done
Chris Fowles
Chris Fowlesover 6 years ago
awesome cheers
btai
btaiover 6 years ago
for those planning on using terraform cli workspaces with TFC (terraform cloud) because of @johncblandii’s awesome demo today, there is a tiny edge case caveat to getting it working in TFC. If you’re using the terraform.workspace value in your terraform code, that value will always be default in TFC so you won’t be able to use it to make logical decisions within your terraform code (I use it for naming conventions, tagging, environment/region specific scenarios). To work around this I’ve introduced a “workspace” variable (see pic) and you can set a local variable to workspace = "${var.workspace != "" ? var.workspace : terraform.workspace}"

The reason I am naming the variable workspace is so I can make minimal changes and it sounds like there is enough fuss from the community that this might not be an issue in the future.

More info here: https://github.com/hashicorp/terraform/issues/22131
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 6 years ago
That's annoying! Why the heck is terraform cloud overloading their own term for "workspace" making it mean one thing in the SaaS and a subtle but different thing in the terraform cli?
johncblandii
johncblandiiover 6 years ago
Gui Paiva
Gui Paivaover 6 years ago
Hey guys, I got a question about Terraform in AWS and its IAM role policy to create resources. At the moment I have attached the full admin policy to the role that terraform is using but I was wondering if there is a simpler way so terraform can create resources(not only ec2, vpc, buckets, etc) and at the same time be not so open with full admin access.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 6 years ago
but you pretty much need to be power user or admin
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 6 years ago(edited)
once you start on the path to take away permissions, you will soon realize that you need most of them to be able to provision AWS 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 6 years ago
unless you provision a specific set of resources, then you can create a specific role with those permissions and give it to the user or to terraform aws provider
Gui Paiva
Gui Paivaover 6 years ago
but then every time you need to create a new set of resources, you need to remember to update the policy
Gui Paiva
Gui Paivaover 6 years ago(edited)
quite tricky
Gui Paiva
Gui Paivaover 6 years ago
those policies from their doc are quite interesting, not just for terraform but for other users too
Gui Paiva
Gui Paivaover 6 years ago
I am going to have a second thought about it. This is not a company requirement but just trying figure out if there are better ways to manage permissions
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 6 years ago(edited)
My current opinion after having worked with this stuff for over a decade is that IAM is best suited for your services and their capabilities (controlling what they can do or wha can access them), but is too low level for restricting who (human) can deploy what (resources). It’s hard to know exactly what policies you need as a human before you provision everything. Iterating and requesting permissions is a huge bottleneck. Instead the current best practice is to stick a VCS+CI/CD (gitops) pipeline in between the humans and the infrastructure. Get the humans out provisioning stuff directly as much as possible to eliminate the need for fine grained access. Then use something like the Open Policy Agent to define the higher order policies that run in your pipelines, combined with a code review approval process.

Just to be clear, these concepts are all relatively recent developments for IaC, but are repossess to all the the problems associated with how “least privilege” failed us from a practical perspective to achieve organizational efficiency.
drexler
drexlerover 6 years ago
Hello fellas i have a question regarding CORS policies on S3 buckets. Is there a way of adding such policies to existing S3 bucket via Terraform?
Brandon Wilson
Brandon Wilsonover 6 years ago
I have a list, with one object in it, but I need to perform some functions on it and i’m trying to see if I’ve got this right…
Brandon Wilson
Brandon Wilsonover 6 years ago(edited)
aliases = [ lower(substr(“${var.service}-${var.branch}.${var.stage}.${var.domain}“, 0, 32)) ]
Brandon Wilson
Brandon Wilsonover 6 years ago
or would I run lower(substr()) on the outside of [] ?
Olivier
Olivierover 6 years ago
I am trying to use module: https://github.com/cloudposse/terraform-aws-elasticache-redis and I typed
 apply_immediately          = true

but does not seem to be part of the resource. so when I changed the parameter , it did not apply it immediately
Brij S
Brij Sover 6 years ago
does anyone know how to use terraform-docs to automatically replace only the content in the readme which it generates? (providers, inputs,outputs). For example, if there is a title and some description text at the top, I wouldnt want it to replace that part
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 6 years ago
I think @antonbabenko has a github pre-commit hook for this.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 6 years ago
Usually this is done with some kind of markers like <!-- terraform-docs begin --> and <!-- terraform-docs end --> and then using a sed+regex to replace the content inbetweeen
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 6 years ago
julien M.
julien M.over 6 years ago
Hello,
I’m currently stuck on an ALB and EC2 problem: currently I have a front-end ALB and EC2s in a Target Group that are created via an ASG + LC. (all managed with terraform)
currently when I launch a deployment with terraform it causes a downtime... because the old EC2's are already in drain mode before the new EC2's are marked “Healthy” and can therefore receive traffic.

I looked further and the new EC2's are in “initial” mode (Target registration is in progress) for a few seconds until they are considered healthy... and since at the same time the old instances are in drain mode I get timeouts (503 or 503) if I make calls at this time to the

Is there a way to make the old EC2s go into “drain” mode ONLY when the new EC2s are in “Healthy” mode and not “initial”?
This would allow the old EC2s to be able to handle the traffic while the new EC2s are OK for TargetGroup.
C
Cloud Posseover 6 years ago
:zoom: Join us for "Office Hours" every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Feb 26, 2020 11:30AM.

👉️ Register for Webinar

:slack: #office-hours (our channel)
Brij S
Brij Sover 6 years ago
Hey all, the docs for api gateway domain name(https://www.terraform.io/docs/providers/aws/r/api_gateway_domain_name.html) suggest that the var for a regional domain is regional_certificate_arn and for edge its certificate_arn . Im in the middle of creating a custom domain module for internal use. Is there a way to conditionally select regional_arn vs certificate_arn 🤔
Andrew Jeffree
Andrew Jeffreeover 6 years ago
If you're writing a module you do some conditional stuff around the endpoint configuration or lack thereof
Andrew Jeffree
Andrew Jeffreeover 6 years ago
infact hmm
Andrew Jeffree
Andrew Jeffreeover 6 years ago
no that won't work I didn't read the docs properly
Andrew Jeffree
Andrew Jeffreeover 6 years ago
I guess you could have two resources in a module using count resources
Andrew Jeffree
Andrew Jeffreeover 6 years ago
and have one that uses regional_certificate_arn and the other using just plain certificate_arn
Chris Fowles
Chris Fowlesover 6 years ago
you can pass null to unset a value
Chris Fowles
Chris Fowlesover 6 years ago
so just pass null to the one you don't want to use and use a couple of locals to work that out
Andrew Jeffree
Andrew Jeffreeover 6 years ago
ah yeah good point
Brij S
Brij Sover 6 years ago
oh really? so in the following
resource "aws_api_gateway_domain_name" "domain" {
  certificate_arn = var.certificate_arn
  regional_certificate_arn = ""

I can set one of those to null based on some conditional? @Chris Fowles
Chris Fowles
Chris Fowlesover 6 years ago
0.12 + supports an actual keyword null
Ryan
Ryanover 6 years ago
Anyone here run into a case where you provision a module, remote state in S3, try to load that remote state in another module but TF can’t find it? Unable to find remote state. State file is where I expect it to be and other modules using remote state work fine. Terraform v0.12.18.
Meb
Mebover 6 years ago
creature
creatureover 6 years ago
anybody have any tool or project recommendations on querying terraform remote state outputs? Figured I'd ask before reinventing the wheel.
PePe Amengual
PePe Amengualover 6 years ago
why wold you need to look at the outputs ?
creature
creatureover 6 years ago
to consume them for other shenanigans
PePe Amengual
PePe Amengualover 6 years ago(edited)
when you create stuff in TF then you can just do data lookup in your next TF
creature
creatureover 6 years ago
I use remote state for TF related stuff. I want to consume those attributes for use with other tools / reporting. I know I can terraform output -json and parse from there, but thought maybe someone had already written something.
creature
creatureover 6 years ago
what would be even better is a project that keeps outputs in an external store for querying. I believe Consul fills this gap, but we aren't using it so maybe something more lightweight
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 6 years ago
data "terraform_remote_state" "eks" {
  backend   = "s3"
  workspace = terraform.workspace
  config = {
    bucket               = "my-state-bucket"
    workspace_key_prefix = "eks"
    key                  = "terraform.tfstate"
    region               = var.region
  }
}

data "terraform_remote_state" "dns" {
  backend   = "s3"
  workspace = terraform.workspace
  config = {
    bucket               = "my-state-bucket"
    workspace_key_prefix = "dns"
    key                  = "terraform.tfstate"
    region               = var.region
  }
}

locals {
  eks_cluster_identity_oidc_issuer = data.terraform_remote_state.eks.outputs.eks_cluster_identity_oidc_issuer
  zone_id                          = data.terraform_remote_state.dns.outputs.zone_id
}

Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 6 years ago
or you are asking how to consume the remote state from other tools, not terraform ?
creature
creatureover 6 years ago
yes
creature
creatureover 6 years ago
I have a good grasp on remote state within TF (consuming upstream output, etc).
creature
creatureover 6 years ago
what I'm looking at writing is a Python json parser that traverses all of my state buckets, grab output and store them somewhere that I can leverage for other things
loren
lorenover 6 years ago
i think it will be somewhat dependent on that "somewhere"... e.g. i think you could just run terraform output -json | aws s3 cp <s3://bucket/key> -
loren
lorenover 6 years ago(edited)
do that in your tf pipeline, now they're in s3
creature
creatureover 6 years ago
yep, that's pretty lightweight. good idea on the pipeline also. thx!
Rajesh Babu Gangula
Rajesh Babu Gangulaover 6 years ago
@here need some advice on how can I move forward with my requirement
I need to deploy multiple lambda across multiple accounts.
I use federated login to access aws.
Terraform is in github.
I need to know if there is a way to deploy module based on conditional basis (probably var.env=prod/stage/dev) instead of using multiple branches for each account.
Brij S
Brij Sover 6 years ago
so I attempted to do the following
data "aws_api_gateway_rest_api" "api" {
  name = var.api_name
}

resource "aws_api_gateway_domain_name" "domain" {
  certificate_arn          = data.aws_api_gateway_rest_api.api.endpoint_configuration == "EDGE" ? var.certificate_arn : null
  regional_certificate_arn = data.aws_api_gateway_rest_api.api.endpoint_configuration == "REGIONAL" ? var.certificate_arn : null
  domain_name              = var.domain_name

  tags = var.tags
}

but I get an error
Error: Error creating API Gateway Domain Name: BadRequestException: A certificate was not provided for the endpoint type EDGE.

any idea of this is possible using the null keyword? I am attempted to make a module which can create a regional domain name or edge
grv
grvover 6 years ago(edited)
Is anyone using terraform-aws-rds module here of Cloudposse? i am runnign into a problem, thouhgh maybe small, have not been able to eliminate it. Hoping someone can help?
Brij S
Brij Sover 6 years ago
can we use locals in outputs, the way we can use vars? eg;
value       = var.aws_env == "prod" ? aws_iam_role.asset_distribution_cross_account.*.arn : "valid only for prod"
Todd Lyons
Todd Lyonsover 6 years ago
I’m trying to have an index count when looping through an array. The application is trying to construct the XML for an AWS MQ config file. This is what I’m doing now:
  <networkConnectors>
    %{~ for item in element(local.mq_network_brokers, count.index) ~}
    <networkConnector name="connector" userName="commonUser" uri="static:(${item})"/>
    %{~ endfor ~}
  </networkConnectors>
Todd Lyons
Todd Lyonsover 6 years ago
The issue is that name="connector" must be a unique label. If I could do equivalent of i++ and then name="connector{{ i }}", it would solve my problem. But I’m struggling trying to find a way to have a counter in that loop.
Todd Lyons
Todd Lyonsover 6 years ago
Any suggestions? Convert it to a map, use for_each, and use the key? maybe?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 6 years ago
Let’s welcome @UUB28NLDS into the mix!
ikar
ikarover 6 years ago
@UUB28NLDS help
Matt Gowie
Matt Gowieover 6 years ago
Hey Cloudposse folks — It looks like https://github.com/cloudposse/terraform-aws-cloudfront-cdn hasn’t been updated in a while. Is that no longer supported or are you folks looking for people to take up the torch on PRs like https://github.com/cloudposse/terraform-aws-cloudfront-cdn/pull/29?
Maxim Mironenko (Cloud Posse)
Maxim Mironenko (Cloud Posse)over 6 years ago
Hey @Matt Gowie! This repo is in my queue to convert to TF 0.12. It is not enough to just replace syntax but we have requirements to cover module with tests. If you want to contribute it in a proper way (see example: <https://github.com/cloudposse/terraform-aws-cloudfront-s3-cdn/pull/45>) you are most welcome! Give it a try, then ping me, so I could review and run tests. This will help us and speed up the process. Otherwise we all have to wait until it will be done as well as many other modules in queue
Matt Gowie
Matt Gowieover 6 years ago
@Maxim Mironenko (Cloud Posse) Got it — I’ll take a crack at it next week. Thanks for the info.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 6 years ago
I think we tackled ~40 PRs from our community backlog this week
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 6 years ago
There's some 210+ in total and ~130 terraform related. It'll take us another few weeks at this rate, but we're getting there!
Andrea
Andreaover 6 years ago
Hi, we have different AWS accounts for dev/stage/production etc..
Can anyone recommend the best way to keep base (eg VPCs, subnets, security groups, K8s cluster etc) and
application (RDS database, S3 buckets etc...) infrastructure, in sync across all the different AWS accounts/environments, please?
Andrea
Andreaover 6 years ago
What I thought could be done for each item (eg subnets), is that subnet ranges (different per each account) would come from input variable files,
and then the outputs would be fed into the application infrastructure. And so on for each item...
Andrea
Andreaover 6 years ago
Can anyone please recommend whether this is achievable and/or it is the best way to go with IaC and Terraform?
Andrea
Andreaover 6 years ago
anyone?
Nikola Velkovski
Nikola Velkovskiover 6 years ago
Hi Andrea, I think you are reffereing to remote states, yes that's one way to do it, the best practice is to move away the fast moving parts from the slow ones, e.g. you won't change the VPC reguraly but you might change the machine types etc.
Nikola Velkovski
Nikola Velkovskiover 6 years ago
usually what is being done is to split the "parts" into folders ( states ) and to have some sort of hierarchy between them e.g VPC needs to be applied first, ecs after the vpc etc...
Nikola Velkovski
Nikola Velkovskiover 6 years ago
Does this help ?
Andrea
Andreaover 6 years ago
Hi, thanks. I was not talking about the remote state much (even though that is something I should do at some point, currently I commit everything to git)
Andrea
Andreaover 6 years ago
so in the top directory I would have the VPC, subnets, security group etc...
Andrea
Andreaover 6 years ago
while in subfolders specific resourses and/or applications
Andrea
Andreaover 6 years ago
is there a command/script to make sure other colleagues will respect the hierarchy too?
Nikola Velkovski
Nikola Velkovskiover 6 years ago
So the hierarchy, should be imposed by the GIT e.g. PRs, README.md and predefined folder structure.
Nikola Velkovski
Nikola Velkovskiover 6 years ago
in a team you should also have "remote state backend"
Nikola Velkovski
Nikola Velkovskiover 6 years ago
and do not check-in the state in SVN
Andrea
Andreaover 6 years ago
ok about remote state and git/svn commits
Andrea
Andreaover 6 years ago
how do you impose the hierarchy with git though?
Andrea
Andreaover 6 years ago
also, and possibly lastly, what do you do when you have multiple AWS accounts/environment?
Nikola Velkovski
Nikola Velkovskiover 6 years ago(edited)

how do you impose the hierarchy with git though
Nikola Velkovski
Nikola Velkovskiover 6 years ago(edited)
You push the first commit with the hierarchy you want.
Andrea
Andreaover 6 years ago
obviously you don't want to copy and paste the whole folde/files hierarchy... per dev/test/prod etc...
Nikola Velkovski
Nikola Velkovskiover 6 years ago(edited)
you can also create a repo template ( in the case of github)
Andrea
Andreaover 6 years ago
oh I see (regarding the git commits)
Andrea
Andreaover 6 years ago
Nikola Velkovski
Nikola Velkovskiover 6 years ago

also, and possibly lastly, what do you do when you have multiple AWS accounts/environment?

You could do folders per ENV, combined with workspaces
Or workspaces with multiple providers
Nikola Velkovski
Nikola Velkovskiover 6 years ago
or just folders for everything
Andrea
Andreaover 6 years ago
a lot of food for thought! Thank you @Nikola Velkovski! I'll investigate all of those...
Nikola Velkovski
Nikola Velkovskiover 6 years ago
👍️
C
Cloud Posseover 6 years ago
:zoom: Join us for "Office Hours" every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Mar 04, 2020 11:30AM.

👉️ Register for Webinar

:slack: #office-hours (our channel)
Francisco Montada
Francisco Montadaover 6 years ago
Hi team started using the cloudposse/terraform-aws-cloudfront-s3-cdn module but I am getting this error https://gapcommerce.com/
Francisco Montada
Francisco Montadaover 6 years ago
ERROR
Failed to contact the origin.


Generated Mon, 24 Feb 2020 17:21:23 GMT
Request ID: _Gnok9D5N1_Cw1Lu7Ld44vXH78Pwj-l26vxQyWbUC6GzDIhMZxhp0w==
Francisco Montada
Francisco Montadaover 6 years ago
namespace = local.workspace["namespace"]
stage = local.workspace["stage"]
name = local.workspace["name"]
aliases = ["<http://gapcommerce.com|gapcommerce.com>", "<http://www.gapcommerce.com|www.gapcommerce.com>"]
use_regional_s3_endpoint = true
origin_force_destroy = true
cors_allowed_headers = ["*"]
cors_allowed_methods = ["GET", "HEAD", "PUT"]
cors_allowed_origins = ["*.<http://gapcommerce.com|gapcommerce.com>"]
cors_expose_headers = ["ETag"]
compress = true
Francisco Montada
Francisco Montadaover 6 years ago
this my config
Francisco Montada
Francisco Montadaover 6 years ago
I am not sure what I am doing wrong 😞
Francisco Montada
Francisco Montadaover 6 years ago
Hi @MattyB
MattyB
MattyBover 6 years ago
I don't have long but IIRC this allows you to reference your gapcommerce site like a CDN through CloudFront. Give me just a minute and I'll try to find the modules you're looking for
MattyB
MattyBover 6 years ago
What's the end goal for what you're trying to do with the ClloudPosse modules? That'll help the community properly figure out what to suggest 🙂
Francisco Montada
Francisco Montadaover 6 years ago(edited)
@MattyB We am trying to use the module to automate the provisioning CloudFront + s3 Website deployment, gapcommerce.com is our company marketing website
Francisco Montada
Francisco Montadaover 6 years ago
Francisco Montada
Francisco Montadaover 6 years ago
We are not sure what we are doing wrong https://gapcommerce.com/
MattyB
MattyBover 6 years ago
i just hit an issue with this a few weeks ago where the CORS rules on the s3 bucket were configured incorrectly. Can you post them here?
Francisco Montada
Francisco Montadaover 6 years ago
MattyB
MattyBover 6 years ago
in AWS console -> s3 bucket -> permissions -> CORS Configuration
MattyB
MattyBover 6 years ago
do you see multiple AllowedOrigins?
Francisco Montada
Francisco Montadaover 6 years ago
Sreekumar
Sreekumarover 6 years ago
Hi Team, I am trying aws beanstalk using terraform. Saw the git repository https://github.com/cloudposse/terraform-aws-elastic-beanstalk-environment but in this, it is referring the module for neanstalk env creaton. From where i can get the module code as well ?
Willie
Willieover 6 years ago
Is it not possible to enable AWS API gateway logging with just terraform? I'm getting "CloudWatch Logs role ARN must be set in account settings to enable logging" when trying to set logging_level in a aws_api_gateway_method_settings resource. The articles I've found on this suggest you need to paste a role ARN into the web console.
Sean Turner
Sean Turnerover 6 years ago
Hey all, how would one deploy to multiple aws accounts at the same time? Any way to replicate the cloudformation stackset functionality?
Milosb
Milosbover 6 years ago
Guys,
I am trying to output huge json with local-exec. It complains about argument list too long.
Is there any smooth way to go around this issue?
Dragos Andronache
Dragos Andronacheover 6 years ago
Hi all! I have more like a general usage TF question. We are starting moving to IaaC using mostly Terraform Registry modules as root ones. The way we configured it is:
From Terraform Registry github (i.e. vpc module - verified one) we make a wrap call to a private Bitbucket registry that we own;
The wrap call is basically using the module structure as it's usage sourcing to Terraform Registry module that is being locked on version. All arguments are pointing to defaulted vars;
We make the final call to the wrapped module from our infra repo that is configuring our infrastructure by filling now the desired arguments.
We have the following request: VPC registry module has predefined subnets and routes (i.e. database, redshift, elasticache etc). We would like to add to this one new subnets and routes arguments (i.e. elasticsearch, awx etc.) in the wrapped module so it will be available to all of our reusable configuration.
Can you please let me know how can we achieve this?
Thank you in advance for your answers!
Andrea
Andreaover 6 years ago
Correct me if I'm wrong, but it seems like Terraform Worspaces is not part of the OpenSource version
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 6 years ago
Confusingly - the way "workspaces" are define in Terraform Cloud & Enterprise do not map 1:1 to the workspaces in the terraform cli (e.g. terraform workspace select dev)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 6 years ago
@johncblandii can shed some more light on the differences also, our #office-hours session from 2 weeks ago
Andrea
Andreaover 6 years ago
thanks @Erik Osterman (Cloud Posse), I've started watching it...
Andrea
Andreaover 6 years ago
the presenter does not seem in super favour of TF cloud for the first couple of minutes... 🙂
Andrea
Andreaover 6 years ago
that's fine by me, as I don't think it's an option for me at this stage
Andrea
Andreaover 6 years ago
but I'm happy to learn more about it
Andrea
Andreaover 6 years ago
what about Terragrunt? is that an option commonly used for managing multiple layers (eg VPC, subnets all the way down to the application)
Andrea
Andreaover 6 years ago(edited)
for multiple environments too (QA, stage, prod etc)
Andrea
Andreaover 6 years ago
oh, a guest mentioned terragrunt too, let's wait and see what they say.. 😄
johncblandii
johncblandiiover 6 years ago
Yeah so TFC is 1 terraform workspace, but locally you can use multiple remote TFC workspaces locally as TF workspaces
johncblandii
johncblandiiover 6 years ago
Example:
project-dev
project-uat
project-prod

^ TFC has them split as such. You target prefix = "project-" in your local backend config and you’ll have locally:

$ terraform workspace list
dev
uat
prod
Andrea
Andreaover 6 years ago
ok, so workspaces in TFC and the OSS version are different things
Andrea
Andreaover 6 years ago
is there some docs on how to get started with the OSS workspaces?
Andrea
Andreaover 6 years ago
Andrea
Andreaover 6 years ago
which requires TFC...
creature
creatureover 6 years ago
I'm a fan of Terragrunt, fwiw. It solves a lot of the same problems as workspaces, and seems to be working for us so far. Keeps everything nice and DRY. (see infrastructure modules)
Andrea
Andreaover 6 years ago
OK, +1 for terragrunt, thanks for your input
Brij S
Brij Sover 6 years ago
I’m currently uploading files to a bucket as follows
resource "aws_s3_bucket_object" "object" {
  for_each      = fileset(var.directory, "**")
  ....
}

does anyone know of a clever way to output the ids of the files uploaded? 🤔
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 6 years ago(edited)
output "objects" {
  value = fileset(var.directory, "**")
}
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 6 years ago
is that not sufficient?
Brij S
Brij Sover 6 years ago
let me try that and let you know!
Brij S
Brij Sover 6 years ago
first time using for_each
Laurynas
Laurynasover 6 years ago
Hi, do you think is a better Idea to get vpc_id in different terraform stack using output of vpc stack or simply by referencing data and finding vpc by it's ID?
Igor Bronovskyi
Igor Bronovskyiover 6 years ago
https://take.ms/cPTqM
How to teach the editor to understand the syntax 0.12.x?
Nikola Velkovski
Nikola Velkovskiover 6 years ago(edited)
Which Editor @Igor Bronovskyi ?
RB
RBover 6 years ago
anyone here use private terraform modules at work? do you leverage terraform registry at all? would love to hear how you have it setup 🙂
RB
RBover 6 years ago(edited)
side question. because it's difficult to find if an aws console resource is managed by terraform... i was thinking abotu creating a terraform tag but instead of a boolean, it could have the value of the git remote . thoughts on this? and thoughts on how to retrieve this or can it only be done using a null-resource / local-exec https://stackoverflow.com/a/49425731 ?
Meb
Mebover 6 years ago
What is the best practice in case of disaster to restore your RDS base setup using terraform?
As current RDS restore from snapshot require a new base.
Could you recreate your db from your snapshot? And then when you set null snapshot_identifier it will destroy the DB.
grv
grvover 6 years ago
getting 404 on it
grv
grvover 6 years ago
Yep, was just wondering why its given in the readme section of this - https://github.com/cloudposse/terraform-aws-elasticache-redis
grv
grvover 6 years ago
in the Examples tab
RB
RBover 6 years ago
duuude put in a pr 😄
grv
grvover 6 years ago
hehe
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 6 years ago
@Maxim Mironenko (Cloud Posse) is reviewing PRs like mad right now, so if you submit it, he'll get to it.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 6 years ago
Btw, just created #pr-reviews so we can track requests better
Michał Czeraszkiewicz
Michał Czeraszkiewiczover 6 years ago
Hi, does anyone have experience with this module: https://github.com/cloudposse/terraform-aws-eks-cluster?

I have issues while using two "worker groups".

When I add a second "worker group" the networks gets crazy:
• on some pods I can't resolve DNS records
• on some pods I don't have network connectivity
• on EC2 nodes the network and DNS are fine
RB
RBover 6 years ago
anyone have luck with getting tflint to work with tf12 ?
RB
RBover 6 years ago
recently compartmentalized some terraform into its own module and had to do over 20 import statements. wrote this up into a gist to make it faster.

https://gist.github.com/nitrocode/7c2f5386f144c7b06e38c2c38292889e

is there anything like this that has already been worked on? id rather not reinvent the wheel