#terraform - October 2018

October 2018
i5okie
i5okieabout 8 years ago
hi
i5okie
i5okieabout 8 years ago
so with your beanstalk module… How can I add user data?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
.ebextensions is not enough?
i5okie
i5okieabout 8 years ago
I need to get add swap to instances as they are created. someone before me had created ebextensions to add script to add swap, but when instances are refreshed or whatever it doesnt work. also he didn’t add it to fstab so on restart no swap.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
we did it that way for swap
i5okie
i5okieabout 8 years ago
also turns out ebextensions are only executed once on environment creation (i think), and any changes later are ignored.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
it’s been a couple years since i looked at it
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
but i think there were sections that run on every build
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
and some that only ran on creation
i5okie
i5okieabout 8 years ago(edited)
hmm i’ll have to research into it more. couldn’t find any details about that yet
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
i found our old code for that
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago

commands:
  00_add_swap_space:
    command: "/tmp/add-swap-space.sh"

files:
  "/tmp/add-swap-space.sh":
    mode: "000755"
    content: |
      #!/bin/bash

      set -o xtrace
      set -e

      SWAP_SIZE=$(/opt/elasticbeanstalk/bin/get-config environment | jq -r ".SWAP_SIZE")

      if [ "$SWAP_SIZE" == "0" ];
      then
        echo "Swap is not enabled"
        exit 0
      fi

      if grep -E 'SwapTotal:\s+0+\s+kB' /proc/meminfo; then
        echo "Enabling swap space (${SWAP_SIZE} mb)"
        dd if=/dev/zero of=/var/swapfile bs=1M count=$SWAP_SIZE
        /sbin/mkswap /var/swapfile
        chmod 000 /var/swapfile
        /sbin/swapon /var/swapfile
      else
       echo "Not creating additional swap space"
      fi

i5okie
i5okieabout 8 years ago
yep thats essentially what i have
i5okie
i5okieabout 8 years ago
minus the command part
i5okie
i5okieabout 8 years ago
i5okie
i5okieabout 8 years ago
hmm so in my case the command does run, but it does nothing because swapfile already exists. it just doesn’t actually re-enable it
i5okie
i5okieabout 8 years ago
if instance is rebooted
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Just call “swapon /the/swapfile”
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
So if the instance is rebooted, it checks for the file. If it’s found it calls swapon, else it creates it and calls swap on
i5okie
i5okieabout 8 years ago
yeah i’ll add that. thank you 🙂
i5okie
i5okieabout 8 years ago
i do like your ENV for swapsize
i5okie
i5okieabout 8 years ago
i might steal that idea
i5okie
i5okieabout 8 years ago
lol
mrwacky
mrwackyabout 8 years ago(edited)
I just had a second former co-worker independently discover this comment and thank me for it
https://github.com/hashicorp/terraform/issues/9368#issuecomment-253950367
</brag>
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
haha, very nice - like the fix
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
also, props for using template_file over HEREDOCs - hate those
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
though in the particular issue, I’d argue the correct fix is to use an iam_policy_document which rather than templated JSON.
mrwacky
mrwackyabout 8 years ago
Well, in my defense, I made that comment 2 years ago, when I had 2 years less TF experience, and might predate iam_policy_document
mrwacky
mrwackyabout 8 years ago
😛
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
haha very likely!
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
😃
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
it’s awesome though.. i have been coming across a lot of members contributions/comments
mrwacky
mrwackyabout 8 years ago
Where's TF 0.12, we're dying here ?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@Maarten van der Hoef, @Daren, @Jamie Nelson (BitFlight) just in the past week
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
expect the announcement at hashiconf probably
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
end of oct
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
<----- excited but not excited
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
we have 100+ modules to update
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
by @stephen
dan
danabout 8 years ago
hey all. I'm quite confused by the ecr_repository_name variable to the terraform-aws-ecs-alb-service-task module. There are no references to it in the configuration. How does one link an ECR repo to a ECS task?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
hey @dan
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
@dan i recommend starting with the terraform-aws-ecs-web-app module for a more "turnkey" app
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
of course, if you need to do something which doesn't fit the mold, you can always use the terraform-aws-ecs-alb-service-task as a building block.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
the web app module will be a good reference implementation for you
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
man! we have too man dans for me to keep track of... 😃
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
did you take a look at the gist for the buildspec.yaml?
tamsky
tamskyabout 8 years ago
I have a quick question about
https://www.terraform.io/docs/providers/aws/r/autoscaling_group.html#waiting-for-elb-capacity

So I typically use min_elb_capacity and deploy a new LC+ASG together when config/code changes within my ASG instances.

A colleague hit me up today with this observation --
"""
Assuming a the old ASG has scaled-out (increased instance-count) due to scaling-rules
-- and because the terraform value for min_elb_capacity is not dynamic
this will cause an unwanted reduction in the instance count when the new ASG is deployed.
"""
Has anyone seen a method of propagating the current desired_capacity from the currently active ASG to the newly proposed ASG during the terraform plan phase?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
I think the best thing would be to ignore_changes to desired_capacity
tamsky
tamskyabout 8 years ago
OK, I'll try that.
Ryan Ryke
Ryan Rykeabout 8 years ago
Ryan Ryke
Ryan Rykeabout 8 years ago
looks like kinesis is in there not 100% sure what its needed for
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
@Ryan Ryke looks like the module was created almost a year ago and never updated. We can take a look at it together if you want to use it
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
@tamsky regarding desired_capacity:
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago

You may want to omit desired_capacity attribute from attached aws_autoscaling_group when using autoscaling policies. It’s good practice to pick either manual or dynamic (policy-based) scaling
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
@dan we took a look into your specific question with specifying the ECR repo with terraform-aws-ecs-alb-service-task module
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
What you want to do is use the terraform-aws-ecs-container-definition module to create a JSON task definition.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
set the container_image to the canonical “docker” url to the image
erik
erikabout 8 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
then once you have all that, pass the container definition JSON as the value of container_definition_json of the terraform-aws-ecs-alb-service-task
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@Andriy Knysh (Cloud Posse) is going to remove that confusing ecr_repository_name which is not used
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
new version released with usage examples and explanation how to setup the container image https://github.com/cloudposse/terraform-aws-ecs-alb-service-task
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
@dan ^
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
going to have @Andriy Knysh (Cloud Posse) review - then we’re good to merge
Gabe
Gabeabout 8 years ago
👍️
Gabe
Gabeabout 8 years ago
thank you
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
@Gabe thanks! merged to master
Gabe
Gabeabout 8 years ago
awesome thank you!
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Matthew
Matthewabout 8 years ago
@Andriy Knysh (Cloud Posse) I've been out of the loop on your EKS plugin, but is it production ready?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
yep
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
all modules were tested on AWS
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Matthew
Matthewabout 8 years ago
Thank you I am going to use it today
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago(edited)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
and here is a complete working example (the one we tested)
Matthew
Matthewabout 8 years ago
Thank you good sir
Andy
Andyabout 8 years ago
Under resource "aws_elastic_beanstalk_environment" "default" { you'll see similar settings
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
@Andy so just insert my settings to resource "aws_elastic_beanstalk_environment" "default" {
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
ok, thanks
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
will try this
Andy
Andyabout 8 years ago
yup or set a variable for value = "8.6.4" part so you can easily switch between versions for other apps
Andy
Andyabout 8 years ago
sorta depends on your use cases and if you'll have multiple apps or environments
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
so I need to clone repo to localhost and change it there
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
got it
Andy
Andyabout 8 years ago
yup
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
hey @Vitalii Morvaniuk, welcome
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
@Andy thanks for answering the questions 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
@Vitalii Morvaniuk if you want to open a PR to add the settings to the module, we’ll review it 🙂
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
@Andriy Knysh (Cloud Posse) sure! Will do this as fast as understood how 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
ok
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
here are the steps that should help you:
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago(edited)
1. Fork the repo into your GitHub account

2. Create a new branch

3. Modify the code in your branch

Add this

setting {
    namespace = "aws:elasticbeanstalk:container:nodejs"
    name      = "NodeVersion"
    value     = "8.6.4"
  }
 


4. Test on AWS (terraform plan/apply)

5. Open a Pull Request against our repo

6. We review and merge your changes into our repo
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
8.6.4 should be a new variable, e.g. nodejs_version
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago(edited)
setting {
    namespace = "aws:elasticbeanstalk:container:nodejs"
    name      = "NodeVersion"
    value     = "${var.nodejs_version}"
  }
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Oh, thanks - I know how to contribute using GitHub - I mean I'm still only newbie in terraform syntax
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Will add this - thanks for a guide
Andy
Andyabout 8 years ago
Don't forget to add to variables.tf as well 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
yep
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
and then rebuild README by executing the following commands in the module folder:
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
make init
make readme/deps
make readme
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
Anyone know if there is a terraform module(+lambda) out there for AWS service limit monitoring ?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
That would be a great one!
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Haven’t seen though...
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
I want a module for billing alerts too
GFox)(AWSDevSecOps
GFox)(AWSDevSecOpsabout 8 years ago
My requirement is to not to delete and not even trying to delete, since S3 is the backend. Currently it's trying to delete and failing since versioning is enabled.
Was trying to set a deletion policy to “retain” (S3) running terraform. (moot)
Was wondering if there is option to tell terraform not to delete backend S3 ??
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@GFox)(AWSDevSecOps can you add some additional context? sounds like you might be talking about the terraform-aws-tfstate-backend module?
GFox)(AWSDevSecOps
GFox)(AWSDevSecOpsabout 8 years ago
Thank you @Erik Osterman (Cloud Posse), working on it, I'm not an aws guy yet, more azure and openstack, but helping a friend right now, while l'm reading up looking into it, and, when I get a better picture, will def ping you back 🙂
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Aha then yes there are a few options
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Sorry, I see now more clear what you are asking
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
So when you create the bucket resource you will want to add a lifecycle block that says prevent destroy
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
@GFox)(AWSDevSecOps you need to put tfstate-backend into a separate folder, provision it using TF local state, then import the state into the tfstate-backend bucket
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
all other modules are in different folders, so when you run terraform destroy on them, TF will not attemp to destroy the state bucket
GFox)(AWSDevSecOps
GFox)(AWSDevSecOpsabout 8 years ago
wow, quick responses, great help and great stuff 🙏
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
@Andriy Knysh (Cloud Posse) PR for adding NodeJS version select https://github.com/cloudposse/terraform-aws-elastic-beanstalk-environment/pull/50
dan
danabout 8 years ago(edited)
Thanks for the help here on the terraform-aws-ecs-web-app ECR issue a few days ago!

I'm now wondering if I've misunderstood the use-case for the module. Is it ok to have the instances it defines live on a public subnet? The only way to specify the subnets is via ecs_private_subnet_ids, which leads me to believe it should only be used on private subnets. In my use-case, the containers need internet access. When I look under the hood, I don't see any reason for the "private" qualifier. Is it just a poorly-named variable?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
it was designed with the expectation that the tasks are on a private subnet
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
which means your VPC needs a NAT gateway to reach the internet
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
have you seen our VPC and subnet modules? Those will take care of it
dan
danabout 8 years ago
Ah, thanks. Will look now.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
(though I suppose it will just work if you give it public subnet IDs)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
even though it's called private_... (just a hunch)
dan
danabout 8 years ago
that's what I was hoping for. Will report back.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
😃
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
if not, and you want to submit a PR, will review it promptly
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Hi again. When trying to
terraform destroy
I'm getting several errors:
module.elastic_beanstalk_environment.aws_s3_bucket.elb_logs (destroy): 1 error(s) occurred: aws_s3_bucket.elb_logs: error deleting S3 Bucket (develop-dev-vitaliimorvaniukdev-logs): BucketNotEmpty: The bucket you tried to delete is not empty status code: 409
module.dev_front_end.module.logs.aws_s3_bucket.default (destroy): 1 error(s) occurred: aws_s3_bucket.default: error deleting S3 Bucket (develop-dev-front-dev-vitalii.morvaniuk-dev-logs): BucketNotEmpty: The bucket you tried to delete is not empty. You must delete all versions in the bucket. status code: 409
force_destroy = true 
is setted
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
what should I check ?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
was force_destroy = true set from the very beginning?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
oh, this is a tricky thing i discovered a few months ago.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
if you change force_destroy = true but do not terraform apply, it doesn’t register
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
@Vitalii Morvaniuk and if the rest has been deleted already, best is to do a terraform apply_target on module.elastic_beanstalk_environment.aws_s3_bucket.elb_logs with the force_destroy = true option. And then the destroy again.
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Yes force_destroy = true was set from the beginning
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
😕
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
[empty bucket]-button
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Its what I'm doing now - but I'd like it to be deleted automatically
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
i’m not quite sure what to look into
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
in older versions of terraform, it was common that versions weren’t force deleted
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
terraform --version
Terraform v0.11.7
+ provider.aws v1.40.0
+ provider.null v1.0.0
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
but that’s been working for me as of relatively recently - using our terraform-aws-tfstate-backend module which has versions
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
Erik, you have a long history of tf issues 🙂
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
hah, that’s ironic
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
since v0.6.16
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
its more tragic hehe, it's super old
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Ok, one more question - I need to add RDS instance to my beanstalk environment - as far as I see there are no option for this - should I do it separatly and after that add something like this:

setting {
    namespace = "aws:elasticbeanstalk:application:environment"
    name = "RDS_USERNAME"
    value = "${aws_db_instance.rds-app-prod.username}"
  }
  setting {
    namespace = "aws:elasticbeanstalk:application:environment"
    name = "RDS_PASSWORD"
    value = "${aws_db_instance.rds-app-prod.password}"
  }
  setting {
    namespace = "aws:elasticbeanstalk:application:environment"
    name = "RDS_DATABASE"
    value = "mydb"
    value = "${aws_db_instance.rds-app-prod.name}"
  }
  setting {
    namespace = "aws:elasticbeanstalk:application:environment"
    name = "RDS_HOSTNAME"
    value = "${aws_db_instance.rds-app-prod.endpoint}"
  }
Andy
Andyabout 8 years ago
Andy
Andyabout 8 years ago
And then just pass RDS Hostname username/password etc as variables to the environment
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Ok, thanks
Andy
Andyabout 8 years ago
https://github.com/cloudposse/terraform-aws-rds so something like that and then you can use the outputs
Miguel Mendez
Miguel Mendezabout 8 years ago
First let me say thanks for contributing great terraform modules. I have a question though. I need a simple redis elastic cache cluster. I have looked at https://github.com/cloudposse/terraform-aws-elasticache-redis but this seems to be geared towards clusters with replication such as https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/Replication.Redis.Groups.html#Replication.Redis.Groups.Cluster. Is this correct or my assumptions are wrong? Thanks in advance for any feedback.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
hey @Miguel Mendez
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Miguel Mendez
Miguel Mendezabout 8 years ago
OK, any plans supporting it?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Most of our modules are driven either by client engagements or community contributions. As of right now, we don’t have any clients asking for it :-)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
We are really good about promptly reviewing PRs and nearly accept all contributions. If you’re interested, please take a stab at it!
Miguel Mendez
Miguel Mendezabout 8 years ago
OK great. I will create then a module and submit a PR. Thanks once again for your contributions.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Awesome @Miguel Mendez ! Thanks for taking a look at it
nicgrayson
nicgraysonabout 8 years ago
Hey all, I'm trying to get https://github.com/cloudposse/terraform-aws-eks-cluster to work. No nodes are showing up when i do kubectl get nodes. The userdata log on the instance looks fine.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@nicgrayson did @Andriy Knysh (Cloud Posse) share the reference implementation?
nicgrayson
nicgraysonabout 8 years ago
I'm using tf from the readme
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
^ what we deployed and tested
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago(edited)
you need to apply the ConfigMap for the worker nodes to join the cluster
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
(EKS does not do it automatically)
nicgrayson
nicgraysonabout 8 years ago
ah okay thanks
nicgrayson
nicgraysonabout 8 years ago
Worked like a charm 👍️
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
nice
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
let us know if any issues
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Hi all. I'm trying to create RDS instance using https://github.com/cloudposse/terraform-aws-rds

module "elastic_beanstalk_rds" {
  source      = "git::<https://github.com/cloudposse/terraform-aws-rds.git?ref=master>"
  namespace   = "${var.namespace}"
  stage       = "${var.environment}"
  name        = "${var.user_account_name}"
  dns_zone_id = "${var.parent_zone_id}"
  host_name                   = "db"
  security_group_ids          = ["${module.vpc.vpc_default_security_group_id}"]
  database_name               = "app"
  database_user               = "admin"
  database_password           = "password"
  database_port               = 5432
  multi_az                    = "false"
  storage_type                = "gp2"
  allocated_storage           = "5"
  engine                      = "postgresql"
  engine_version              = "9.6.6"
  instance_class              = "db.t2.micro"
  db_parameter_group          = "default.postgres9.6"
  #parameter_group_name        = "default.postgres9.6"
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
But getting
module.elastic_beanstalk_rds.aws_db_instance.default: Resource 'aws_db_parameter_group.default' does not have attribute 'name' for variable 'aws_db_parameter_group.default.*.name'
error
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
what I'm doing wrong ?
jarv
jarvabout 8 years ago
try passing a list instead
jarv
jarvabout 8 years ago(edited)
see if that works, unfamiliar with the module but might point you in the right direction if so
jarv
jarvabout 8 years ago
er hm, likely from the commented out parameter_group_name
jarv
jarvabout 8 years ago
looks like it's checking for the length of that here: https://github.com/cloudposse/terraform-aws-rds/blob/master/main.tf#L54
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
hey @Vitalii Morvaniuk and @jarv
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
if parameter_group_name is not provided, the default is "" (empty string)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
it looks like a race condition
jarv
jarvabout 8 years ago(edited)
oh, hm didn't see that.. so guess just a depends_on should prevent that as well can ignore this suggestion probably, think I need more sleep
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
looks like it’s the ‘famous’ issue with TF counts after the latest release (count expressions have been changed)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
@Vitalii Morvaniuk can you try the previous release source = "git::<https://github.com/cloudposse/terraform-aws-rds.git?ref=tag/0.4.0>"
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
that one was working for a long time
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
and we’ll look into the issue with the latest release 0.4.1
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago(edited)
also, don’t use master in your code (git::<https://github.com/cloudposse/terraform-aws-rds.git?ref=master>), always pin to a release (for all modules). Modules get changed and there is a possibility of regression
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
thanks for tips - will try
shaiss
shaissabout 8 years ago
does anyone have an example for using the event selector in the cloudtrail module?
https://registry.terraform.io/modules/cloudposse/cloudtrail/aws/0.5.0?tab=inputs
trying to capture all S3 events https://www.terraform.io/docs/providers/aws/r/cloudtrail.html#event_selector
dan
danabout 8 years ago
has anyone here used the terraform-aws-alb and terraform-aws-alb-ingress modules to configure a load balancer to redirect http to https? It's an obvious option when adding an ingress rule via the AWS UI, but I'm lost finding the equivalent option in the cloudposse modules. I feel like I'm missing something simple...
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@dan we don’t have this implemented right now. @Ryan Ryke wanted the same thing. Not sure what he ended up doing.
dan
danabout 8 years ago
@Erik Osterman (Cloud Posse) good to know - thanks
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Looks like support for this was released in August (after our first look at it)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago

Support for fixed-response and redirect actions has been merged into master via #5430 and will release with version 1.33.0 of the AWS provider, later this week. (August 20)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@dan if you get a chance to implemented, we would love a PR
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago

resource "aws_lb_listener" "front_end" {
  load_balancer_arn = "${aws_lb.front_end.arn}"
  port              = "80"
  protocol          = "HTTP"

  default_action {
    type = "redirect"
    redirect {
      port = "443"
      protocol = "HTTPS"
      status_code = "HTTP_301"
    }
  }
}
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@dan ☝️
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@LeoGmad I think you guys might dig this too….
LeoGmad
LeoGmadabout 8 years ago
@LeoGmad has joined the channel
Ryan Ryke
Ryan Rykeabout 8 years ago
yeah, i would be interested in that. i can try and put a pr in this weekend. for now they are handling it inside the container
Ryan Ryke
Ryan Rykeabout 8 years ago
not totally sure how you would implement it though
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@Ryan Ryke I think we would add this to the terraform-aws-alb module


resource "aws_lb_listener" "https_redirect" {
  count = "${var.https_redirect_enabled == "true" ? 1 : 0}"
.... # code from above
}

Somewhere here.
https://github.com/cloudposse/terraform-aws-alb/blob/master/main.tf#L27

And then a ternary here to select the appropriate ARNs:
https://github.com/cloudposse/terraform-aws-alb/blob/master/outputs.tf#L38
Ryan Ryke
Ryan Rykeabout 8 years ago
ahh right something like that might make sense
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
(or maybe return 0 ARNs for HTTP if redirect enabled)
Ryan Ryke
Ryan Rykeabout 8 years ago
when is .12 coming out?
Ryan Ryke
Ryan Rykeabout 8 years ago
maybe hashiconf?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
yea, that’s my expectation
ff
ffabout 8 years ago
hey 👋
ff
ffabout 8 years ago
quick one
ff
ffabout 8 years ago
let’s say I want to only deploy your beautiful jenkins terraform solution
ff
ffabout 8 years ago
what is a good practice to start?
ff
ffabout 8 years ago
copy the main.tf from the examples folder to the root of the project and run terraform init .. plan .. apply?
ff
ffabout 8 years ago
I wasnt able to find that skimming the readme
ff
ffabout 8 years ago
thx
ff
ffabout 8 years ago
for now I created a deploy folder and run terraform init deploy/ etc.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
hey @ff
ff
ffabout 8 years ago
hey 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
so I think you already have your project structure similar to this https://github.com/cloudposse/terraform-root-modules/tree/master/aws
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
each app/module should be in a separate folder
ff
ffabout 8 years ago
got it
ff
ffabout 8 years ago
ah I see
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
you add for example jenkins folder
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
and copy our example in there
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago(edited)
then cd to the folder and run terraform plan/apply
ff
ffabout 8 years ago
thanks that was the hint I was missing
ff
ffabout 8 years ago
back to work 👍️
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
1 sec 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
and pin all modules to a release
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
don’t use master
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
master can be changed/updated and there is a possibility to introduce regression
ff
ffabout 8 years ago
got it
ff
ffabout 8 years ago
this is awesome stuff … how did we infrastructure back in the 2000s 🤩
ff
ffabout 8 years ago
or even further back
Mark Mutti
Mark Muttiabout 8 years ago
@ff That is terrifying to think about. We did it very badly.
ff
ffabout 8 years ago
lots of metal and cables 😆
ff
ffabout 8 years ago
and sometimes even documentation
Mark Mutti
Mark Muttiabout 8 years ago
Oh man, and SSHing into a million machines to apply patches, config drift, you name it
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
or DRBD 🙂
ff
ffabout 8 years ago
hey again… making some progress with the jenkins-terraform
ff
ffabout 8 years ago
but
ff
ffabout 8 years ago
ending up here
ff
ffabout 8 years ago
ff
ffabout 8 years ago
I already learned about the other issue with the empty value for the github auth token and added a - in the value as described in https://github.com/cloudposse/terraform-aws-jenkins/issues/11
ff
ffabout 8 years ago
but here is a dead end. I don’t see a name tag either. Please advise 🙂
ff
ffabout 8 years ago
I now worked - it was obvious after asking the question here - using another tag name and value (before: Terraform = “true”, after Department = “abc”)
ff
ffabout 8 years ago
whatever 🙈
ff
ffabout 8 years ago
It only works once. Once the tag has been set, the next run fails.
ff
ffabout 8 years ago
this is my config - please help, I am at a dead end
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
hey @ff what’s exactly are you seeing? terraform apply first time works, but second time fails?
Ryan Ryke
Ryan Rykeabout 8 years ago
anyone have a good read replica module
Ryan Ryke
Ryan Rykeabout 8 years ago
rds read replica
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@Ryan Ryke hrmmm I believe we recently made some fixes to support this use-case
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Trying to remember who here was working in that
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@Andriy Knysh (Cloud Posse) do you remember who?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Gladly?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Daren gave us the example code that they used, but it was to help someone else
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Let me check
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Here’s the link to the discussion
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Maybe some more juice there
Ryan Ryke
Ryan Rykeabout 8 years ago
right, so no module just a raw resource
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
@Erik Osterman (Cloud Posse) we helped Gladly with this module terraform-aws-rds-replica-vpc
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
they have it, but it’s not public
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Yea, so IMO not sure it makes sense to have a module for it
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
This customer created a private module that does this by creating a vpc, subnet and RDS instance configured as a replica
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
But then that means that vpc should be used basically for nothing else
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
yes, that was mostly for cross-region replica
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
so more complex than prob needed
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
This is good for checking the box on a pci compliance box
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@Ryan Ryke can you provide the use-case you are solving?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
@Ryan Ryke if you use Aurora, you don’t need all of that 🙂
Ryan Ryke
Ryan Rykeabout 8 years ago
they just want a read replica in prod
Ryan Ryke
Ryan Rykeabout 8 years ago
also true
Ryan Ryke
Ryan Rykeabout 8 years ago
i dont need a module for it
Ryan Ryke
Ryan Rykeabout 8 years ago
there really isnt a whole lot of tf theere
Ryan Ryke
Ryan Rykeabout 8 years ago
i modified @Andriy Knysh (Cloud Posse)'s sample and im done
Ryan Ryke
Ryan Rykeabout 8 years ago
10 minutes
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
replica for plain RDS is mostly useful if you need cross-region replication
Ryan Ryke
Ryan Rykeabout 8 years ago
agreed, they want to hit a separate endpoint for reporting
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
ok, then you need it
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
why not Aurora? (historical reasons?)
Ryan Ryke
Ryan Rykeabout 8 years ago
just comfort level for them
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
yea, but Aurora is faster 🙂
joshmyers
joshmyersabout 8 years ago
Terraform and encrypted cross-region read replicas used to be a pain IIRC - not sure if still the case
joshmyers
joshmyersabout 8 years ago
Also Aurora you don’t really have to care about storage space again 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago(edited)
lvh
lvhabout 8 years ago
is there a cloudposse e2e terraform module that just gets me an ecs cluster I can run some one-off (and scheduled, but not persistent) tasks on? looks like the most plausible public one is https://github.com/arminc/terraform-ecs
lvh
lvhabout 8 years ago
(or, ya know, start writing your own terraform :))
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
yes/no
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
we’ve taken a more decomposed approach to ECS fargate
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
do you require CI/CD?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
this module does most of the heavy lifting
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
you give it a vpc and subnets, and it deploys a traditional “webapp”
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
it takes care of setting up the service task, task definition, codebuild, codepipeline, alb rules, autoscaling,etc
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
however, if you have something very simple - where you basically just want “heroku” but on AWS - there’s something else to consider
lvh
lvhabout 8 years ago
Yeah; I saw that one -- it seems to do almost everything besides the cluster, and it's the cluster I'm really after 🙂
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
the cluster is a 1-liner
lvh
lvhabout 8 years ago
OTOH what eventually want is a cron job running a container with some persistent storage, so maybe I should just run k8s instead
lvh
lvhabout 8 years ago
the cluster resource itself is a oneliner but the instances in it aren't, right?
lvh
lvhabout 8 years ago
you still need an autoscaling group and a launch configuration etc
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
oh, we only deal with Fargate
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
😃
lvh
lvhabout 8 years ago
heh that'll do it -- what do you do for persistence in fargate? iirc it doesn't support docker volumes
lvh
lvhabout 8 years ago
just bind mounts
lvh
lvhabout 8 years ago
and IIRC you can't make the bind mount just point at an EBS volume you control
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
yea, that’s still a drawback. we’re mostly a kubernetes shop.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
in an “ideal world” you don’t need the durability in your containers and can offload that to an external backing service ala S3 with object storage
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
but i realize that’s not right for everything
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
lvh
lvhabout 8 years ago
i mean, I can mostly use goofys I think, but it sounds easier to just use k8s + ebs storage in the pod
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
I’m using this for simple off off stuff.
lvh
lvhabout 8 years ago
none of the apps that I'm currently using reallllly want a POSIX fs
lvh
lvhabout 8 years ago
it's more of a key value store that maybe uses fopen
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
yea, goofys is a hack
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
plus it requires elevated capabilities for fuse
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
not sure if it will work on ECS Fargate
lvh
lvhabout 8 years ago
as a security person I don't want fuse anywhere near my containers tbh
lvh
lvhabout 8 years ago
OK: you convinced me, time to deploy some k8s
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
we’ve only used goofys as a last resort
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
we have EKS modules too 😃
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@jsanchez
jsanchez
jsanchezabout 8 years ago
@jsanchez has joined the channel
George
Georgeabout 8 years ago(edited)
Hey guys, thanks for all the work you've put into the modules on github, it's an awesome collection. I'm trying to build out a pipeline to deploy an ecs cluster, and the target group that is created seems to have no targets - I'm having trouble digging through and finding a reason that might cause it. Has anyone run into similar before? I'm using terraform-aws-alb, terraform-aws-ecs-web-app, and terraform-aws-ecs-alb-service-task
George
Georgeabout 8 years ago
^ (semi) solved it myself - tasks were never working in the ecs cluster (unable to access my ECR to pull the image, unsure why), so there never were any targets to register. Womp.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@George thanks so much!
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Sounds like you figured it out
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
also, did you see our example buildspec.yaml?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
George
Georgeabout 8 years ago
I did, wasn’t quite sure how I would integrate it into what I was using
George
Georgeabout 8 years ago
(Or if it replaced some or all of the components, aside from the vpc and subnets)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Hrm… I thought we had a full example somewhere
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
looking
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
here’s a more complete example that helped @Ryan Ryke get up. We really need to add this to our terraform-root-modules
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
George
Georgeabout 8 years ago
I’ll check it when I get back to my machine, thanks for the heads up!
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
cool - just ping me if you’re stuck
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
TIL:

output "sensitive" {
  sensitive = true
  value     = VALUE
}
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago

Note that this is mostly useful in the CI scenario as anyone with access to the state can always terraform output or read it directly.
Ryan Ryke
Ryan Rykeabout 8 years ago
yeah i can help if needed
ff
ffabout 8 years ago
Found some weird behaviour and managed to file a new issue even with the current Github outage. https://github.com/cloudposse/terraform-aws-rds-cluster/issues/37
ff
ffabout 8 years ago
thanks for looking into it 🙂
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
@ff try without the availability_zones
ff
ffabout 8 years ago
It does not accept - it’s a mandatory variable
ff
ffabout 8 years ago
been there 😉
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
and with availability_zones = []
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
ff
ffabout 8 years ago
stupid me
ff
ffabout 8 years ago
tryin’
ff
ffabout 8 years ago
nope
ff
ffabout 8 years ago
availability_zones.#: “3” => “0" (forces new resource)
availability_zones.1126047633: “eu-central-1a” => “” (forces new resource)
availability_zones.2903539389: “eu-central-1c” => “” (forces new resource)
availability_zones.3658960427: “eu-central-1b” => “” (forces new resource)
ff
ffabout 8 years ago
and in turn the cluster nodes are also forced new resoruces
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
just remove everything, and create again without az's
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
then it should work and keep working
ff
ffabout 8 years ago
Got ya. Testing.
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
🙂
ff
ffabout 8 years ago
first w/o terraform destory
ff
ffabout 8 years ago
let’s see
ff
ffabout 8 years ago
thanks for the hint
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
sure np, wanted to do a quick pr, but GH is still suffering it seems
ff
ffabout 8 years ago
good old single points of failure 😄
ff
ffabout 8 years ago
will report back in about 15mins when the “apply” has finished
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
haha 🙂
ff
ffabout 8 years ago
does not work
ff
ffabout 8 years ago
🙂
ff
ffabout 8 years ago
availability_zones.#: “3” => “0" (forces new resource)
availability_zones.1126047633: “eu-central-1a” => “” (forces new resource)
availability_zones.2903539389: “eu-central-1c” => “” (forces new resource)
availability_zones.3658960427: “eu-central-1b” => “” (forces new resource)
ff
ffabout 8 years ago
availability_zones = []
ff
ffabout 8 years ago
shall I use a terraform destroy and rebuild the environment?
ff
ffabout 8 years ago
but should not make a difference I think
ff
ffabout 8 years ago
so we should be able to actually drop the variable
ff
ffabout 8 years ago
because I assume that an empty bracket means something else than a non existing variable
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
ah sorry man, i think you're right
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
@Andriy Knysh (Cloud Posse) do you have thoughts on this
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Let me check
ff
ffabout 8 years ago
for the time being I worked around it by using native terraform aws resources
ff
ffabout 8 years ago
nevertheless I thought it might be helpful to fix this for the community
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
Yea, I think we should just drop that variable from the module altogether
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@ff if we do that, does it fix your problems?
George
Georgeabout 8 years ago
We currently use 0.6 in prod (from before my time) and I’ve been tasked with upgrading to current. State files stored in s3. Anyone got any general guides or info sources on how to go about upgrading? I see projects like terraforming etc, and/or regeneration of state files.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
terraform 0.6.x -> 0.11.x?
George
Georgeabout 8 years ago
(I’ve been tasked with can be read as “hey I wanna do some terraform” “ok here” “no wait wha-“
George
Georgeabout 8 years ago
@Erik Osterman (Cloud Posse) yes
George
Georgeabout 8 years ago
*0.7 actually, my bad
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
hrm… haven’t had to do that big of a jump
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
most important thing is to backup the state files so you have recourse
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
typically, terraform cli is great about upgrading (never downgrading) tf state
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
there maybe some syntax change, but you’ll be warned by terraform of those
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
objective should be to run a terraform plan and see no changes; however, sometimes terraform changes default values for things in which case you will see changes.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
i think in the CHANGELOG for every release they publish an upgrade path (if necessary)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
i would put together some notes after combing though those release notes that can be you run book
George
Georgeabout 8 years ago
Hmm, so back up the state files, upgrade terraform binary, and terraform plan until it shows no errors?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
yea, that’s all I can think of off the top of my head
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@Maarten van der Hoef @Jamie Nelson (BitFlight) any tips?
Jamie Nelson (BitFlight)
Jamie Nelson (BitFlight)about 8 years ago
yo
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@George is upgrading some legacy infra from 0.7.x -> 0.11.x; any words of wisdom?
Jamie Nelson (BitFlight)
Jamie Nelson (BitFlight)about 8 years ago
Wait for 12?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
HA
George
Georgeabout 8 years ago
Lol
Jamie Nelson (BitFlight)
Jamie Nelson (BitFlight)about 8 years ago
It comes with an upgrade script
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
good point
George
Georgeabout 8 years ago
Oh, seriously?
Jamie Nelson (BitFlight)
Jamie Nelson (BitFlight)about 8 years ago
ya
George
Georgeabout 8 years ago
Lemme go investigate
Jamie Nelson (BitFlight)
Jamie Nelson (BitFlight)about 8 years ago
the jump to 11 breaks a lot of "sloppy" code practices of older versions
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
pre-release available!
Jamie Nelson (BitFlight)
Jamie Nelson (BitFlight)about 8 years ago
11 stopped allowing things like output references that are lists when the lists are empty
Jamie Nelson (BitFlight)
Jamie Nelson (BitFlight)about 8 years ago
and we spend half of our time working around that with ugly interpolations
Jamie Nelson (BitFlight)
Jamie Nelson (BitFlight)about 8 years ago
11 also broke 'count', so that calculated count values arent allowed in modules
Jamie Nelson (BitFlight)
Jamie Nelson (BitFlight)about 8 years ago
These things will likely still break in 12, but the fixes are much more elegant
Jamie Nelson (BitFlight)
Jamie Nelson (BitFlight)about 8 years ago
in terraform 10 you can do

output "the_alb_name" {
value = "${aws_ecs_service.app_with_lb_awsvpc.*.name}"
}
Jamie Nelson (BitFlight)
Jamie Nelson (BitFlight)about 8 years ago
and if aws_ecs_service.app_with_lb_awsvpc wasn;t created... it calmly shows an empty value
Jamie Nelson (BitFlight)
Jamie Nelson (BitFlight)about 8 years ago
without crying about it
Jamie Nelson (BitFlight)
Jamie Nelson (BitFlight)about 8 years ago
in 11, if you want your output to work with an empty list from a resource, then its all:

output "the_alb_name" {
value = "${join("",compact(concat(list(""),aws_ecs_service.app_with_lb_awsvpc.*.name)))}"
}
Jamie Nelson (BitFlight)
Jamie Nelson (BitFlight)about 8 years ago
to get the same output without an error
George
Georgeabout 8 years ago
Hmm, I’ll have to review the erm...state of the current configs we have. Unsure if my predecessor wrote anything that takes advantage of those features. Thanks for the explanation and heads up about 0.12!
Jamie Nelson (BitFlight)
Jamie Nelson (BitFlight)about 8 years ago
Your predesessor will have taken advantage of them
Jamie Nelson (BitFlight)
Jamie Nelson (BitFlight)about 8 years ago
since pre 11, that was just how it was done.. in all the terraform examples too
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
thanks @Jamie Nelson (BitFlight)! great summary
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)

11:26 Pacific Daylight TimeWe have resumed webhook delivery and will continue to monitor as we process the backlog of events. <- github
Jamie Nelson (BitFlight)
Jamie Nelson (BitFlight)about 8 years ago
Ha, thanks 😛 Sorry @George I would give you more guidance on upgrade to 0.11, but 0.12 is a breaking change anyway and you might as well just rewrite your code once instead of twice.
Jamie Nelson (BitFlight)
Jamie Nelson (BitFlight)about 8 years ago
You may also find there is very little to rewrite after the upgrade tool is out as well.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
🤞
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
I added a #atlantis channel since we're doing a lot more with it these days (related to runatlantis.io).
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
hot off the press!
joshmyers
joshmyersabout 8 years ago
Any plans to include that emailcli into packages?
joshmyers
joshmyersabout 8 years ago
Nvm @Erik Osterman (Cloud Posse) just seen https://github.com/cloudposse/packages/pull/95 👍️
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
thanks @joshmyers!
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
actually, we have a PR for it already - checking status
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Yea, we’ll get that merged tomorrow probably
joshmyers
joshmyersabout 8 years ago
Given you can't invoke modules with counts and TF isn't an ideal language as yet, how do you see using/invoking that module? The example mentions creation of users, which I'd imagine is done as a list of vars. A module per user? I see other use cases but was interested in that as I have run into similar use case before.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
i think we’re going to move away from the list-of-users model
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
and instead do one tf file per user
joshmyers
joshmyersabout 8 years ago
Agreed that would be more flexible at the moment although I'd hope 0.12 with hcl2 would improve that with rich data structures...
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
i’m getting this setup for our demo on thursday
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
IAM user account management with #atlantis
joshmyers
joshmyersabout 8 years ago
Ahh, nice. 👍️
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
so, in this case, would invoke with each user.
joshmyers
joshmyersabout 8 years ago
I see how that all fits together now
joshmyers
joshmyersabout 8 years ago
Nice way of doing it
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
what i don’t like is all the vars I have to pass related to smtp
joshmyers
joshmyersabout 8 years ago
Modules all the way down
joshmyers
joshmyersabout 8 years ago
You could infer some of them if not otherwise provided, then again...HCL :(
joshmyers
joshmyersabout 8 years ago
Also, I'm on mobile right now.
loren
lorenabout 8 years ago(edited)
heads up, v0.11.10 was just released and the download urls are different... they no longer have the v in the version token... works:
• https://releases.hashicorp.com/terraform/0.11.10/terraform_0.11.10_linux_amd64.zip
doesn't work:
• https://releases.hashicorp.com/terraform/v0.11.10/terraform_v0.11.10_linux_amd64.zip
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@loren ugh, thanks for the heads up
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
sucks for our packaging system
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
wonder if that was deliberate on their behalf
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
luke - maintainer of #atlantis is joining HashiCorp to work fulltime on the project (announced at HashiConf)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
btw! if any of you are at hashiconf, reach out here! I know @antonbabenko is there
antonbabenko
antonbabenkoabout 8 years ago
Yes, I am inside the keynote room now
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
Hey Anton, how many people are there compared to Amsterdam ?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago

Terraform collaboration for everyone.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)

State Storage, Locking, and History. No more state files! Automatically store and access state remotely whenever and wherever you run Terraform. State access is automatically locked during Terraform operations. In the UI, view a history of changes to the state, who made them, and when.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
sarkis
sarkisabout 8 years ago
I just discovered tfenv - any good/bad experiences with this here? Seems like it is going to be useful with the new 0.12.x version coming up and working with "legacy" terraform.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
isn’t that in ruby?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
hate to install that runtime just to switch envs in terraform
sarkis
sarkisabout 8 years ago
bash
sarkis
sarkisabout 8 years ago
sarkis
sarkisabout 8 years ago
I think this is less of an issue when doing this all The Right Way™ (using container via geodesic)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
haha
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
i think tfenv might be helpful initially
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
though i think there’s some way to specify the version compatibility
sarkis
sarkisabout 8 years ago
in fact things like this just reminds me I need to use geodesic everywhere 😛
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
to support hcl 1 and 2
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
i guess only one or the other will be supported
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
they recommend adding a version constraint on terraform
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
just released https://github.com/cloudposse/terraform-aws-iam-user to manage IAM user account for humans
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
it supports automatically associating the user with a list of groups as well as password generation using pgp+keybase
sarkis
sarkisabout 8 years ago
yea so the problem that led me to tfenv - was working with multiple repos/modules with different terraform version constraints
sarkis
sarkisabout 8 years ago
also - i was used to the rbenv style of .terraform-version in the root
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
yea
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
yea, given that - i think it’s probably the best alternative
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
i can see where in long-lived environments that managing the terraform versions will be essential and risk to keep everything up to date perhaps
jarv
jarvabout 8 years ago
any project open on the new hcl migration? can probably help out with that if there's a burn down list.
(haven't been following tf updates just know that was likely going to be released soonish)
jarv
jarvabout 8 years ago
or maybe just beta.. might try a couple just to dig into the new stuff. haven't been deep in tf dev in a bit
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@jarv that would be HUGE
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
i think it’s still too early, but when the time comes, I’ll create a “Terraform Module Upgrade” project and add everything there.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
will announce that as soon as it’s there.
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
I can also help out, but have a few modules of my own to do first.. but happy to join forces for problem solving ofc.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Thanks guys! we’re going to need the help
sarkis
sarkisabout 8 years ago(edited)
whoaaaa ... @Erik Osterman (Cloud Posse) you see Atlantis team is joining Hashicorp?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
yea! that’s both scary and exciting
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
sarkis
sarkisabout 8 years ago
ah thanks 😛
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
but yea, relevant here too
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
i want to use atlantis for more than terraform
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
but this doesn’t bode well for that roadmap
sarkis
sarkisabout 8 years ago
well now there's github actions 😄
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
yea
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
waiting on invite
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
anyone already have access to github actions?
sarkis
sarkisabout 8 years ago
not yet - waiting as well
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
GitHub Actions vs. HashiCorp/Atlantis 🙂
jarv
jarvabout 8 years ago(edited)
(possibly) Somewhat related:
Noticed systems manager added a wait for user input action. unfamiliar with ssm but was curious if something like terraform ci was one of the use cases for adding that
eh not sure that makes sense.. unsure why I was thinking about that now
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
Shall we create a 0.12 channel so problems & solutions don't get lost in other talks ?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
I think it’s a good suggestion
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Jeremy
Jeremyabout 8 years ago
Does anyone know of a way to provide multiple SSLCertificateArns for a beanstalk environment that is using an ALB? The name is called SSLCertificatesArns which implies that you can specify multiple arns but I have tried:
setting {
    namespace = "aws:elbv2:listener:443"
    name = "SSLCertificateArns"
    value = "<someArnForFirstCert>"
  }

  setting {
    namespace = "aws:elbv2:listener:443"
    name = "SSLCertificateArns"
    value = "<someArnForSecondCert>"
  }

And also tried:
setting {
    namespace = "aws:elbv2:listener:443"
    name = "SSLCertificateArns"
    value = "<someArnForFirstCert>,<someArnForSecondCert>"
  }

And:
setting {
    namespace = "aws:elbv2:listener:443"
    name = "SSLCertificateArns"
    value = ["<someArnForFirstCert>", "<someArnForSecondCert>"]
  }

Neither of these approaches seems to work correctly and cannot find any other documentation on it other than: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/command-options-general.html#command-options-general-elbv2-listener
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Sorry - not sure how to do it, but if you figure it out and need to make some changes - we accept nearly all PRs
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago(edited)
@Jeremy Looking at the Elastic Beanstalk Documentation it seems that Arns actually refer to just one ARN, maybe they wanted to be prepared future wise. Could you actually add multiple in the AWS Console ? As alternative you could create a new Certificate in ACM with support for multiple domain names, and have just one ARN.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
cool idea - too bad not maintained
praveen
praveenabout 8 years ago
Hi, I am looking for information on how to source passwords from azure keyvault using remote-exec(terraform). Basically I will have to copy a property file to the server which I will source it from git. I do not want the properties file to have sensitive information like secrets/ passwords. so I would want to append the file to password/secret's from keyvault in azure platform
praveen
praveenabout 8 years ago
or either using cloud-init

can I use variable which will query keyvault?
Mark Mutti
Mark Muttiabout 8 years ago
@praveen Not able to use the native resource (https://www.terraform.io/docs/providers/azurerm/r/key_vault.html)?
praveen
praveenabout 8 years ago
@Mark, yes I use key vault. But my question is if I can query keyvault by passing query keyvault variable in remote-exec or cloud-init
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
@praveen what about giving the server access to that keyvault, and retrieve those values at boot ?
praveen
praveenabout 8 years ago
how can I do that. can I have any example if already done. I mean if there is any ref to git code
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
praveen
praveenabout 8 years ago
I will try this and let you know the result
johncblandii
johncblandiiabout 8 years ago
Could I get some reviews on this PR? https://github.com/EasterSealsBayArea/terraform-aws-elastic-beanstalk-environment/pull/1

It is a fork not going to our master yet (using gitflow; will test/validate it internally from the develop branch with our projects then commit back). I didn’t want to go straight to the official yet without giving it some solid testing.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@johncblandii don't see anything controversial with the PR
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
to rebuild the readme run
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago

make init
make readme
johncblandii
johncblandiiabout 8 years ago
i did the readme part but not init. will do that now. thx
johncblandii
johncblandiiabout 8 years ago
oh, that’s the easier way to install everything. 🙂
johncblandii
johncblandiiabout 8 years ago
oh…maybe this is needed to be resolved for a full rebuild?


/bin/bash: gomplate: command not found
make: *** [readme/build] Error 127
johncblandii
johncblandiiabout 8 years ago
1 sec
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
oh, looks like we're missing a dep
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
you can run:
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago

make packages/install/gomplate
johncblandii
johncblandiiabout 8 years ago
ahh
johncblandii
johncblandiiabout 8 years ago
pushed
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
make readme/deps
johncblandii
johncblandiiabout 8 years ago
i mistook docs/terraform.md for the readme without even checking the filename
johncblandii
johncblandiiabout 8 years ago(edited)
good to know @Andriy Knysh (Cloud Posse)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
PR looks good to me 😃
johncblandii
johncblandiiabout 8 years ago
cool
johncblandii
johncblandiiabout 8 years ago
will get it tested on our stuff then PR it to your upstream
johncblandii
johncblandiiabout 8 years ago
a few other things will come soon too
johncblandii
johncblandiiabout 8 years ago
sidebar: this .12 stuff in TF will clean up the env vars (for/foreach loops) and settings (null values) tremendously
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Yea looks like it will be very cool
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
yea, we're anxiously awaiting #terraform-0_12
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@johncblandii if you want to help us update the beanstalk module (when the time comes), we'd be grateful
johncblandii
johncblandiiabout 8 years ago
absolutely, @Erik Osterman (Cloud Posse). going to do more updates to help out
johncblandii
johncblandiiabout 8 years ago(edited)
we use it heavily (3 envs done and about 20 more; between all stages)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@johncblandii any reason you're not using ECS or Kubernetes?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
(we found beanstalk deployments to be more flakey in the long run, which is why we have moved to the other systems)
johncblandii
johncblandiiabout 8 years ago
moving there. devops is a small team here so it comes down to time and we have the beanstalk stuff down
johncblandii
johncblandiiabout 8 years ago
and yes…beanstalk deployments……(sigh)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
LGTM
dan
danabout 8 years ago
a bit confused about what to expect of Terraform when using terraform-aws-ecs-web-app. If a new task definition is created, should Terraform automatically redeploy the service with the new definition? I'm currently seeing the new definition show up, but just the old tasks remain running. I guess the question is, should aws ecs update-service be necessary if my terraform-aws-ecs-web-app is configured correctly?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
the terraform-aws-ecs-web-app module is designed to work with CodeBuild/CodePipeline to automatically deploy changes merged to master.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
this only works if you add a buildspec.yaml to your projects
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
so, to answer your quesiton, calling aws ecs update-service is not needed if configured correctly 😃
dan
danabout 8 years ago
awesome, thanks for the quick help!
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
hope it helps!
johncblandii
johncblandiiabout 8 years ago
How do you guys manage your multi-env settings? Currently we use terragrunt, but that’s going away.

I know we can do tfvar files, but is that the best way? I don’t want to end up doing tf plan -var-file=prod.tfvars and tf plan -var-file=prod.tfvars -var-file=uswest2.tfvars and so on and so forth to separate the vars for reuse across a lot of configs (4 app stages, multi-region, multi-account).

Thoughts?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
hey @johncblandii, maybe you already saw that, but here is what we do:
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Although there are many possible ways of doing that, we use containers + ENV vars pattern.
As you mentioned, template rendering is another pattern (as implemented in terragrunt).

We store the ENV vars in either AWS SSM (secrets) or in Dockerfiles (not secrets).

Here are more details:

1. We have a collection of reusable TF modules https://github.com/cloudposse/terraform-root-modules.
The modules have no identity, everything is configurable via ENV vars.
(In other words, they don’t care where they will be deployed and how).

2. We deploy each stage (root, prod, staging, dev, testing) in a separate AWS account for security and better management

3. For each AWS account/stage (root, prod, staging, dev, testing), we have a GitHub repo which is a container (for which we use geodesic https://github.com/cloudposse/geodesic):
https://github.com/cloudposse/root.cloudposse.co
https://github.com/cloudposse/prod.cloudposse.co
https://github.com/cloudposse/staging.cloudposse.co
https://github.com/cloudposse/dev.cloudposse.co
https://github.com/cloudposse/testing.cloudposse.co

Not secret ENV vars are defined in the Dockerfiles, e.g. https://github.com/cloudposse/prod.cloudposse.co/blob/master/Dockerfile#L17
In other words, the account containers have identity defined via the ENV vars.

4. https://github.com/cloudposse/terraform-root-modules is added to the containers https://github.com/cloudposse/prod.cloudposse.co/blob/master/Dockerfile#L36

5. Inside the containers, users assume IAM roles ro access the corresponding AWS account and then provision TF modules.

6. Inside the containers we use chamber (https://github.com/segmentio/chamber) to read secrets from SSM (per AWS account)

So when we run a container (e.g. prod), we already have all ENV vars setup, and we read all the secrets from the account SSM store.
An account/stage can be in any region (also specified via ENV var, e.g. https://github.com/cloudposse/prod.cloudposse.co/blob/master/Dockerfile#L14)

Take a look at our docs for more details:
https://docs.cloudposse.com/reference-architectures/
https://docs.cloudposse.com/reference-architectures/cold-start/
https://docs.cloudposse.com/reference-architectures/notes-on-multiple-aws-accounts/
https://docs.cloudposse.com/geodesic/
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Hi all.
I still trying to create/destroy beanstalk using
 source    = "git::<https://github.com/cloudposse/terraform-aws-elastic-beanstalk-environment.git?ref=tag/0.5.0>"

Option
force_destroy         = true

But getting this
1 error(s) occurred:

* module.dev_front_end.module.logs.aws_s3_bucket.default (destroy): 1 error(s) occurred:

* aws_s3_bucket.default: error deleting S3 Bucket (develop-dev-front-dev-vitalii-logs): BucketNotEmpty: The bucket you tried to delete is not empty. You must delete all versions in the bucket.
        status code: 409, request id: 01B995B9AA71BAC0, host id: KwxTu/DhRRig0CtYmdq0qokvgEgCGDeiUAUB2b4yQna9hmnzWieVdtcSi8aGzg6oF4mk5JRff2s=
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
What is wrong ?
George
Georgeabout 8 years ago
Was the bucket created with force destroy set to true?
George
Georgeabout 8 years ago
George
Georgeabout 8 years ago
(Also https://stackoverflow.com/questions/49611774/aws-s3-bucket-delete-issue ) last comment mentions lifecycle policy, maybe check that too
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
yea if the bucket was created without force_destroy = true and it was added later, it will not be force destroyed
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
try to apply again and then destroy
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
or destroy manually and then apply with force destroy
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
but looks like the issue is still not solved or just does not work in some cases https://github.com/terraform-providers/terraform-provider-aws/issues/208
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Yes bucket was created with force_destroy
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Can I prevent creating bucket for logs ?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
not in the current version. If you open a PR, we’ll review it 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
resource "aws_s3_bucket" "elb_logs" {
  count         = "${var.elb_logs_ebnabled == "true" ? 1: 0}"
  bucket        = "${module.label.id}-logs"
  acl           = "private"
  force_destroy = "${var.force_destroy}"
  policy        = "${data.aws_iam_policy_document.elb_logs.json}"
}
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
johncblandii
johncblandiiabout 8 years ago
I’m fighting the beanstalk env with a silly error about the label. I passed in tags, didn’t pass tags, etc and it keeps giving me fits.


module.sc-api-env-active.module.elastic_beanstalk_environment.module.label.data.null_data_source.tags_as_list_of_maps: data.null_data_source.tags_as_list_of_maps: value of 'count' cannot be computed


Version:

  source = "git@github.com:cloudposse/terraform-aws-elastic-beanstalk-environment.git?ref=0.5.0"
johncblandii
johncblandiiabout 8 years ago
any thoughts?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
johncblandii
johncblandiiabout 8 years ago
yeah
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
the latest versions of null-label added a lot of stuff, but it breaks in complex configurations
johncblandii
johncblandiiabout 8 years ago
i saw the new stuff in there
johncblandii
johncblandiiabout 8 years ago
it got complex pretty quickly 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
so we need to change it to ref=tags/0.3.3 which does not have all that stuff and was working
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
want to open a PR? 🙂
johncblandii
johncblandiiabout 8 years ago
sure. which do you prefer? moving the version back or changing the source?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
we’ll fix the null-label
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
(simplify it)
johncblandii
johncblandiiabout 8 years ago
cool. PR incoming in a sec
johncblandii
johncblandiiabout 8 years ago
testing
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
thanks
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
tested? 🙂
johncblandii
johncblandiiabout 8 years ago
about to test on my module
johncblandii
johncblandiiabout 8 years ago
i tested in the example
johncblandii
johncblandiiabout 8 years ago
Switched to source = "git@github.com:eastersealsbayarea/terraform-aws-elastic-beanstalk-environment.git?ref=53c5aa8"

Plan: 48 to add, 0 to change, 0 to destroy.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
ok thanks, will merge
johncblandii
johncblandiiabout 8 years ago
coolio
johncblandii
johncblandiiabout 8 years ago
sweet. thx
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
johncblandii
johncblandiiabout 8 years ago
have you guys seen an attribute listed in docs but error as an unavailable attribute?
johncblandii
johncblandiiabout 8 years ago
docs clearly say description is an attribute, but it isn’t working when output: https://www.terraform.io/docs/providers/aws/r/elastic_beanstalk_environment.html#description-1
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
hmm, we saw something like that for other modules, but not in EB
johncblandii
johncblandiiabout 8 years ago
testing stuff locally
johncblandii
johncblandiiabout 8 years ago
could be some cache issue. it works when i reference the local module
johncblandii
johncblandiiabout 8 years ago
removed it
johncblandii
johncblandiiabout 8 years ago
pushed a documentation fix
johncblandii
johncblandiiabout 8 years ago
test results added
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
thanks, will review
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
@johncblandii the PR looks good, thanks, just one comment
johncblandii
johncblandiiabout 8 years ago
cool. checking
johncblandii
johncblandiiabout 8 years ago
pushed
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
terraform fmt
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
please run
johncblandii
johncblandiiabout 8 years ago
ugh…meant to do that
johncblandii
johncblandiiabout 8 years ago
i turned off auto-format in VS
johncblandii
johncblandiiabout 8 years ago
it made me lazy to formatting. 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
thanks
sarkis
sarkisabout 8 years ago(edited)
If any gophers or terraform provider authors are around and have some free time - could use a review: https://github.com/terraform-providers/terraform-provider-pagerduty/pull/99
sarkis
sarkisabout 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Hi again. I'm still fighting with getting beanstalk and rds work together. Need some help here please:

module "elastic_beanstalk_environment" {
  source    = "git::<https://github.com/cloudposse/terraform-aws-elastic-beanstalk-environment.git?ref=tag/0.6.2>"
  namespace = "${var.namespace}"
  stage     = "${var.environment}"
  name      = "${var.user_account_name}"
  zone_id   = "${var.parent_zone_id}"
  app       = "${module.elastic_beanstalk_application.app_name}"


  # associate_public_ip_address = false
  
  instance_type           = "${var.instance_type}"
  autoscale_min           = 1
  autoscale_max           = 4
  updating_min_in_service = 0
  updating_max_batch      = 1
  autoscale_lower_bound   = 20 
  autoscale_upper_bound   = 80 

  updating_max_batch = 1
  updating_min_in_service = 1
  wait_for_ready_timeout = "20m"


 loadbalancer_type      = "application"
  vpc_id                = "${module.vpc.vpc_id}"
  public_subnets        = "${module.subnets.public_subnet_ids}"
  private_subnets       = "${module.subnets.private_subnet_ids}"
  security_groups       = ["${module.vpc.vpc_default_security_group_id}"]
  solution_stack_name   = "64bit Amazon Linux 2018.03 v4.5.3 running Node.js"
  tier                  = "WebServer"
  force_destroy         = true
  keypair               = "${aws_key_pair.dev_ssh_key.key_name}"
  ssh_listener_enabled  = true 
  ssh_listener_port     = "22"
  ssh_source_restriction = "0.0.0.0/0"

  http_listener_enabled = true  ## Enable port 80 (http)
  # instance_refresh_enabled = true ## Enable weekly instance replacement.
  update_level = "minor" ## The highest level of update to apply with managed platform updates
   
  preferred_start_time = "Sun:10:00"
  rolling_update_type = "Health"
  root_volume_size = "10"
  root_volume_type = "gp2"

  setting {
    namespace = "aws:elasticbeanstalk:application:environment"
    name = "RDS_USERNAME"
    value = "${rds_instance.default.database_user}"
  }
  setting {
    namespace = "aws:elasticbeanstalk:application:environment"
    name = "RDS_PASSWORD"
    value = "${rds_instance.default.database_password}"
  }
  setting {
    namespace = "aws:elasticbeanstalk:application:environment"
    name = "RDS_DATABASE"
    value = "${rds_instance.default.name}"
  }
  setting {
    namespace = "aws:elasticbeanstalk:application:environment"
    name = "RDS_HOSTNAME"
    value = "${rds_instance.default.instance_endpoint}"
  }
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
and RDS

module "rds_instance" {
      source                      = "git::<https://github.com/cloudposse/terraform-aws-rds.git?ref=tag/0.4.1>"
      namespace                   = "${var.namespace}"
      stage                       = "${var.environment}"
      name                        = "${var.user_account_name}-db"
      dns_zone_id                 = "${var.parent_zone_id}"
      host_name                   = "db"
      dns_zone_id                 = "${var.parent_zone_id}"
      security_group_ids          = ["${module.vpc.vpc_default_security_group_id}"]
      database_name               = "app_db"
      database_user               = "dbuser"
      database_password           = "password"
      database_port               = 5432
      multi_az                    = "false"
      storage_type                = "gp2"
      allocated_storage           = "5"
      storage_encrypted           = "false"
      engine                      = "postgres"
      engine_version              = "9.6.6"
      instance_class              = "db.t2.micro"
      db_parameter_group          = "postgres9.6"
      #parameter_group_name        = "mysql-5-7"
      publicly_accessible         = "false"
      subnet_ids                  = ["${module.subnets.public_subnet_ids}"]
      vpc_id                      = "${module.vpc.vpc_id}"
      auto_minor_version_upgrade  = "true"
      allow_major_version_upgrade = "false"
      apply_immediately           = "false"
      maintenance_window          = "Mon:03:00-Mon:04:00"
      skip_final_snapshot         = "true"
      copy_tags_to_snapshot       = "true"
      backup_retention_period     = 7
      backup_window               = "22:00-03:00"
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Getting
Error: module 'elastic_beanstalk_environment': unknown resource 'rds_instance.default' referenced in variable rds_instance.default.database_user
Error: module 'elastic_beanstalk_environment': unknown resource 'rds_instance.default' referenced in variable rds_instance.default.database_password
Error: module 'elastic_beanstalk_environment': unknown resource 'rds_instance.default' referenced in variable rds_instance.default.instance_endpoint
Error: module 'elastic_beanstalk_environment': unknown resource 'rds_instance.default' referenced in variable rds_instance.default.name
Error: module "elastic_beanstalk_environment": "setting" is not a valid argument
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
what I'm doing wrong
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
hey @Vitalii Morvaniuk
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
so let’s see here
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
I've found that I missed value = "${module.rds_instance.database_user}"
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
first, fix the errors in referencing the module value = "${module.rds_instance.default.instance_endpoint}"
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
but now I'm getting only Error: module "elastic_beanstalk_environment": "setting" is not a valid argument
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
(add module... in front)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
ok, to provide ENV vars to the elastic beanstalk module, use it like this https://github.com/cloudposse/terraform-aws-jenkins/blob/master/main.tf#L49
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Let me try this
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
(setting is not a valid argument b/c it’s not exposed as var in the current release of the module. We have some PRs and issues opened to do it, we’ll look into that since it would fix some other issues)
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago(edited)
added

  env_vars = "${
    merge(
      map("RDS_HOSTNAME","${module.rds_instance.instance_endpoint}",
          "RDS_USERNAME","${module.rds_instance.database_user}",
          "RDS_PASSWORD","${module.rds_instance.database_password}",
          "RDS_DATABASE","${module.rds_instance.name}"
          ), env_vars
      )
    }"
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Now getting
Error: module 'elastic_beanstalk_environment': "database_user" is not a valid output for module "rds_instance"
Error: module 'elastic_beanstalk_environment': "name" is not a valid output for module "rds_instance"
Error: module 'elastic_beanstalk_environment': "database_password" is not a valid output for module "rds_instance
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Have separate
env_vars
defined previously to set some app staff
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago

env_vars = "${
      map(
        "environment", "${var.environment}",
        "namespace", "${var.namespace}",
        "user", "${var.user_account_name}",
        "API_HOST", "${var.api_host}",
 ...
      )
   }"
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
HM, than how could I get all staff to connect my app to RDS DB ?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
database_user and database_password you already know when providing them here https://github.com/cloudposse/terraform-aws-rds/blob/master/variables.tf#L44`
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
so you need user and password (which you know and you provide them to the RDS module and to the EB module as ENV vars)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
My fault
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Thanks for help - all clear now
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
nice
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
then in you app (e.g. NodeJs), you use those ENV vars, like this:
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
function getDbSettings() {
  return (
    {
      host: process.env.DB_HOST,
      database: process.env.DB_NAME,
      user: process.env.DB_USER,
      password: process.env.DB_PASSWORD
    });
}
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
so you need 4 ENV vars
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
env_vars = "${
      map(
        "DB_HOST", ""${module.rds_instance.instance_endpoint}",
        "DB_NAME", "xxxxxxxx",
        "DB_USER", "xxxxxxxxxx",
        "DB_PASSWORD", "xxxxxxxxx",
      )
   }"
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Quick question
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago(edited)
could password be autogenerated ?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
it could be, it’s outside the module anyway
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
and you can use any script to autogenerate it, even TF like this https://www.terraform.io/docs/providers/random/index.html
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
How could I incert it in DB_PASSWORD ?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Thanks
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
resource "random_string" "password" {
  length = 16
  special = true
  override_special = "/@\" "
}

module "rds_instance" {
      source                      = "git::<https://github.com/cloudposse/terraform-aws-rds.git?ref=tag/0.4.1>"
      namespace                   = "${var.namespace}"
      stage                       = "${var.environment}"
      name                        = "${var.user_account_name}-db"
      dns_zone_id                 = "${var.parent_zone_id}"
      host_name                   = "db"
      dns_zone_id                 = "${var.parent_zone_id}"
      security_group_ids          = ["${module.vpc.vpc_default_security_group_id}"]
      database_name               = "app_db"
      database_user               = "dbuser"
      database_password           = "${random_string.password.result}"
 ....
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
And than use the same
"DB_PASSWORD", "${random_string.password.result}"
in setting envs ?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Yes
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Thanks for help. Will try to contribute to code next week
ALI
ALIabout 8 years ago
hi I am a newbie on terraform, and got a good amount of AWS experience.. I want to know how to start scripting in terraform
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Welcome @ALI
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Probably take a look at some modules
ALI
ALIabout 8 years ago
thanks @Andriy Knysh (Cloud Posse)
ALI
ALIabout 8 years ago
yea I am looking at some of the modules .
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
What AWS resources do you want to script?
ALI
ALIabout 8 years ago
I want to script on VPC, EC2 ,Dynamo DB
ALI
ALIabout 8 years ago
for now
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
We have that :)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Give me a few minutes
ALI
ALIabout 8 years ago
sure!
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
@ALI take a look at these modules
Dan
Danabout 8 years ago
At first it might be daunting but itll get easier
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
let us know if any questions
ALI
ALIabout 8 years ago
thanks @Andriy Knysh (Cloud Posse)
ALI
ALIabout 8 years ago
haha I will take that @Dan
Dan
Danabout 8 years ago
😁
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@ALI you've signed up for a master class in terraform 😃
ALI
ALIabout 8 years ago
I bet @Erik Osterman (Cloud Posse), @Andriy Knysh (Cloud Posse) I will if I am stuck anywhere ..
Dombo
Domboabout 8 years ago
How do we all handle modelling the infrastructure within accounts in multi-account architectures? Do you have a declarative file that lists all of it? Some of it? Or do you bundle it all per project and just rely on the knowledge of the team to know what's live?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@Dombo are you referring to our reference architectures or asking generally?
Dombo
Domboabout 8 years ago
Generally - but also reference arch if you want to talk in terms of that
Dombo
Domboabout 8 years ago
I'm interested in how others handle this - I don't see many mature IaC code bases
Dombo
Domboabout 8 years ago
There are some things that are persistent and not tied to apps like management servers, bastions, VPCs, sec groups, IAM roles/policies/users
Dombo
Domboabout 8 years ago
These are genreally per account
Dombo
Domboabout 8 years ago
Then there are application specfiic deployment dependencies
Dombo
Domboabout 8 years ago
I wonder if you treat them the same or differently?
Dombo
Domboabout 8 years ago
Common modules folder + declarative master stack/.tf file per account + .tf file per project? Where do you track state in both cases? Other ways of tackling this?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
So we publish all of our reference architectures here: https://cpco.io/reference-architectures
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
high-level, we have one repo (terraform-root-modules) which contain the “root level” terraform module invocations
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
then we have one repo per AWS account. This allows us to easily keep stages separately, but also reuse code between stages in the form of modules.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
(@Andriy Knysh (Cloud Posse) re-share that thread) 😃
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Andriy explained it pretty well in this thread: https://sweetops.slack.com/archives/CB6GHNLG0/p1540514525000100
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
thanks @Erik Osterman (Cloud Posse) 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
so yea, the main idea is when a user logs into an account from a geodesic module (let’s say staging), they can’t see and do any damage (even accidentally) to other accounts (root, prod, etc.) - completely separated
Dombo
Domboabout 8 years ago
Of course - I also practice account separation iwth about 15-20 accounts implemented in my org.

Some interesting patterns described in that thread. What do you do about provisioning IAM users/tfstate backing resources?
Dombo
Domboabout 8 years ago
Bootstrap 0 if you will
Dombo
Domboabout 8 years ago
If this is documented somewhere feel free to point me there
Dombo
Domboabout 8 years ago
I know you guys aren't just here to answer q;s
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
we’re not optimizing for the coldstart right now - as in one command to spin up 20 accounts
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
namely the problem is an order of operations that needs to be performed that terraform is not well suited for, especially if there are any cycles in the graph
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
some of it could be optimized using terragrunt, however, customers seldom if ever start over from scratch again from bootstrap 0
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
we also provision one state bucket per account, rather than a centralized state bucket
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
this is just sticking with our “share nothing” approach, which also convolutes the process of bootstrapping
Dombo
Domboabout 8 years ago
Interesting choice regarding one bucket/dynamo table per account - is there a reason why?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
share nothing
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
so for example, if you share the state bucket, there’s no way to stage changes to the statebucket without if affecting all stages all at once
Dombo
Domboabout 8 years ago
Hmmm good point
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
but by allowing each stage/account to have their own bucket, they are entirely standalone
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
yea, and we do provision accounts and add users to roles
Dombo
Domboabout 8 years ago
Powerful way of doing it
Dombo
Domboabout 8 years ago
Impressed - kudos
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
problem with it is terraform seems to be nudging people to a single shared state bucket
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
and using workspaces
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
we don’t use terraform workspaces
Dombo
Domboabout 8 years ago
Yeah neither do I - not a fan for reasons
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
terraform is also coming out with “managed state as a service” free for all
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
curious to see how that works and fits in
Dombo
Domboabout 8 years ago
Yeah that's off the back of the atlantis acquisition I presume
Dombo
Domboabout 8 years ago
'acqusition'
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
😃
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
i don’t think it’s related persay, but to your point atlantis is big on workspaces
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
and aligned therefore with their trajectory.
Dombo
Domboabout 8 years ago
Doesn't have to be though - pretty sure you can define the lifecycle as you wish with custom workflows and such
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
yea
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
we have it working without it
Dombo
Domboabout 8 years ago
At which point it could compliment your guys system quite nicely
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
are you using terragrunt?
Dombo
Domboabout 8 years ago
Collaborative plan & release to certain stages
Dombo
Domboabout 8 years ago
terragrunt at work
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
cool
Dombo
Domboabout 8 years ago
atlantis + similar system to yours when consulting in Aus
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
have you seen our fork of atlantis?
Dombo
Domboabout 8 years ago
Yeah I did
Dombo
Domboabout 8 years ago
Just so you could add your own name?
Dombo
Domboabout 8 years ago
hahaha
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
haha
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
actually, have zero interest in maintaining it
Dombo
Domboabout 8 years ago
I wish I could hard fork alexa & google assistant for the same reason
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
just it’s very hard to get features accepted into atlantis right now
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
but luke hears our requests loud and clear and is working to incorporate them
Dombo
Domboabout 8 years ago
Yeah I guess that's them signalling the future of the project?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
yea
Dombo
Domboabout 8 years ago
My bet is EOL and roll internal to Hashi
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
heh 😃
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
are you excited about GitHub actions?
Dombo
Domboabout 8 years ago
Reasonably - lots of my customers are on Bitbucket/Gitlab/Self hosted stuff
Dombo
Domboabout 8 years ago
I'm curious to see how it plays out in the community
Dombo
Domboabout 8 years ago
Actions are a direct shot at the best monetised section of githubs partner ecosystem
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
yes, to a degree
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
or it could also be seen as a way of allowing them to have tighter integration with GitHub
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
i see actions evolving into something like Salesforce for GitHub
Dombo
Domboabout 8 years ago
Yeah I'd be interested to see how that integration goes down
Dombo
Domboabout 8 years ago
Good to meet some other people pushing the limits of the modern IAC toolchain
Dombo
Domboabout 8 years ago
Even if you are on the other side of the world 😉
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
thanks!! you’re among friends
Dombo
Domboabout 8 years ago
Anyway it's the middle of the day over here
Dombo
Domboabout 8 years ago
Gotta get back to work
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
ttyl
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago(edited)
@Dombo I forgot where you are located? 🙂
Dombo
Domboabout 8 years ago
AEST - East Coast Australia
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Hi, if using terraform-aws-dynamic-subnets what Actions should I allow in policy to make it work? Creating separate user for terraform now and trying to limit access
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Now getting
module.subnets.aws_eip.default[0]: aws_eip.default.0: UnauthorizedOperation: You are not authorized to perform this operation.
on apply
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
@Vitalii Morvaniuk The same logic to Terraform applies to general AWS usage. Limiting the Terraform user is generally not making things easier.
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
@Maarten van der Hoef is there any kind of recommendations to read? Idea is to give developers terraform files so they could spin up envs for usage - so I don't want them to create or destroy something with admin access
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
sure, one moment
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
Most companies do this adding AWS accounts for different purposes like testing.
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Thanks a lot.
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
Sure man, good luck
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
After reading those article I see that I really need it...
shaiss
shaissabout 8 years ago
this isn’t sweetops specific, but hoping someone here can give me some insight. I’m using resource “aws_iam_user_policy” and getting a limit error “Error putting IAM user policy CloudCheckr_RO: LimitExceeded: Maximum policy size of 2048" when running terraform apply. But I can create this policy just fine in the console
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
hey @shaiss
shaiss
shaissabout 8 years ago
Hi @Andriy Knysh (Cloud Posse) looks like it’s an AWS limit https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_iam-limits.html. odd that it works in the console though
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
you create exactly the same policy in the console?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
it’s better not to attach policies to individual users
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
use groups instead
shaiss
shaissabout 8 years ago
yes, I’m aware, this a requirment of a customer
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
the limit for a group is higher
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
Or create a customer managed policy and attach that one.
shaiss
shaissabout 8 years ago
yes, that will be my suggestion to them again, but they claim they don’t need to b/c they can create that policy in the console
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
yea, we can create everything in the console 🙂
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
Maybe Terraform has pre-flight checks which aren't valid, or calculates differently from AWS.
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
Like AWS does not count spaces of the document and Terraform does.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago

IAM does not count white space when calculating the size of a policy against these limitations
shaiss
shaissabout 8 years ago
seems like the policy is 6850 bytes, or 4230 if ignore whitespaces
shaiss
shaissabout 8 years ago
so either way, it should technically fail since it’s over the 2048 limit
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
Where did you read that ?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
so maybe you could break it into a few aws_iam_user_policy(s) @shaiss
shaiss
shaissabout 8 years ago
@Andriy Knysh (Cloud Posse) yeah, that’s a good option!
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
or
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago(edited)
take this as input for your iam_user_policy
"${replace(data.template_file.init.rendered,"/\\s/","" )}"
shaiss
shaissabout 8 years ago
@Maarten van der Hoef sorry, not sure I’m following. atm, I’m going to try creating the policy as an IAM policy vs the iam_user_policy, then attaching it to the user, we’ll see if that works
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
sure, what ever works for you.
shaiss
shaissabout 8 years ago
ha, that worked
shaiss
shaissabout 8 years ago
I agree not the best way to do it, but it worked, and it’ll have to do for now
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
what did you try now ?
shaiss
shaissabout 8 years ago
@Maarten van der Hoef I created the user, created the iam policy, then attached the iam policy to the user. b/c it’s a generic iam policy and not a user policy, that limit doesn’t imply
shaiss
shaissabout 8 years ago
*apply
shaiss
shaissabout 8 years ago
Anyone here using terraform w/ a marketplace ami? Seems like you have to use the we console to subscribe to the marketplace item first b4 you can call it from TF
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Yes that’s correct. We use terraform with pfSense firewall Ami from market place
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
I believe we first had to activate the subscription
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
We didn’t attempt to automate market place subscriptions. This was more than a year ago. Not sure if it is possible.
i5okie
i5okieabout 8 years ago
hi, any plans to add support for ssl negotiating policy to the terraform-aws-elastic-beanstalk-environment module?
(https://www.terraform.io/docs/providers/aws/r/lb_ssl_negotiation_policy.html)
i5okie
i5okieabout 8 years ago
or to specify an existing aws policy?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
@i5okie if you open a PR, we’ll promptly review 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
thanks
i5okie
i5okieabout 8 years ago
alrighty. was just wondering. thanks
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
thank you for pointing that out, nice addition to the module
Kenny Inggs
Kenny Inggsabout 8 years ago
I have a question related to https://github.com/cloudposse/terraform-aws-route53-cluster-zone. How do you deal with the "production" stage + zones? Typically, I'd have a zone for "dev" (e.g. dev.example.com, containing e.g. api.dev.example.com), and have the prod zone on the apex (example.com containing e.g. api.example.com). I'm wondering whether you guys do something like creating a prod.example.com zone, with Alias in the parent / apex zone? Or how else do you deal with the prod stage to apex mapping?
Kenny Inggs
Kenny Inggsabout 8 years ago(edited)
(this is particularly relevant on public, customer facing URLs in the prod zone. where you don't want them to see prod. in every URL. Like websites or public API endpoints)
Kenny Inggs
Kenny Inggsabout 8 years ago
p.s. I also typically have a "global" stage or something similar that will have my MX, SPF and DKIM records.
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Hi all
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Having problems on terraform init
Error downloading modules: Error loading modules: error downloading '<https://github.com/cloudposse/terraform-aws-s3-website.git?ref=tag/0.5.3>': /usr/bin/git exited with 1: error: pathspec 'tag/0.5.3' did not match any file(s) known to git.
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
How to correctly set revision ?
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago(edited)
source              = "git::<https://github.com/cloudposse/terraform-aws-dynamic-subnets.git?ref=tag/0.3.7>"
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago(edited)
or
 source = "<http://github.com/cloudposse/terraform-aws-dynamic-subnets/releases/tag/0.3.7|github.com/cloudposse/terraform-aws-dynamic-subnets/releases/tag/0.3.7>"
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
@Kenny Inggs it’s depends on a use case and a customer. We usually have two cases here:
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
1. Use subdomains for stages (prod.example.net, staging.example.net), and use an alias or CNAME from the public domain (e.g. example.com) to the prod stage (CNAME prod.example.net for example.com)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago(edited)
2. The same as #1, but using diff TLDs for stages, e.g. <http://example.net|example.net> for prod and <http://example.qa|example.qa> for staging. Then CNAME for <http://example.com|example.com> pointing to <http://example.net|example.net>
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago(edited)
all MX, SPF and DKIM records are in the global/public domain
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Kenny Inggs
Kenny Inggsabout 8 years ago
Awesome. Thanks @Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
np
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
@Vitalii Morvaniuk are you missing git:: in front of <https://github.com/cloudposse/terraform-aws-s3-website?ref=tag/0.5.3>?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
if not, maybe some DNS or caching issues on you computer
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
@Andriy Knysh (Cloud Posse) according to https://www.terraform.io/docs/modules/sources.html#github for github git:: could be missed
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
about DNS or caching - we tried this on two PCs in Ukraine and in Germany
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
can you try adding git:: and test again?
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
It was there from beginning
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
removed .terraform and tried again
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
get this
Andy
Andyabout 8 years ago(edited)
try tags/0.5.3 with an S not just tag?
Andy
Andyabout 8 years ago(edited)
<https://github.com/cloudposse/terraform-aws-s3-website?ref=tags/0.5.3>
Andy
Andyabout 8 years ago
yup @Vitalii Morvaniuk I just tried without the s and received the same error
Vitalii Morvaniuk
Vitalii Morvaniukabout 8 years ago
Nice catch! @Andy you saved me!
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
ah yea 🙂 no s
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
thanks @Andy
nukepuppy
nukepuppyabout 8 years ago
just toying with you guys terraform for creating VPCs.. any suggestions on where to start?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
hey @nukepuppy
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
nukepuppy
nukepuppyabout 8 years ago
i been messing with dynamic subnets.. its neat.. though it insists on cutting up a VPC in an odd way 😜
nukepuppy
nukepuppyabout 8 years ago
like a /24 vpc into 4 /28s instead of 4 /26s when giving 2 azs priv/pub was odd.. was trying to see how to force that if even possible
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
there are many ways of cutting up a VPC 🙂
nukepuppy
nukepuppyabout 8 years ago
im getting the gist of it.. really cool collection of stuff you guys going
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
we have a few diff modules for that
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
nukepuppy
nukepuppyabout 8 years ago
i love the uber high re-use of these
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
nukepuppy
nukepuppyabout 8 years ago
so i just want my /24 into 2 AZs priv/pub as /26s what would be your recommendation on which one to try out?
nukepuppy
nukepuppyabout 8 years ago
actually.. terraform-aws-multi-az-subnets is one im trying now 😜
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
you can specify any number of subnets you need
nukepuppy
nukepuppyabout 8 years ago
ooh so i assume if the value isnt set.. it uses AZ count as the subnet count maybe?
nukepuppy
nukepuppyabout 8 years ago
i didnt look that deep into it yet ..
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
yes, if not set, it uses all AZs
nukepuppy
nukepuppyabout 8 years ago
ah i had done this availability_zones = ["us-east-1a", "us-east-1c"]
nukepuppy
nukepuppyabout 8 years ago
and it did make 4 subnets.. but still cut up into /28s
nukepuppy
nukepuppyabout 8 years ago
making me only have 10ips in each hehe
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
try to set max_subnet_count
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
it should divide correctly
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
or maybe this will be better for your use-case https://github.com/cloudposse/terraform-aws-multi-az-subnets#usage
nukepuppy
nukepuppyabout 8 years ago
hm still did /28s
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
since you provide CIDR blocks
nukepuppy
nukepuppyabout 8 years ago
is what im using as an example.. and modifying appropriately
nukepuppy
nukepuppyabout 8 years ago
im using the usage.. modified the AZ lists per pub/priv to just 2 AZs set.. az nat gw to 2
nukepuppy
nukepuppyabout 8 years ago
setup variables to be some of the usual stuff /namespace/app etc
nukepuppy
nukepuppyabout 8 years ago
cidr block set to a /24 but still get a /28 cut up
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
nukepuppy
nukepuppyabout 8 years ago
private_az_subnet_ids = {
  us-east-1a = subnet-xxxxx
  us-east-1c = subnet-xxxxx
}
public_az_subnet_ids = {
  us-east-1a = subnet-xxxxx
  us-east-1c = subnet-xxxxx
}
nukepuppy
nukepuppyabout 8 years ago
let me have a look
nukepuppy
nukepuppyabout 8 years ago
oh hmm different format of max_subnets o.O
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
if you need any changes to the subnet modules to accommodate your requirements, please open a PR
nukepuppy
nukepuppyabout 8 years ago
oh hmm.. still cut into /28s despite having that var set
nukepuppy
nukepuppyabout 8 years ago
yeah ill have a look..
nukepuppy
nukepuppyabout 8 years ago
i mean terraform vpc there is bazillion templates
nukepuppy
nukepuppyabout 8 years ago
i just liked the idea of minimal go for re-use
nukepuppy
nukepuppyabout 8 years ago
its probably very useful for most normal larger VPCs but smaller ones may just be a bit too much?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
yea it’s not easy to come up with a universal module to create subnets, too many possible ways of doing it
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
for our usage, the three modules were enough
nukepuppy
nukepuppyabout 8 years ago
yeah its doing 99% of everything id want it to do
nukepuppy
nukepuppyabout 8 years ago
except.. cut up the /24 into 4 /26s instead of /28s
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
the calculations are here
nukepuppy
nukepuppyabout 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
(been long time since we reviewed that)
nukepuppy
nukepuppyabout 8 years ago
thanks again @Andriy Knysh (Cloud Posse)... ill re-evaluate some other time.. i have this working in terraform i built by hand.. just wanted to give something shinier a whirl 😜
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
@nukepuppy outsider question, why do you want to have something smaller than a /24 when you have a /8 at disposal ?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
yea that’s why we did not pay much attention to how /8 was divided 🙂
nukepuppy
nukepuppyabout 8 years ago
@Maarten van der Hoef because when you own IP space that must be routable between things and use IPAM at enterprise level
nukepuppy
nukepuppyabout 8 years ago
smaller VPCs are created for specific purposes..
nukepuppy
nukepuppyabout 8 years ago
and multi- account (aws) strategies become a bigger thing
nukepuppy
nukepuppyabout 8 years ago
no one should or would cut up a /8 into 1 VPC in aws i hope and run a company out of it
nukepuppy
nukepuppyabout 8 years ago
use about 20-30 aws accounts for stag and 25ish for prod
nukepuppy
nukepuppyabout 8 years ago
and all have different use cases / purposes
nukepuppy
nukepuppyabout 8 years ago
and different peering requirements
nukepuppy
nukepuppyabout 8 years ago
so.. its always viable
nukepuppy
nukepuppyabout 8 years ago
but a /24 for an individual person.. to use in multiple subnets in a VPC
nukepuppy
nukepuppyabout 8 years ago
seems pretty much a normal use case to me
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
Wow, those are a lot of accounts per stack. Using /16 per vpc/account ourselves, I guess you have a lot of different teams/apps then
nukepuppy
nukepuppyabout 8 years ago
yes a lot... quite a lot 😞
nukepuppy
nukepuppyabout 8 years ago
but in general wanted to just "can" as in soup.. the process of getting smaller stuff without everyone re-writing stuff ya know
nukepuppy
nukepuppyabout 8 years ago
all good .. very cool to see
nukepuppy
nukepuppyabout 8 years ago
i was just building a small VPC for a specific reason and had a /24 available to toy with
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
Thanks for explaining the use-case. I know someone who works at a company which makes route planning sw for cars .. the sheer size of different aws/azure accounts is just mindblowing.
nukepuppy
nukepuppyabout 8 years ago
oh right thats other thing the /8 has multi cloud uses etc etc.. so getting a small cut etc is usually something like a /20 for a team until its used up
nukepuppy
nukepuppyabout 8 years ago
and from that /20 you cut up what you can
nukepuppy
nukepuppyabout 8 years ago
but even then a class C VPC can host a bunch of things ya know
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
True, but on the other hand, if you divide a /8 in /16's you have 255 VPC's. If the company can't fit inside 255 VPCs it's maybe time to do things differently
nukepuppy
nukepuppyabout 8 years ago
well there is data centers using up a ton of the space too
nukepuppy
nukepuppyabout 8 years ago
all good though.. everyone's got different use cases..
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
yep
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
@nukepuppy we can create another subnets module specifically for that use-case. want to try together? 🙂
nukepuppy
nukepuppyabout 8 years ago
@Andriy Knysh (Cloud Posse) for sure ! in a few days.. i still gotta get something finished up here.. gonna roll my manual made one for now and wrap that up
nukepuppy
nukepuppyabout 8 years ago
then id love to re-visit as i feel everytime someone needs to make a vpc here.. they basically re-invent the wheel
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
last cents @nukepuppy 🙂 A /21 for a VPC allows 6 /24's public+private subnets in 3 az's which is also quite economical, and does allow for enough growth within a VPC. And 8K of VPC's 🙂
nukepuppy
nukepuppyabout 8 years ago
i cut up a /20 into 7 /23s and 2 /24s for smaller things
nukepuppy
nukepuppyabout 8 years ago
the /23s usually used for rebuilding kops/eks clusters to test out
nukepuppy
nukepuppyabout 8 years ago
and the small /24s for things like permanent smaller infra / smaller management vpcs etc
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
Sounds like you have a huge platform, how much is terraformed ?
nukepuppy
nukepuppyabout 8 years ago
depends on diff teams and use cases
nukepuppy
nukepuppyabout 8 years ago
most is hybrid stuff.. some are heavily orchestrated etc... so it isnt all one stop shop for things
johncblandii
johncblandiiabout 8 years ago