#terraform - March 2019

stobiewankenobi
stobiewankenobiover 7 years ago
hooray
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
we don’t use it as a wrapper
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
though it does support being called as a wrapper
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
we use it with direnv which has a stdlib
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
we have one repo per stage.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
nope
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
terraform-root-modules is our service catalog
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
(do read the README.md on that repo though b/c it’s often misunderstood)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
So we can call a module with very few lines of code
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago

# Import the remote module
export TF_CLI_INIT_FROM_MODULE="git::<https://github.com/cloudposse/terraform-root-modules.git//aws/acm?ref=tags/0.35.1>"
export TF_CLI_PLAN_PARALLELISM=2

use terraform
use atlantis
use tfenv
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
So when calling terraform init, it will download the remote module and initialize it
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
using tfenv, we pass all parameters via the TF_CLI_ARGS_init environment variable
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
this allows us to call all terraform commands without a wrapper
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
however, all this stuff is a PIA to implement if you don’t use a preconfigured shell
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
that’s why we have geodesic
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
it’s 1000 things
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
there’s no one workflow
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
yup, we need to add more docs.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
it’s far from complete.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
😃
erik
erikover 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
note it calls out OSX
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
we have hundreds of issues 😃 no shortage
Erik Weber
Erik Weberover 7 years ago
Does anyone know of a good way to check whether or not a lot of terraform configurations are up to date? Basically I'd like to run terraform plan on $x amount of services/environments and get some sort of alterting if the plan isn't empty
Nikola Velkovski
Nikola Velkovskiover 7 years ago
terraform plan -detailed-exitcode would do the trick
Nikola Velkovski
Nikola Velkovskiover 7 years ago
as for the process it depends on what you have
Nikola Velkovski
Nikola Velkovskiover 7 years ago
e.g. if you have jenkins 🙂
then you will do a bash scripts that runs in a CI pipeline 😄
Erik Weber
Erik Weberover 7 years ago(edited)
The plan was to use atlantis rather than pipelines for terraform, but I guess I could do both (where the pipeline only does a plan)
loren
lorenover 7 years ago
this is what i do, yes, terraform plan -detailed-exitcode
loren
lorenover 7 years ago
have scheduled jobs in CI with read-only privs that runs that every day for all tf configs
Erik Weber
Erik Weberover 7 years ago
Cheers 🙂
Dave Yu
Dave Yuover 7 years ago
how can I show the state of a single resource instance?
 $ terraform state list | grep aws_subnet.public
module.subnets.aws_subnet.public[0]
module.subnets.aws_subnet.public[1]
module.subnets.aws_subnet.public[2]
module.subnets.aws_subnet.public[3]
 
 $ terraform state show module.subnets.aws_subnet.public[1]
Multiple instances found for the given pattern!

This command requires that the pattern match exactly one instance
of a resource. To view the matched instances, use "terraform state list".
Please modify the pattern to match only a single instance.
Dave Yu
Dave Yuover 7 years ago
thanks. no, it’s the same Multiple instances found for the given pattern!
Dave Yu
Dave Yuover 7 years ago
hmm well terraform show does output the expected state for module.subnets.aws_subnet.public.1
Dave Yu
Dave Yuover 7 years ago
I guess that will work for now
sytten
syttenover 7 years ago
Just wanted to say thanks for the terraform modules, it helps me a lot for my school project
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
welcome @sytten
loren
lorenover 7 years ago
i think it is for verification of user identity using pgp keys
loren
lorenover 7 years ago
joshmyers
joshmyersover 7 years ago
Keybase is used to encrypt user credentials so that only that user can decrypt the terraform output (or email)
joshmyers
joshmyersover 7 years ago
It's nice as otherwise you need to figure out how to let users know their credentials in non plain text
deftunix
deftunixover 7 years ago
hi all, I have a quick question. how do you provide estate visibility within dynamic infrastructure?
joshmyers
joshmyersover 7 years ago
There is a "portal" that links you to k8s dashboard, grafana, Prometheus, Kibana
deftunix
deftunixover 7 years ago(edited)
I am not referring just to k8s env dashboard but also to multi-cloud deployment of resource provisioned with terraform for example
joshmyers
joshmyersover 7 years ago
What kind of visibility are you after?
joshmyers
joshmyersover 7 years ago(edited)
Ah, of all your Terraform provisioned infra? Like, what have I actually created?
deftunix
deftunixover 7 years ago
I would like to create a visual single source of truth in which show all the resource provisioned
joshmyers
joshmyersover 7 years ago
Terraform graph and feed it to graphiz?
joshmyers
joshmyersover 7 years ago
terraform graph | dot -Tpng > graph.png
deftunix
deftunixover 7 years ago
I was referring to something able to put terraform resources information into a kv to be consumed with other tools
deftunix
deftunixover 7 years ago
statefile -> exporter -> kv
joshmyers
joshmyersover 7 years ago
Consumed by anything specific?
deftunix
deftunixover 7 years ago
not currently
Steven
Stevenover 7 years ago
@deftunix Terraboard might be the type of end result you're looking for https://github.com/camptocamp/terraboard
deftunix
deftunixover 7 years ago
I will check
deftunix
deftunixover 7 years ago(edited)
yes, it's exactly what I was looking for.
deftunix
deftunixover 7 years ago
good starting point. thank you @Steven
Steven
Stevenover 7 years ago
welcome
joshmyers
joshmyersover 7 years ago
Nice
loren
lorenover 7 years ago
not sure about the geodesic part, but any variable in terraform would generally just be var.stage, and terraform will automatically read it from the env TF_VAR_stage if present
loren
lorenover 7 years ago
TF_CLI_ARGS is something else entirely
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Though those variables are all for the s3 backend in the terraform provider.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Terraform backends do not support interpolation
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
That’s why we use the TF_CLI_ARGS pattern
Maxim Tishchenko
Maxim Tishchenkoover 7 years ago(edited)
../modules/aws_cross_account_role

resource "aws_iam_role_policy_attachment" "policies" {
  count = "${length(var.role_policies)}"
  policy_arn = "${var.role_policies[count.index]}"
  role = "${aws_iam_role.role.id}"
}


another file

module "remote_role" {
  source = "../modules/aws_cross_account_role"

  aws_external_account_id = "${local.aws_account_id}"
  role_name = "remote_role"
  role_policies = [
    "arn:aws:iam::aws:policy/ReadOnlyAccess",
    "${aws_iam_policy.invoke_flow_resources_lambda_access.arn}",
    "${aws_iam_policy.create_function_flow_resources_lambda_access.arn}"
  ]
}


does anyone know what am I missing here?
I’m getting error
* module.remote_role.aws_iam_role_policy_attachment.policies: aws_iam_role_policy_attachment.policies: value of 'count' cannot be computed
Samuli
Samuliover 7 years ago
could one of the aws_iam_policy in the latter file be missing when you are trying to execute it?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
@Maxim Tishchenko that’s the infamous count error
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
in your case, TF can’t calculate dynamic counts across modules
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
(if you put all the resources into one module, it should be ok)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
the easiest (but not prettiest) way to fix it in your case would be to add var.role_policies_count variable to the modules/aws_cross_account_role module and provide it from module "remote_role"
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
module "remote_role" {
  role_policies_count = 3
  role_policies = [
    "arn:aws:iam::aws:policy/ReadOnlyAccess",
    "${aws_iam_policy.invoke_flow_resources_lambda_access.arn}",
    "${aws_iam_policy.create_function_flow_resources_lambda_access.arn}"
  ]
}
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
resource "aws_iam_role_policy_attachment" "policies" {
  count = "${var.role_policies_count}"
  policy_arn = "${var.role_policies[count.index]}"
  role = "${aws_iam_role.role.id}"
}
Maxim Tishchenko
Maxim Tishchenkoover 7 years ago
Mmmm
Maxim Tishchenko
Maxim Tishchenkoover 7 years ago
I got you, thanks
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
we do it in some of our modules, like i said not pretty, but works
Richy de la cuadra
Richy de la cuadraover 7 years ago
hi,somebody have a module for create a cloudwatch event for keep lambda warm?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
@Richy de la cuadra found this example, can be put into a module https://github.com/terraform-providers/terraform-provider-aws/issues/3040
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Richy de la cuadra
Richy de la cuadraover 7 years ago
very helpfull thanks @Andriy Knysh (Cloud Posse)!
squidfunk
squidfunkover 7 years ago
Anyone else experiencing very slow TF Lambda deployments? 30 seconds+ for every single function. This was much faster before. Already upgraded to the newest TF, no changes.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
we did not test lambda recently
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
does it eventually finish successfully ?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
can you test the same from the AWS console and compare the times?
tamsky
tamskyover 7 years ago
@squidfunk are things slow inside a geodesic container?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
(he's not using geodesic yet)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
😃
tamsky
tamskyover 7 years ago
ok, just curious if it was related to TF_CLI_PLAN_PARALLELISM=2 -- but looking at that var now, I'm guessing it only affects plan ?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Yea... but you're right, that does slow things down.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
We have that due to running terraform under ECS wher we experienced AWS platform rate limits
Maarten van der Hoef
Maarten van der Hoefover 7 years ago
Hi, is anyone using Travis with private modules, what’s the best way to get git+ssh working there ?
squidfunk
squidfunkover 7 years ago(edited)
Hey guys. I was at my girlfriend's parents for the last days and, well, it turned out it was just the upstream. They have a really bad internet connection. Stupid me 😉 Sorry!
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
@squidfunk AWS WorkSpaces FTW!
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
of anyone is looking for something to contribute, looks like there's a bug affecting a lot of users: https://github.com/cloudposse/terraform-aws-cloudfront-s3-cdn/issues/39
h3in3k3n
h3in3k3nover 7 years ago(edited)
has anyone tried terratest ? new toy from gruntwork ?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
geez, so many breaking changes
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
(our bad for not pinning provider)
loren
lorenover 7 years ago
breaking changes are what major releases are for, features go in minor releases 🙂
deftunix
deftunixover 7 years ago
hi all, I am deploying a consul and vault cluster on aws using terraform. do you have any example of pipeline able to manage the rolling upgrade of the cluster? all the nodes are in ASG
Brandon Wilson
Brandon Wilsonover 7 years ago
Anyone running into x509: certificate signed by unknown authority with registry.terraform.io?
Brandon Wilson
Brandon Wilsonover 7 years ago
running into it when running a terraform init
tweetyix
tweetyixover 7 years ago
@Brandon Wilson Is your certstore up to date? Does curl connect without issues?
Brandon Wilson
Brandon Wilsonover 7 years ago
curl connects without issues
Brandon Wilson
Brandon Wilsonover 7 years ago
curl <https://registry.terraform.io/.well-known/terraform.json>
{"modules.v1":"/v1/modules/","providers.v1":"/v1/providers/"}
cabrinha
cabrinhaover 7 years ago
It'd be nice if we could configure EKS workers to be spot instances here: https://github.com/cloudposse/terraform-aws-eks-workers
cabrinha
cabrinhaover 7 years ago
So, I just used this example to get up and running with an EKS cluster: https://github.com/cloudposse/terraform-aws-eks-cluster
cabrinha
cabrinhaover 7 years ago
But I'm getting the error: no nodes available to schedule pods
cabrinha
cabrinhaover 7 years ago
I can see that I have one worker node up, but I can't see that node via kubectl
cabrinha
cabrinhaover 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
@cabrinha have you seen our example in terraform-root-modules
cabrinha
cabrinhaover 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
@Andriy Knysh (Cloud Posse) has been keeping this up to date
cabrinha
cabrinhaover 7 years ago
@Erik Osterman (Cloud Posse) thanks for pointing me in the right direction, just wondering why the module doesn't have a complete working example in it 🤔
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
keep in mind, recent changes in terraform-aws-provider has introduced some issues; pinning to a pre-2.0 module might help or ensuring you're on 2.1
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
there are so many moving pieces in terraform between versions of terraform, versions of providers and versions of modules. We cannot test all combinations. We try to keep it all stable and working, but we are not a large company 😃 and all maintenance is paid for by cloudposse ($10K/mo!) Despite that, we try to stay on top of our 130+ terraform modules, 60 tools in cloudposse/packages, 50+ helm charts in cloudposse/charts, 300+ original open source projects on our github, dozens of pull requests every week, code reviews, etc. It's not easy and pull requests are greatly appreciated.
cabrinha
cabrinhaover 7 years ago
😅 I understand thanks
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
@cabrinha the example here https://github.com/cloudposse/terraform-aws-eks-cluster/tree/master/examples/complete is a complete and working example, was tested about a month ago and also deployed by a few other people
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Samuli
Samuliover 7 years ago
variable?
Samuli
Samuliover 7 years ago
How is it hard coding if you have it as a variable?
loren
lorenover 7 years ago
you can look it up, but you have to pass the provider to the data source so the credential has permissions to query the other account
loren
lorenover 7 years ago(edited)

data "aws_vpc" "accepter_vpc" {
  provider = "aws.<ALIAS_ACCEPTER>"

  tags = {
    Name = "${var.stage}-vpc"
  }
}
Samuli
Samuliover 7 years ago
but that would require some identification to be added eg. the var.stage in the example (+ the provider configuration) so why not just use vpc-id?
loren
lorenover 7 years ago
personally i agree, i don't like implicit references like that, but i'm just offering an answer to the question tho 😄
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Here’s how we used the terraform remote state provider to achieve a similar requirement https://github.com/cloudposse/terraform-root-modules/blob/master/aws/root-dns/ns/main.tf#L27-L51
Jan
Janover 7 years ago
what was the tf tool being used to mask passwords
Maarten van der Hoef
Maarten van der Hoefover 7 years ago

output "db_password" {
  value       = aws_db_instance.db.password
  description = "The password for logging in to the database."
  sensitive   = true
}
`
Jan
Janover 7 years ago
sensitive = true
Jan
Janover 7 years ago
oh really
Jan
Janover 7 years ago
tfmask was the tool though
Jan
Janover 7 years ago(edited)
thanks dude
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
you create IAM users and then add them to the groups (admin or readonly)
xluffy
xluffyover 7 years ago
You should start with https://github.com/cloudposse/terraform-aws-iam-user/blob/master/main.tf

terraform-aws-iam-user help to create a normal IAM user (u can login with password aws_iam_user_login_profile). After that, U need to assign this user to a group.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
xluffy
xluffyover 7 years ago
terraform-aws-iam-assumed-roles is a big module, create a group, role, policy ….
Arvind
Arvindover 7 years ago
@Arvind has joined the channel
Arvind
Arvindover 7 years ago
Do we have any sample git repo where we are provisioning the infrastructure as per environments basis ex:stage/lab/prod.And i need to understand how terraform.tf file will read the environments specific variables. ex: in lab i require Redis instance smaller size while in stage and prod i require bigger size Redis instances.
joshmyers
joshmyersover 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
@joshmyers’s suggestion is spot on
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
basically, you put all common code in terraform-root-modules (like blueprints)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
then you use that code in all of your account environments (E.g. <http://testing.cloudposse.co|testing.cloudposse.co> is ours)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
we have root, staging, prod, etc.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
that's where the custom settings go for each environment.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
it's been our experience that we want to run identical code (e.g. modules) in environments like staging and prod, but staging and prod necessarily differ in areas like instance sizes or perhaps the number of clusters
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
for example, in production for an Enterprise SaaS platform, there might be one cluster per enterprise customer, but you wouldn't run one cluster per enterprise customer in staging.
xluffy
xluffyover 7 years ago(edited)
Hey, I have a issue with terrarom-aws-iam-user. First i run apply, my user add to group admin, but when i run again, my user is remove to group


  ~ module.assumed_roles.aws_iam_group_membership.admin
      users.#:          "1" => "0"
      users.1070365961: "quang" => ""



  ~ module.quang.aws_iam_user_group_membership.default
      groups.#:          "0" => "1"
      groups.3764225916: "" => "g_ops_users"
joshmyers
joshmyersover 7 years ago
@xluffy what are you running?
Arjun
Arjunover 7 years ago(edited)
is there any possibility of mentioning an EBS resource in launch configuration using interpolation? ( i am trying to abstract out this , to conditionally mention an extra block device in LC)
Jan
Janover 7 years ago(edited)
heads up this is no longer valid, https://github.com/cloudposse/terraform-root-modules/blob/master/aws/account-dns/main.tf#L26-L35

* aws_route53_record.dns_zone_soa: 1 error(s) occurred:

• aws_route53_record.dns_zone_soa: [ERR]: Error building changeset: InvalidChangeBatch: [Tried to create resource record set [name='some.example.com.', type='SOA'] but it already exists]
 
Jan
Janover 7 years ago
used to work, 27 days ago for sure
Jan
Janover 7 years ago
but also not needed as SOA gets created by default
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Jan
Janover 7 years ago
ah
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago(edited)
It’s not that we want to create a new SOA; it is essential to set a low SOA TTL to prevent negative ttl caching in dynamic service discovery environments. E.g. a negative cache hit can cause 15 minutes of downtime for no good reason.
mrwacky
mrwackyover 7 years ago
Has this guy popped up in here yet?
https://github.com/iann0036/AWSConsoleRecorder
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Hah, wow, haven't seen this before
mrwacky
mrwackyover 7 years ago
I wonder if it works
h3in3k3n
h3in3k3nover 7 years ago(edited)
Hi everyone. I don’t know which channel to post this question but as I tend to implement with TF so I posted here.

My question is : How did you manage ECR repository on Terraform or alternative tools you using ?
The problem I facing is
- i want to create multiple ecr repositories in 1 TF code, so which is the good way to generate those codes ? I writing go template. I think I will use that to generate the TF code from Yaml File
My yaml file format like this

[name-of-repository]
    - [username-arn-id-a] - pull
    - [username-arn-id-b] - push
[name-of-repository2]
    - [username-arn-id-d] - pull
    - [username-arn-id-c] - push

Any one has other ideas feel free to tell me once, Appreciate !
Nikola Velkovski
Nikola Velkovskiover 7 years ago
@h3in3k3n the way I would do it is to create/find a module that has a proper ecr policy set and just use that one.
The resource by itself in terraform is super small but with the policy it can get a bit bigger also you might need to pass the policy per repo or whatever your use case requires
Nikola Velkovski
Nikola Velkovskiover 7 years ago
Hi People, any recommendation for an IDE or a vim setting in which I can detect unused variables in terraform ?
Nikola Velkovski
Nikola Velkovskiover 7 years ago(edited)
I think that one of the jetbrains IDEs has this feature.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Jetbrains IDEA has a very nice terraform support
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Can detect missing and unused variables
Nikola Velkovski
Nikola Velkovskiover 7 years ago
yeah that's what I remember as well
Nikola Velkovski
Nikola Velkovskiover 7 years ago
but I don't want to switch my precious VIM 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
And highlights everything
giphy
giphyover 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
It can emulate VIM editor so you don’t lose anything :)
Nikola Velkovski
Nikola Velkovskiover 7 years ago
hmmm
Nikola Velkovski
Nikola Velkovskiover 7 years ago
ok let me check that out
Nikola Velkovski
Nikola Velkovskiover 7 years ago
thanks!
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
It’s a very nice product in my opinion
loren
lorenover 7 years ago
vscode does this also for terraform
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Supports all possible languages and frameworks
Nikola Velkovski
Nikola Velkovskiover 7 years ago
ok vscode I'ave installed but I haven't seen this feature in action , do I need to install a special plugin @loren?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Vscode not bad too
loren
lorenover 7 years ago
yes, you need a plugin, one sec and i'll get the name
Nikola Velkovski
Nikola Velkovskiover 7 years ago
thanks, I already have the tf plugin installed though
loren
lorenover 7 years ago
yeah, should just be terraform
loren
lorenover 7 years ago
you don't see references like this?
Nikola Velkovski
Nikola Velkovskiover 7 years ago
no I get
Nikola Velkovski
Nikola Velkovskiover 7 years ago
Nikola Velkovski
Nikola Velkovskiover 7 years ago
for every var..
Nikola Velkovski
Nikola Velkovskiover 7 years ago
I think it's the way we use terraform ..
loren
lorenover 7 years ago
oh, it might not work when the references are in a different file?
Nikola Velkovski
Nikola Velkovskiover 7 years ago
yes they are in a different folder..
Nikola Velkovski
Nikola Velkovskiover 7 years ago
and then we sprinkle some makefile magic
Nikola Velkovski
Nikola Velkovskiover 7 years ago
to make an apply 🙂
loren
lorenover 7 years ago
well, different file, same directory works fine
Nikola Velkovski
Nikola Velkovskiover 7 years ago
sorry different folder
Nikola Velkovski
Nikola Velkovskiover 7 years ago
so the plugin thinks it's a different state
Nikola Velkovski
Nikola Velkovskiover 7 years ago
ok now I get it
loren
lorenover 7 years ago
loren
lorenover 7 years ago
that's in a variables.tf file, same directory as main.tf where all the references are
Nikola Velkovski
Nikola Velkovskiover 7 years ago
yeah , that's not the case where I currently work at
loren
lorenover 7 years ago
gotcha
Nikola Velkovski
Nikola Velkovskiover 7 years ago
but thanks for the help ! now I know that vscode works as expected 🙂
loren
lorenover 7 years ago
it is hard to do this kind of lint/validation when you have some wrapper/generator around your tf code
loren
lorenover 7 years ago
well, harder, i guess. need to generate it, then validate the output
Nikola Velkovski
Nikola Velkovskiover 7 years ago
yeah, I am not a fan of the wrappers as well.
Nikola Velkovski
Nikola Velkovskiover 7 years ago
but there's a way , the Makefile generates a main.tf file
Nikola Velkovski
Nikola Velkovskiover 7 years ago
so I can just use that one 🙂
loren
lorenover 7 years ago
yep
Nikola Velkovski
Nikola Velkovskiover 7 years ago
yeah 🙂
Nikola Velkovski
Nikola Velkovskiover 7 years ago
thanks guys for your help!
Nikola Velkovski
Nikola Velkovskiover 7 years ago
Much appreciated
Nikola Velkovski
Nikola Velkovskiover 7 years ago
hmm I found out that the current plugin in vsode sets 0 references even if the variable is invoked inside a function
Nikola Velkovski
Nikola Velkovskiover 7 years ago(edited)
like so "{tf_function(var.something)}"
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
IDEA can actually see and analyze variables in functions (and even in plain text)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
it does not show the ref count as vscode does
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago(edited)
but when you click “Find Usages” on anything
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
it shows all references
Nikola Velkovski
Nikola Velkovskiover 7 years ago
oh nice..
Nikola Velkovski
Nikola Velkovskiover 7 years ago
I am definitelly trying it out
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
highlights wrong/missing vars
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
johncblandii
johncblandiiover 7 years ago
that moment when you’re getting a product demo from a company and you notice a Cloud Posse browser tab + the label module naming format
johncblandii
johncblandiiover 7 years ago
Turns out, @Erik Osterman (Cloud Posse) is known from the days at Sumo (by Nick Coffee)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Hahah wow! small freggin world.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Yea, I remember Nick.
johncblandii
johncblandiiover 7 years ago(edited)
He’s at Harness https://harness.io/
mrwacky
mrwackyover 7 years ago
if I want to to test terraform 0.12 beta.. how do I get providers built against it?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
we do DNS zone delegation. In the root DNS, we provision the parent domain (<http://acme.co.uk|acme.co.uk>) Zone and subdomains (e.g. <http://test.acme.co.uk|test.acme.co.uk>) zones, then we provision the DNS zone <http://test.acme.co.uk|test.acme.co.uk> in the test account, and then add the name server for <http://test.acme.co.uk|test.acme.co.uk> in the parent DNS zone in the root account
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
for root ^
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
for member account ^
mmuehlberger
mmuehlbergerover 7 years ago
I was wondering the same thing today. I’m debating with myself if it makes sense to split your naming into an infastructure zone like acme.cloud and have a separate one for your public-facing domain, being more picky which namespaces to use.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
we recommend having a service discovery domain which is not your branded vanity domain
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
so we have <http://cloudposse.org|cloudposse.org> as our infra domain
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
<http://cloudposse.com|cloudposse.com> as our vanity domain
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
(and <http://cloudposse.co|cloudposse.co> is just for our examples)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Yes so CloudPosse.org has NS records for all our member accounts
mmuehlberger
mmuehlbergerover 7 years ago
Makes total sense now that you say it. 😄
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
mmuehlberger
mmuehlbergerover 7 years ago
The only issue I thought might be a little bit tricky is with certificates, but I think it would just be the load balancer that needs to have a certificate for the vanity domain applied, so not too bad.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
it’s possible now to associate multiple ACM ARNs with an ALB
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
the alternative is to use SANs
mmuehlberger
mmuehlbergerover 7 years ago
Ah, I somehow missed that. If possible, I try to avoid using SANs on vanity domains, it’s just neater.
mmuehlberger
mmuehlbergerover 7 years ago
Thanks for the clarification.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
what we have in the root account cloudposse.co
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
certificates get provisioned in each account separately https://github.com/cloudposse/terraform-root-modules/blob/master/aws/acm/main.tf
Igor
Igorover 7 years ago
Since we're on the subject, is it a good idea to create a hosting zone for a customer's site and ask them to create an NS record, instead of giving them the ALB CNAME? My thoughts are that this way I can change the ALB if ever required without having to request a customer to make a DNS change.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
if the service discovery domain will not change, then probably yes, give them the NS records
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
on the other hand, if you have a service discovery domain, e.g. <http://prod.mycompany.co|prod.mycompany.co> which you control, you can ask them to add the CNAME record from their main (vanity) domain to <http://prod.mycompany.co|prod.mycompany.co> (which should never change as well). No ALBs involved
Igor
Igorover 7 years ago
Is one approach better than the other?
Alex Siegman
Alex Siegmanover 7 years ago(edited)
Amazon charges for CNAMEs, if that kind of penny pinching bothers you. Clarification: cnames count towards your queries, but those are stupidly cheap anyways. I've had bosses squawk over it though. A ALIAS records do not.
Alex Siegman
Alex Siegmanover 7 years ago
I think the two approaches, it's mostly just how you want to manage it.
Igor
Igorover 7 years ago
In this case, I think the CNAME would be on the customer side, so no additional cost
Alex Siegman
Alex Siegmanover 7 years ago
Oh, I missed the "customer's site" - in that case, I might say you want to control your own domain stuff, make them use cnames, but that's just me
Igor
Igorover 7 years ago(edited)
CNAME has more visibility. Someone can access the site using <http://customer.mycompany.co|customer.mycompany.co> instead of <http://site.customer.co|site.customer.co> as intended.
Igor
Igorover 7 years ago(edited)
I just tried the NS approach for the first time, and the ask was met with some skepticism (from the customer)
S
SweetOpsover 7 years ago
Are you using some of our terraform-modules in your projects? Maybe you could leave us a testimonial! It means a lot to us to hear from people like you.
tweetyix
tweetyixover 7 years ago
Could resources be deployed into two different AWS accounts having a single "TF environment" and still use some sort of ENV variables for keys? Is there even support for vault?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
we handle that by having “generic” TF modules (that provision resources but all settings come from variables, they have no identity by itself). Then we have Docker containers (geodesic in our case) for each AWS account. We copy the modules into each containers. All those TF vars get populated from ENV vars which can come from a few different places (Dockerfile, CI/CD pipeline, AWS SSM, chamber, https://direnv.net, etc.). When starting a container for a particular account, it already has all the code (TF, Helm, Helfiles, etc.) and all the ENV vars populated, so the container has all the info to provision infra for that account
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago(edited)
since all settings are coming from ENV vars (including AWS regions and account IDs), and all TF modules use terraform-null-label module to uniquely name all resources, the same setup could be used with multiple AWS accounts, or just with one if needed (in which case all containers will login/assume role into the same AWS account, but all the resources will have diff environment/stage in their names, so all names are unique anyway)
Bitjunkie
Bitjunkieover 7 years ago
Does this work cross accounts? Would I just need to create another hosted zone with the same NS?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
it is cross-accounts
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
in the member accounts (prod, staging, dev) you create corresponding DNS zones (e.g. prod.cloudposse.co)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago(edited)
then get the NS servers from them
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
and update the NS records in the root account
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
obviously you update the NS in the registrar (if it’s not Route53) to the root DNS zone name servers (first record in the image above)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
then, to put it very simple, every request to let’s say prod.cloudposse.co will first hit the root zone for cloudposse.co and the DNS server will return the name servers for prod.cloudposse.co
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago(edited)
then all other records for prod.cloudposse.co will be requested from its name servers (which could be in the same AWS account or diff, does not matter)
johncblandii
johncblandiiover 7 years ago
Anyone else get access to TFE for the free remote state? I’m curious how people feel about it.
antonbabenko
antonbabenkoover 7 years ago
I think @johncblandii meant Terraform Saas which is expected to be affordable for many once it is public. If so, my question is more like this:
1. Will I be able to run Terragrunt there (as a custom docker image with all other deps)?
mmuehlberger
mmuehlbergerover 7 years ago
You need a public zone for sandbox.acme.co.uk.
mmuehlberger
mmuehlbergerover 7 years ago
Private zones only work within the VPC.
mmuehlberger
mmuehlbergerover 7 years ago
If you want to see how that works you can use dig to see the delegations happening: dig ns <http://sandbox.acme.co.uk|sandbox.acme.co.uk> +trace +nodnssec
mmuehlberger
mmuehlbergerover 7 years ago
As far as I know you can’t scan DNS records easilly
loren
lorenover 7 years ago(edited)
We have that split dns setup in Route53. I wouldn't do it again, starting over
loren
lorenover 7 years ago
Did it for that same reason, originally
loren
lorenover 7 years ago
Not with the same zone name, for sure
mmuehlberger
mmuehlbergerover 7 years ago
A quick article I just found on this topic is https://hackertarget.com/find-dns-host-records/. The only way to get the records is doing a DNS Zone Transfer, which doesn’t work from the public.
loren
lorenover 7 years ago
Use a totally different zone name in the private zone, and that's generally ok
mmuehlberger
mmuehlbergerover 7 years ago
If you have a private zone, I’d use something like .local or .acme as the root zone.
mmuehlberger
mmuehlbergerover 7 years ago
The problem is, if you are working from outside (like you do with a multi-account AWS setup), you will run into more problems than not, if stuff doesn’t work.
loren
lorenover 7 years ago(edited)
Also, we run into problems we with multi account setups, where the accounts cannot resolve each other's records from the private zones. There are ways of doing it, but it's kludgey. So if you have internal services that should be resolvable across accounts, private zones end up being painful. This basic NS delegation that works with public zones does not work at all with private zones
loren
lorenover 7 years ago
I'm hoping the new route53 resolvers will get me out of this mess
mmuehlberger
mmuehlbergerover 7 years ago
This was about finding DNS records out of the blue.
mmuehlberger
mmuehlbergerover 7 years ago
Nothing, I was just backing up my claim.
oscarsullivan [1:00 PM]

Oh really? A long time ago someone told me “don’t put internal infra on public DNS, it just makes it easier for intruders to map out your kit”
mmuehlberger
mmuehlbergerover 7 years ago
Ah, now I understand what you mean. You should only receive all the records that are for acme.co.uk (and not any subdomain).
loren
lorenover 7 years ago
That site is doing more than just attempting a zone transfer (which doesn't work with route53). They're also scraping for site info and guessing common names
mmuehlberger
mmuehlbergerover 7 years ago
Yes, it was more about the text, than the tool itself.
xluffy
xluffyover 7 years ago
Hey, I checked some modules about VPC (on GitHub). But nobody implements a Network ACL on AWS. What are the reasons why don’t do that?
Valter Henrique
Valter Henriqueover 7 years ago
Hey guys, I'm trying to create a domain zone on AWS Route53 and create a certificate that will be used in an ALB later. However, I need to validate the record first. I'm trying the validation_method="DNS" but is taking ages and nothing happens. My impression is that I need to create the domain, create the ALB, assign a record to the ALB and then try to validate the record. Is that right?

By the way, I'm talking about this:
https://www.terraform.io/docs/providers/aws/r/acm_certificate_validation.html
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Hi @Valter Henrique so the dns zone you are requesting the certificate for should be accessible von the internet, meaning if the registrar is not Route53, then you need to create the zone in Route53, get the name servers and update the NS records in the registrar
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
In other words, if AWS can’t access your dns records, the validation will take forever and never finish
Valter Henrique
Valter Henriqueover 7 years ago
Right, thank you very much for that @Andriy Knysh (Cloud Posse)!
Valter Henrique
Valter Henriqueover 7 years ago
I was almost bashing my head against my keyboard hehehe
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Yea, we were bitten by that ad well. Certificate Manager adds the validation record to your zone, and you think if it can write it, it should be able to read it as well :)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
But no, it tries to read using public internet, not the internal network
Valter Henrique
Valter Henriqueover 7 years ago
Indeed
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
But of course, the purpose of that is to validate that you own the domain. Only if you bought the domain, you could update its name servers. Otherwise I could create a DNS zone for your domain and request a cert for it
mmuehlberger
mmuehlbergerover 7 years ago(edited)
I was looking at the ecs modules and I’m trying to figure out, how it’s supposed to work with atlantis. The terraform-aws-ecs-atlantis module uses the default-backend for both service tasks (404 and atlantis), but shouldn’t it use the container image from ECR for the atlantis instance? (I hope this is the right place).
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
@mmuehlberger sounds about right. You might notice that we hardcore the image to the default backend. That is by design.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
The image should always be updated using codebuild/codepipeline, so if you see the 404 then you know it didn’t work :-)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
This is our example implementation
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
That’s the build spec we use
mmuehlberger
mmuehlbergerover 7 years ago
God, I’m an idiot (was a long day already). Of course it get’s updated in the CodePipeline.
mmuehlberger
mmuehlbergerover 7 years ago
Because I used a private helmfiles image, the CodeBuild project failed, and it didn’t update yet. Duh.
mmuehlberger
mmuehlbergerover 7 years ago
The problem is completely on my end. Sometimes I get lost in the dependency chain of Terraform modules.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Ok! Let me know it goes...
mmuehlberger
mmuehlbergerover 7 years ago
It deploys fine now, as it should be (thank you for the pointers). Adding encrypted secrets to the buildspec is kind of a hastle though, because the user is burried deep in modules. Also there is issues with the healthcheck and atlantis shutting down. I need to investigate further tomorrow.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
we use chamber to write all secrets to ssm
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
@mmuehlberger are you using atlantis with #geodesic?
mmuehlberger
mmuehlbergerover 7 years ago
Yes, I’m using geodesic in a slightly more modern form than reference-architectures (like testing). It can read the parameters from SSM just fine, the role just doesn’t have permission to decrypt them.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Cool!
niek
niekover 7 years ago
Does anyone knows if it is possible to construct from a map a string.

For example

{
   "key1" = "val1"
   "key2" = "val2"
}

To: "key1, val1, key2, val2"

Currently having a tf module wich accepts as input a map for tagging in AWS. But there is on one place I need to pass the tags as a list. I prefer to keep my work backwards compatible.
loren
lorenover 7 years ago
the argument has been deprecated for a while, just printing warnings instead error and exit non-zero
loren
lorenover 7 years ago
removing it is backwards compatible for a reasonable number of versions
mmuehlberger
mmuehlbergerover 7 years ago
Just use the branch as a ref. ref=master for instance
mmuehlberger
mmuehlbergerover 7 years ago
Well, you can when looking up a remote terraform state.
rbadillo
rbadilloover 7 years ago
Guys, is there a way to do a split and get the last element of the list ?
rbadillo
rbadilloover 7 years ago
or do I need to know the size of the list ?
rbadillo
rbadilloover 7 years ago
-1 doesn’t work
rbadillo
rbadilloover 7 years ago
I ended up using length function
xluffy
xluffyover 7 years ago
For getting last element in a list, I think can use like that


variable "public_subnet" {
  default = ["10.20.99.0/24" , "10.20.111.0/24", "10.20.222.0/24"]
}

output "last_element" {
  value = "${element(var.public_subnet, length(var.public_subnet) - 1 )}"
}

😄
xluffy
xluffyover 7 years ago
will return to last_element = 10.20.222.0/24
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
@oscarsullivan_old https://github.com/cloudposse/terraform-aws-jenkins was tested by us about a year ago (was deployed many times at the time), so prob a lot of things changed since then
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago(edited)
i think some people used it
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
just need to find them 😉
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
We have not personally used it for some time
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Mohamed Lrhazi
Mohamed Lrhaziover 7 years ago
Hello! is this the place to ask for help about https://github.com/cloudposse/terraform-aws-cloudfront-s3-cdn ?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
@oscarsullivan_old what we noticed before, if deployment fails for any reasons, you need to manaully destroy those data pipelines
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
TF does not detroy them (uses CloudFormation)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
and also, all of that should be updated to use https://aws.amazon.com/backup/
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Honestly, looking at the architecture for deploying a single, non HA Jenkins on beanstalk is enough for me to say I just don’t think it’s worth running jenkins. Plus, to get HA with Jenkins you have to go enterprise. At that point might as well look at other options.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
instead of the efs-backup module which was a hack
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Ya @Andriy Knysh (Cloud Posse) is correct that pipelines should now be replaced with the backups service
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
How long did you wait @Mohamed Lrhazi ?
Mohamed Lrhazi
Mohamed Lrhaziover 7 years ago
Any idea what I am missing?
Mohamed Lrhazi
Mohamed Lrhaziover 7 years ago
oh.. maybe 10 mins ?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
It takes sometimes up to 30 minutes to create a distribution
Mohamed Lrhazi
Mohamed Lrhaziover 7 years ago
h.. but it says DEPLOYED as status...
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Hrm
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
10 m is faster than I have seen. Not sure what is wrong based on your example.
Mohamed Lrhazi
Mohamed Lrhaziover 7 years ago
and I think its actually been more than 30mins
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
You can go into the webconsole and poke around
Mohamed Lrhazi
Mohamed Lrhaziover 7 years ago
still giving same error...
Mohamed Lrhazi
Mohamed Lrhaziover 7 years ago
yes, I looked at the s3 bucket and looks like it did assign the right perms, from what I can guess
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Did you use a custom SSL cert?
Mohamed Lrhazi
Mohamed Lrhaziover 7 years ago
Nope!
Mohamed Lrhazi
Mohamed Lrhaziover 7 years ago
could that be it? docs dont says thats required!
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
If yes, the cloudfront provided one will not work
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
In your code I see the cert ARN provided
Mohamed Lrhazi
Mohamed Lrhaziover 7 years ago
Oh sorry you're right!!! I did add that
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Also the S3 bucket is not a website
Mohamed Lrhazi
Mohamed Lrhaziover 7 years ago
oh.. the module does not do that for me?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
You have to access any file by adding its name after the URL
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
There is another module for that
Mohamed Lrhazi
Mohamed Lrhaziover 7 years ago
ok. let me poke around and see if I can make it work... one last question.. is the module supposed to also create route53 records needed?
Mohamed Lrhazi
Mohamed Lrhaziover 7 years ago
cause it does not seem like it did in my simple test case.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
@Mohamed Lrhazi 1 sec
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
@oscarsullivan_old did you find the pipelines in the AWS console? you need to manually delete them 😞
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
what do you see in CloudFormation? Delete those stacks manually
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
when you run terraform destroy, what’s the error?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
hmm, never seen those 😞
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
i guess just go to Route53 and delete those records
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
(and fix the variables and outputs so it does not complain)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
i hope you did that in a test account that you could nuke somehow 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
i mean anything to destroy it, not exactly using the nuke module
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
so I think part of the issue was that the pipelines are not supported in the region (and maybe other resources)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
but the error reporting is bad
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago(edited)
@Mohamed Lrhazi this module https://github.com/cloudposse/terraform-aws-s3-website does create an S3 website
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
and then you use this https://github.com/cloudposse/terraform-aws-cloudfront-cdn to add CDN for it
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
(this https://github.com/cloudposse/terraform-aws-cloudfront-s3-cdn creates a regular S3 bucket, not a website, and points a CloudFront distribution to it)
Mohamed Lrhazi
Mohamed Lrhaziover 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
since it’s not a website, you have to access all the files by their names, e.g. https://d18shdqwx0ry07.cloudfront.net/index.html
Mohamed Lrhazi
Mohamed Lrhaziover 7 years ago
I tried that, still getting same "Failed to contact the origin." error!
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
and since you provided your own SSL cert (I guess you created it already), you have to access the site by the DNS name (not CloudFront) : https://catholic.edu/index.html
Mohamed Lrhazi
Mohamed Lrhaziover 7 years ago
ok.. maybe that's the key... thanks a lot!... I will see whats wrong with the DNS part...
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
@Mohamed Lrhazi to create an S3 website (not just a regular S3 bucket) and then point a CloudFront CDN to it, we have this example https://github.com/cloudposse/terraform-root-modules/blob/master/aws/docs/main.tf
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
working because it’s how https://docs.cloudposse.com is deployed
Mohamed Lrhazi
Mohamed Lrhaziover 7 years ago
Cool! Thanks!. I will start from that example then. I was really hoping a found a module that would allow me to just say : my site is foo.dom.ain, my content is in content folder... go do it 🙂
but this example does not look that bad.. just need to learn me a bit of terraform 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
such a module could be created by using these two
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
which the example above shows
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
one of the reasons that they have not been combined in one module is that you can use an S3 website without any CDN
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
point your DNS to the S3 website endpoint
Mohamed Lrhazi
Mohamed Lrhaziover 7 years ago
I think s3 website does not support ssl, so one needs cloudfront for that...

@oscarsullivan_old yes, I meant here is my static content, host it in s3+cloudfront... use my pre-created ssl cert, but do everything else for me. s3/perms/route53/cloudfront. all from one config file where I just say: fqdn, content path, and I guess ssl arn. but I think with all the examples you guys pointed out... am probably very close.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Just copy the example, it’s two modules to call instead of one
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
And the data source will lookup your pre-created SSL cert
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Or just provide the cert ARN and don’t use the lookup
xluffy
xluffyover 7 years ago
@Andriy Knysh (Cloud Posse) I have a question this module

https://github.com/cloudposse/terraform-aws-vpc-peering/blob/master/variables.tf#L2

enabled is boolean variable. I see you check this variable in main.tf like that count = "${var.enabled == "true" ? 1 : 0}"

Why not use enabled = true and check with syntax count = "${var.enabled ? 1 : 0}". I think it is make sense.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
because TF recommends using strings instead of boolean values 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
the issue is that booleans are always converted to strings in TF
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago(edited)
but that conversion is different (not consistent) in variables, tfvars files and on the command line
xluffy
xluffyover 7 years ago
wow, thank you 🙂 I see
xluffy
xluffyover 7 years ago
👍️ thank @oscarsullivan_old I’m just curious 😄
Arvind
Arvindover 7 years ago
I have successfully inserted my AWS_ACCESS_KEY and AWS_SECRET_KEYS in Vault

MacBook-Pro-5$ vault read  secret/infra_secrets
Key                 Value
---                 -----
refresh_interval    xxh
access_key          xxxxx
secret_key          xxxx


Now can anyone can suggest to me how should i use this keys in my main.tf(code piece) so i can provision the infra in aws.
Arvind
Arvindover 7 years ago
#vault
Arvind
Arvindover 7 years ago
xluffy
xluffyover 7 years ago
hey, I have a module for creating 2 VPC + another module for creating a peering between 2 VPC. In peering module, I have a read-only for counting route table. But I can’t count if i don’t create VPC first. How to depend them?


Error: Error refreshing state: 2 error(s) occurred:

* module.peering.data.aws_route_table.acceptor: data.aws_route_table.acceptor: value of 'count' cannot be computed
* module.peering.data.aws_route_table.requestor: data.aws_route_table.requestor: value of 'count' cannot be computed
xluffy
xluffyover 7 years ago
yeah, just create 2 vpcs, after that, will create a peering between them
xluffy
xluffyover 7 years ago
terraform doesn’t support depend on modules.
xluffy
xluffyover 7 years ago
yeah, will work. because u create VPC first (run vpc.tf).

But if u have a peering module + vpc module in a project. Peering module can’t query data in a VPC if VPC doesn’t create.
xluffy
xluffyover 7 years ago(edited)
https://github.com/cloudposse/terraform-aws-vpc-peering/blob/master/main.tf#L38


data "aws_route_table" "requestor" {
  count     = "${var.enabled == "true" ? length(distinct(sort(data.aws_subnet_ids.requestor.ids))) : 0}"
  subnet_id = "${element(distinct(sort(data.aws_subnet_ids.requestor.ids)), count.index)}"
}


Lookup route tables from a VPC. If this VPC doesn’t create. will fail
xluffy
xluffyover 7 years ago
I see
joshmyers
joshmyersover 7 years ago
I prefer a main, at least for common elements
joshmyers
joshmyersover 7 years ago
¯\(ツ)/¯
joshmyers
joshmyersover 7 years ago
That is quite a preference thing, not sure if quite a coding standard
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
agree with both
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
I like a main.tf for common stuff like the provider definition
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
and then when a lot of stuff is going on, break it out by .tf files like @oscarsullivan_old
erik
erikover 7 years ago
joshmyers
joshmyersover 7 years ago
What about that sneaky data source that gets used all over the shop by r53.tf vpc.tf etc
joshmyers
joshmyersover 7 years ago
Anyway, the shed is blue.
joshmyers
joshmyersover 7 years ago
¯\(ツ)/¯
joshmyers
joshmyersover 7 years ago
Where is this?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
(PST)
joshmyers
joshmyersover 7 years ago
01:30 GMT? @oscarsullivan_old
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
6:30 PM Wednesday, Greenwich Mean Time (GMT)
joshmyers
joshmyersover 7 years ago
ahh, I need to nip out but would be good to try and make that 👍️
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
you can definitely use the same vpc
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
the “SHOULD” would come down to a business requirement
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
e.g. “the EKS cluster SHOULD run in a separate PCI compliant VPC”
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
yup
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
haha, 🤞
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
and open issues that others have encountered with EKS
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
most of the time errors encountered are due to missing a step
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
tested many times by a few people
DaGo
DaGoover 7 years ago
Has anyone tried this wizardry? https://modules.tf
xluffy
xluffyover 7 years ago
😄 we have author of this site here.
mmuehlberger
mmuehlbergerover 7 years ago
I like defaults, for things, where I know that I’m not going to change them in every stage/environment or variables where it makes sense to have one. (e.g. require stage to be set explicitly, but like you said, something like instance_type is fine, even though, you might want to change it in every stage anyways). For something like PORT I’d also set a default, usually to whatever the default is for the technology I’m using.

My take on defaults is basically: try to make it as easy as possible to use whatever you are building with ar little extra configuration as possible.
mmuehlberger
mmuehlbergerover 7 years ago(edited)
Valid RFC1918 CIDRs are only those 3:

10.0.0.0        -   10.255.255.255  (10/8 prefix)
172.16.0.0      -   172.31.255.255  (172.16/12 prefix)
192.168.0.0     -   192.168.255.255 (192.168/16 prefix)
mmuehlberger
mmuehlbergerover 7 years ago
Everything else is in the public IP space.
mmuehlberger
mmuehlbergerover 7 years ago
AWS only allows /16 networks, not more.
mmuehlberger
mmuehlbergerover 7 years ago
No. /8 means, the first 8 bits must be fixed (10 in this case). /16 means, the first 16 bits must be fixed (so the first 2 parts)
Steven
Stevenover 7 years ago
10.0.0.0/16 is fine, 14.0.0.0/16 is not
Samuli
Samuliover 7 years ago
10.0.0.0/16 is subnet of 10.0.0.0/8
mmuehlberger
mmuehlbergerover 7 years ago
With VPCs in AWS you can have a range of 10.0.0.0/16 to 10.255.0.0/16 as valid network CIDRs. Also 172.16.0.0/16 to 172.31.0.0/16 and 192.168.0.0/16.
mmuehlberger
mmuehlbergerover 7 years ago
(plus 100.64.0.0/16 to 100.127.0.0/16 but this is carrier-grade NAT, so better stay away from that)
mmuehlberger
mmuehlbergerover 7 years ago
Exactly.
loren
lorenover 7 years ago
I forget where I read it, but there is a lot of truth to the quote, "in the cloud, everyone is a network engineer"
loren
lorenover 7 years ago
VPCs, subnets, security groups, NACLs, VPNs, WAFs, load balancers, oh my! 😂
Samuli
Samuliover 7 years ago
try terraform destroy without the changes first?
rbadillo
rbadilloover 7 years ago
Team any suggestions on how to fix this error ?


"data.aws_vpc.vpc.tags" does not have homogenous types. found TypeString and then TypeMap in ${data.aws_vpc.vpc.tags["Name"]}
rbadillo
rbadilloover 7 years ago
I have some Kubernetes Tags, doing some googling says to delete those tags but I want to avoid that
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
so you can delete/update those tags in kubernetes. Terraform data sources just read them and you can’t change them on the fly
ldlework
ldleworkover 7 years ago
In the terraform-aws-ecs-codepipeline module, it has an example buildspec: https://github.com/cloudposse/terraform-aws-ecs-codepipeline#example-buildspec
ldlework
ldleworkover 7 years ago
Where are the variables like $REPO_URL and $IMAGE_REPO_NAME coming from?
ldlework
ldleworkover 7 years ago
They're not official build environment variables.
ldlework
ldleworkover 7 years ago
Oh I see, it's provided by the module.
LeoGmad
LeoGmadover 7 years ago(edited)
Has anyone encountered issues lately with the module terraform-aws-dynamic-subnets.
I believe it has something to do with AWS adding an AZ (us-west-2d) to the us-west-2 region some time in Feb.

I ended up removing the subnets in one of my environments before being able to recreate them.
Not something I want to do in OPS.
Any insight on the issue?


-/+ module.subnets.aws_subnet.public[0] (new resource required)
      id:                                                                                         "subnet-XXX" => <computed> (forces new resource)
      arn:                                                                                        "arn:aws:ec2:us-west-2:XXX:subnet/subnet-XXX" => <computed>
      assign_ipv6_address_on_creation:                                                            "false" => "false"
      availability_zone:                                                                          "us-west-2a" => "us-west-2a"
      availability_zone_id:                                                                       "usw2-az2" => <computed>
      cidr_block:                                                                                 "10.0.96.0/19" => "10.0.128.0/19" (forces new resource)

-/+ module.subnets.aws_subnet.public[1] (new resource required)
      id:                                                                                         "subnet-XXX" => <computed> (forces new resource)
      arn:                                                                                        "arn:aws:ec2:us-west-2:XXX:subnet/subnet-XXX" => <computed>
      assign_ipv6_address_on_creation:                                                            "false" => "false"
      availability_zone:                                                                          "us-west-2b" => "us-west-2b"
      availability_zone_id:                                                                       "usw2-az1" => <computed>
      cidr_block:                                                                                 "10.0.128.0/19" => "10.0.160.0/19" (forces new resource)
    
LeoGmad
LeoGmadover 7 years ago
Sorry I don't have the actual error output but it complained about the CIDR already existing
LeoGmad
LeoGmadover 7 years ago
Well technically its that module.subnets.aws_subnet.public[1] which never gets deleted
LeoGmad
LeoGmadover 7 years ago
or updated.
LeoGmad
LeoGmadover 7 years ago
I may be able to use 2d and 2b in this case, but not sure what my solution for my OPS env which currently uses a,b, and c. I may have to just ride a second VPC for a migration.
LeoGmad
LeoGmadover 7 years ago
Actually, I may have found the solution! I'll just temporarily go down to 2 subnets in OPS c, and d which should not conflict.
Dave Yu
Dave Yuover 7 years ago
I ran into this too. It has to do with how the module subdivides CIDR blocks. With the addition of us-west-2d, the module wants to create a new private subnet, but it tries to assign to it the CIDR block that’s already given to the public subnet for us-west-2a
LeoGmad
LeoGmadover 7 years ago
I see
Dave Yu
Dave Yuover 7 years ago
i couldn’t figure out a fix.. fortunately my env was in a state where i could delete and recreate all the subnets
Dave Yu
Dave Yuover 7 years ago
if you’re using terraform-aws-dynamic-subnets directly, it looks like you should be able to set availability_zones = ["us-west-2a", "us-west-2b", "us-west-2c"]
LeoGmad
LeoGmadover 7 years ago
Thanks I am using terraform-aws-dynamic-subnets
I'll give it a go.
ldlework
ldleworkover 7 years ago
What might be going wrong with my cloudposse/terraform-aws-ecs-codepipeline if I'm getting the following error on deploy:
ldlework
ldleworkover 7 years ago
I'm not sure what IAM role is relevant here (there so many O_O)
ldlework
ldleworkover 7 years ago
I'm not even sure what is being uploaded to s3?
ldlework
ldleworkover 7 years ago
Oh I deleted these lines from my buildspec:

      - printf '[{"name":"%s","imageUri":"%s"}]' "$CONTAINER_NAME" "$REPO_URI:$IMAGE_TAG" | tee imagedefinitions.json
artifacts:
  files: imagedefinitions.json
ldlework
ldleworkover 7 years ago
Perhaps these are important? Not sure why there would an IAM role error?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
@idlework are you using this in thee context of one of our other modules? e.g. the ecs-web-app?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
if not, I would look at how our other modules leverage it
ldlework
ldleworkover 7 years ago
@Erik Osterman (Cloud Posse) yeah I've been following right out this, https://github.com/cloudposse/terraform-aws-ecs-web-app/blob/master/main.tf
ldlework
ldleworkover 7 years ago
Adding those lines I removed seemed to make it work.
ldlework
ldleworkover 7 years ago
The last thing that's not working is that the Github Webhook doesn't seem to do anything.
ldlework
ldleworkover 7 years ago
I cut a release, Github says it sent the webhook, and nothing happens in CodePipeline even after 10 minutes.
ldlework
ldleworkover 7 years ago
I could probably post my HCL since I don't think it contains any secrets.
ldlework
ldleworkover 7 years ago
Maybe it only works if the commit is different
ldlework
ldleworkover 7 years ago
😞 still nothing
ldlework
ldleworkover 7 years ago
@Erik Osterman (Cloud Posse) do I have to click this even though I setup the terraform module with an oauth token with the right permissions? terraform was able to setup the webhook on my repo afterall, http://logos.ldlework.com/caps/2019-03-21-22-33-18.png
ldlework
ldleworkover 7 years ago
It would be nice if AWS listed the webhook events somewhere
ldlework
ldleworkover 7 years ago
I don't have a clue what could be wrong
ldlework
ldleworkover 7 years ago
When I list the webhooks via the AWS cli I see that there is a "authenticationConfiguration" section with a "SecretToken"
ldlework
ldleworkover 7 years ago
I don't see this secret token anywhere in the webhook on the github side
ldlework
ldleworkover 7 years ago
Oh that's probably the obscured "Secret"
ldlework
ldleworkover 7 years ago
I have no idea 😐️
ldlework
ldleworkover 7 years ago
The response on the github side says 200
ldlework
ldleworkover 7 years ago
SSL Verification is enabled
ldlework
ldleworkover 7 years ago
Even got a x-amzn-RequestId header in the response
ldlework
ldleworkover 7 years ago(edited)
Filters on the Webhook:

                "filters": [
                    {
                        "jsonPath": "$.action", 
                        "matchEquals": "published"
                    }
                ]
ldlework
ldleworkover 7 years ago
Websocket Payload:

{
  "action": "published",
  "release": {
ldlework
ldleworkover 7 years ago
Hmm it was the password.
ldlework
ldleworkover 7 years ago
That's worrying. Oh well.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
you mean the webhook secret?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
As I recall, the webhook secret on GitHub cannot be updated (rotated)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
You need to delete/recreate
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
....at least with Terraform
ldlework
ldleworkover 7 years ago
That might've been it.
ldlework
ldleworkover 7 years ago(edited)
When I try to destroy an ecs_codepipeline module by removing it from my HCL, I get:

Error: Error asking for user input: 1 error(s) occurred:

* module.ecs_codepipeline.module.github_webhooks.github_repository_webhook.default: configuration for module.ecs_codepipeline.module.github_webhooks.provider.github is not present; a provider configuration block is required for all operations
Igor
Igorover 7 years ago(edited)
I have an order of creation issue with an AutoScaling policy. I have one module that creates and ALB and Target Group and another that creates the AutoScaling policy, where I specify the target group resource_label. Terraform proceeds to create the AutoScaling policy using the Target Group, before the ALB->TargetGroup ALB listener rule is created, which causes an error. I tried a depends_on workaround by passing in the alb_forwarding_rule_id as a depends_on variable to the ASG module, but I assume I am still missing a step where I need to use this variable within the aws_autoscaling_policy resource block. Do I stick it in the count property?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
depends_on will not always work though
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
because TF does it already automatically
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
it’s just a hint
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
but if something could not be done b/c of AWS API or other things, it will not help
Igor
Igorover 7 years ago(edited)
It's a cross-module scenario, which I think isn't supported until 0.12
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
you can try --target which is not pretty
Igor
Igorover 7 years ago
It works fine if I run the apply twice. 🤷‍♂️
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
that’s another method 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
@Igor just out of curiosity, https://www.terraform.io/docs/providers/aws/r/autoscaling_policy.html#example-usage, instead of autoscaling_group_name = "${aws_autoscaling_group.bar.name}" can you try autoscaling_group_name = "${aws_autoscaling_group.bar.id}" and see what happens ?
ldlework
ldleworkover 7 years ago
@Igor you can resolve module dependencies by using the "tags" attribute on stuff I've found.
ldlework
ldleworkover 7 years ago
@Igor If you don't use a variable, it is optimized away
ldlework
ldleworkover 7 years ago
And doesn't affect ordering
ldlework
ldleworkover 7 years ago
So an easy way to use a dependent variable even though you don't need it, is to stick the variable in a Tag on the resource
ldlework
ldleworkover 7 years ago
This worked perfectly for me, for your exact use case which I also ran into
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
oh nice, this is a solution we did not think about, thanks
loren
lorenover 7 years ago
heh, tags and descriptions, had to do that before myself. any reference to force terraform to see the dependency when it resolves the config
Igor
Igorover 7 years ago
But then you're left with an unwanted tag 🤣
ldlework
ldleworkover 7 years ago
It's not the worst thing, because afterall the module the tagged asset comes from does depend on whatever asset you're interpolating into the tag.
Igor
Igorover 7 years ago
It actually makes sense, but I don't think ASG policy has tags
ldlework
ldleworkover 7 years ago
Yeah that's the only limitation I guess 🙂
ldlework
ldleworkover 7 years ago
I can't think of a way around it if there's no field to stick the dependency on
ldlework
ldleworkover 7 years ago(edited)
Something crazy like, using the dependency in a locals block in which you take just the first letter or something, and then use that local in the name of your ASG policy
ldlework
ldleworkover 7 years ago
lol
ldlework
ldleworkover 7 years ago
Igor
Igorover 7 years ago
I can use count
Igor
Igorover 7 years ago
Can't I?
Igor
Igorover 7 years ago
Just check that the depends_on is not empty
Igor
Igorover 7 years ago
useless check, but maybe it will work
Igor
Igorover 7 years ago
I'll give it a try and report back
loren
lorenover 7 years ago
policies have descriptions 🙂
ldlework
ldleworkover 7 years ago
hah
Igor
Igorover 7 years ago
Description not on the policy, count didn't work, ended up putting the first letter of the ID in the policy name 🤷‍♂️ Seems to work
ldlework
ldleworkover 7 years ago
lmfao
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Or you can create a list, put in there two items, one is real name, the other is something from the dependency
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Then get the first item
ldlework
ldleworkover 7 years ago
Every time I apply I get the following change:

module.frontend-ecs.module.ecs_codepipeline.aws_codepipeline.source_build_deploy: Modifying... (ID: us-west-1-qa-frontend-codepipeline)
  stage.0.action.0.configuration.%:          "4" => "5"
  stage.0.action.0.configuration.OAuthToken: "" => "<OAUTH-TOKEN>"

What is actually being changed here? Why is it always 4 -> 5? Why does it think the OAuth token is changing each time even though it is not?
ldlework
ldleworkover 7 years ago
@Andriy Knysh (Cloud Posse) XD
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
We had similar issues with oauth tokens before
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
I think terraform or AWS ignore them, forgot who
Igor
Igorover 7 years ago
@Andriy Knysh (Cloud Posse) Great suggestion, thanks
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
So you need to put the field into ignore changes, not good though if if ever changes
ldlework
ldleworkover 7 years ago
@Andriy Knysh (Cloud Posse) Can anything use a lifecycle block?
ldlework
ldleworkover 7 years ago
Also, I can't add the lifecycle block to a CloudPosse module right?
ldlework
ldleworkover 7 years ago
Since this is happening inside the ecs_codepipeline module
ldlework
ldleworkover 7 years ago
I wonder if anything on AWS is actually getting changed though. Or if it is just needlessly updating the Terraform state each time.
ldlework
ldleworkover 7 years ago
@Erik Osterman (Cloud Posse) any clue?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
lifecycles cannot be interpolated or passed between modules
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
maybe this is changing in 0.12? not sure
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago(edited)
in the end, I think it’s hard to escape needing a task runner for complicated setups
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago(edited)
(e.g. make or variant or terragrunt or astro)
ldlework
ldleworkover 7 years ago
How does that help in this case?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
you use terraform ... -target= to surgically target what you want to modify
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
in the order in which needs to happen that terraform was not able to figure out
ldlework
ldleworkover 7 years ago
In this case it just thinks that the github oauth token inside of the ecs-codepipeline is changing each time. So I'm not sure that it is a matter of ordering.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
oh sorry, i didn’t look closely enough at what you wanted to solve
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
in this case, yea, maybe not the best solutino
ldlework
ldleworkover 7 years ago
If the thing that's being updated is just the Terraform state, it might be no big deal to just let it update the state needlessly each time.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
really, the only option is to put the lifecycle in the module but then it applies to everyone
ldlework
ldleworkover 7 years ago
I can't really tell by the message what's changing though.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
jumping on call
ldlework
ldleworkover 7 years ago
Yeah.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
it’s not that it’s changing
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
it’s that terraform doesn’t know what the current value is
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
so it always changes
ldlework
ldleworkover 7 years ago
But does it have any consequences?
ldlework
ldleworkover 7 years ago
Like changing infrastructure etc? Seems not? Can't tell though.
Charlie Mathews
Charlie Mathewsover 7 years ago
Might @anyone know why the cloudposse/terraform-aws-ecs-container-definition doesn't include VolumesFrom or Links? I'm trying to figure out if I should tack those things onto the json output in a hacky way or submit a MR.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Please open a PR
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Idlework, no consequences
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
It just consider the token a secret
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
And does not know, or thinks it doesn’t know, the value
ldlework
ldleworkover 7 years ago
Makes sense
ldlework
ldleworkover 7 years ago
How do you all handle different environments? Right now I've got a single branch, with a bunch of HCL, and a Summon secrets.yml which contains the shared and environment specific Terraform variables. So by selecting an environment with Summon, different variables go into Terraform and I'm able to build different environments in different regions like that.
ldlework
ldleworkover 7 years ago
However, another way I've been thinking about it, is having a branch per environment so that the latest commit on that branch is the exact code/variables etc that deployed a given enviornment.
ldlework
ldleworkover 7 years ago
This allows different environments to actually have different/varying HCL files. So in a given environment you can actually change around the actual infrastructure and stuff. Then by merging changes from say, staging into production branch, you can move incremental infrastructure changes down the pipeline.
ldlework
ldleworkover 7 years ago
I wonder what you are all doing.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
The long-lived branches tend to be discourage by most companies we've worked with. typically, they only want master as the only long-lived branch.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
some use a hierchy like prod/us-west-2/vpc, staging/us-west-2/vpc and then a modules/ folder where they pull from
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
we use a 1-repo-per-account approach
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
<http://prod.cloudposse.co|prod.cloudposse.co>, <http://root.cloudposse.co|root.cloudposse.co>, etc
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
we treat our repos like a reflection of our AWS accounts
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
then we have our terraform-root-modules service catalog
ldlework
ldleworkover 7 years ago
@Erik Osterman (Cloud Posse) Do you know what I mean though? Let's say you wanted to make substantial changes of a major module by say, switching from self-hosted DB to Aurora Serverless (say). If you have just one branch that holds all the environments, then all environments must see this module change at the same time. VS, if you have a branching strategy, then you can make the major changes in a dev environment, then merge it into a staging environment, and then finally merge it into production/master. Is that not a problem you face / approach you see being used?
loren
lorenover 7 years ago
I've tried using branches, but the trouble is that they are not visible enough. People lose track of them too easily. So for them to work, we need another layer over them anyway that cycles through various actions (validate/plan/apply/etc). Separate directories or repos seem easier to grok
ldlework
ldleworkover 7 years ago
Huh, branches having low visibility. Not sure I understand that exactly, but thank you for your input.
ldlework
ldleworkover 7 years ago
I like the prospect of branching allowing for experimental environments to be spun up and destroyed creatively.
ldlework
ldleworkover 7 years ago
It appears this is the strategy that Hashicorp recommends for it's enterprise users.
loren
lorenover 7 years ago
My personal approach has been to write a module that abstracts the functionality, and then one or more separate repos that invoke the module for a specific implementation
ldlework
ldleworkover 7 years ago
Sure but consider a major refactor of that module.
ldlework
ldleworkover 7 years ago
I appreciate how parametic modules allows you to achieve multiple environments, that's what I'm doing right now.
ldlework
ldleworkover 7 years ago
But it seems like the repo would overall be more simple with less going on in a given branch.
loren
lorenover 7 years ago
Tags as versions, each implementation is isolated from changes to the module
ldlework
ldleworkover 7 years ago
This requires you to use external modules, no?
loren
lorenover 7 years ago
Define "external"... You can keep the module in the same repo if you like
ldlework
ldleworkover 7 years ago
So now you have the maintanience overhead of a new repo * how many modules.
ldlework
ldleworkover 7 years ago
How do you tag modules that live in the same repo?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
we use one repo for our service catalog
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
and one repo to define the invocation
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
since we don't use the same repo, we can surgically version pin every single module
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
and never force an upgrade we didn't want
loren
lorenover 7 years ago
It's easier to pin if they are separate repos, for sure
loren
lorenover 7 years ago
Not impossible if they are in the same repo, but some mental gymnastics are needed
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
it's the only way it can work for us since we build infra for multiple organizations
ldlework
ldleworkover 7 years ago
It makes perfect sense to me that you'd externalize modules you're reusing across organizations sure.
ldlework
ldleworkover 7 years ago
I'm just one person at a tiny startup right now though.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
I know monorepos have gained in popularity over recent years and major companies have come out in favor of them
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
i think they can work well inside of an organization that builds 99% of their stuff internally
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
but open source is necessarily democratized into a number of smaller repos (think NPM, ruby, perl, go, and ....... terraform modules)
ldlework
ldleworkover 7 years ago
It's not really a mono-repo in the sense that are multiple disjoint substantial projects in a single repo.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
it sounds like you're going to have modules within your repo that have different SDLCs
ldlework
ldleworkover 7 years ago
No that's the point, they don't really have different SDLCs.
ldlework
ldleworkover 7 years ago
They're not really indepdent abstractions we intend to reuse in multiple places.
ldlework
ldleworkover 7 years ago
Most of my HCL code is just invocations of high-level CloudPosse modules.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago

Let's say you wanted to make substantial changes of a major module by say, switching from self-hosted DB to Aurora Serverless (say). If you have just one branch that holds all the environments, then all environments must see this module change at the same time.
ldlework
ldleworkover 7 years ago
What has different SDLCs are the overall architecture of the different environments.
ldlework
ldleworkover 7 years ago
Yes, that is in terms of the HCL defining our architecture, but I don't really have a reusable module which defines arbitrary DBs
loren
lorenover 7 years ago
You can try tf workspaces also, that might work for your use case
ldlework
ldleworkover 7 years ago
I guess I was just waiting to hear "Yeah branches sound like they'll work for your use-case too"
ldlework
ldleworkover 7 years ago
Thank you!
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
there's no right/wrong, only different opinions 😃
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
you can achieve what you want to do with branches/workspaces
loren
lorenover 7 years ago
My experience has just between that I always, eventually, need to separate configuration from logic
ldlework
ldleworkover 7 years ago
I think you could even combine approaches.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
what i don't like about workspaces is the assume a shared statebucket
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
we never share a statebucket across stages or accounts
ldlework
ldleworkover 7 years ago
I don't intend on using Workspaces. I've read enough about them to avoid them for now.
ldlework
ldleworkover 7 years ago
I almost went full Terragrunt, but I'm trying to thread the needle with my time and workload 😅
ldlework
ldleworkover 7 years ago
"Why didn't you use k8s or EKS?"
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
haha, ok, so yea that's one approach that appeals to many. just another camp 😃
ldlework
ldleworkover 7 years ago
"Uhh, because I can fit CodePipeline in my head" lol
ldlework
ldleworkover 7 years ago(edited)
@Erik Osterman (Cloud Posse) So basically your suggestion to me is to have two repos: 1 repo that contains all the modules. 1 repo which contains the environment-specific calls into those modules to build environment-specific infrastructure. Each environment's HCL that uses the modules can pin to different versions of those modules, even though all the modules are in one repo, since each module call has a different source parameter. Is this close?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
nailed it
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
😃
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago(edited)
so technically, it can all be in one repo and still use source pinning to the git repo, just my head hurts doing it that way
loren
lorenover 7 years ago
Mental gymnastics
loren
lorenover 7 years ago
Also, Chinese finger traps
ldlework
ldleworkover 7 years ago
@Erik Osterman (Cloud Posse) so I guess my only concern is moving enhancements made in one environment's HCL to another environment, and the next. It all comes down to how much per-environment HCL there is to arrange the modules in order to construct the infrastructure. To put things at their extremes to make the point; in order to have essentially zero manual maintanece of moving changes between environments, the modules would have to be essentially so high-level that they simply implement the entire environment. Rather than the environment HCL having room to vary in their composure of slightly-lower level modules and being able to express different approaches in infrastructure architecture.
ldlework
ldleworkover 7 years ago
little wordy there, i just hope to be understood (so that I can in turn understand too)
ldlework
ldleworkover 7 years ago
This is contrasted to the "per-environment HCL" being in branches and letting Git do the work
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago(edited)
yes, it comes at a trade off of being more effort to manage changes
ldlework
ldleworkover 7 years ago
(whether or not you use remote modules)
ldlework
ldleworkover 7 years ago
Basically my point is, you'll always have to manually "merge" across environments if the per-environment HCL is in a single branch, regardless of module use.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago

, the modules would have to be essentially so high-level that they simply implement the entire environment
ldlework
ldleworkover 7 years ago
(so maybe if you combined approaches you'd achieve the theoretical minimum maintainence)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
this = "terraform root module"
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
the root invocatin of a module is a module
ldlework
ldleworkover 7 years ago
Oh you're saying there is literally no per-environment HCL?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
yep
ldlework
ldleworkover 7 years ago
Per-environment HCL is a tagged "highest-level-module" ?
ldlework
ldleworkover 7 years ago
🤔
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
ldlework
ldleworkover 7 years ago
lol why you link that?
erik
erikover 7 years ago
ldlework
ldleworkover 7 years ago
lol no I know
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
to define what a "Root module" is from the canonical source
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
so just saying the root is a module. treat it like one. invoke it everywhere you need that capability
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
use .tfvars to tailor the settings
loren
lorenover 7 years ago
Module in module inception
ldlework
ldleworkover 7 years ago
Yeah but you might have variance across environment in the implementation of that module.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
this is why we like the terraform init -from-module=...
ldlework
ldleworkover 7 years ago
And so what you're saying is that, you achieve that, by having a separate repo, which simply refers to a tagged implementation of that module.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
yup
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
so you can invoke an environment 20 times
ldlework
ldleworkover 7 years ago
Clever.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
just call terraofrm init -from-module 20x
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
(in different folders)
loren
lorenover 7 years ago
And then you have terragrunt 😆
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
haha
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
so i gotta run, but this is why we came up with tfenv
ldlework
ldleworkover 7 years ago
In the configuration repo, where each environment simply invokes a tag of the remote root-module, the amount of code you have per-environment is so little there's no reason to have different branches.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
it's the unwrapper for using terraform init -from-module=...
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
you can just specify environment variables that terraform init already supports
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
and it will clone the remote module
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
and initialize it
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
here's my post on it 😃
loren
lorenover 7 years ago
Also, highly recommend dependabot or something similar for bumping version tags
ldlework
ldleworkover 7 years ago
The root modules don't really seem like all encompassing modules which compose many (say) AWS modules to build a whole infrastructure.
ldlework
ldleworkover 7 years ago
Like one just sets up an ECR registry, etc. A real service is going to require a bunch of these "root modules" composed together to make it happen.
ldlework
ldleworkover 7 years ago
So any "environment repo" which is supposed to just call out to a single tagged remote root module to build the entire environment doesn't seem like it would work with these root modules. Like, the HCL in the environment repo is going to be pretty substantial in order to tie these together. And then you have the manual merging problem again.
loren
lorenover 7 years ago
Modules all the way down, that's where inception comes in
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago(edited)
you, and every company that want to use the strategy, should create their own catalog of root modules (which is logic), and then invoke them from diff repos (config)
loren
lorenover 7 years ago
Discussion reminds me of this thread, https://github.com/gruntwork-io/terragrunt/issues/627
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
so you put in there as many modules of diff types as you want to have in ALL your environments
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
just diff approaches
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
but I agree with @loren, it comes to separation of logic from config
ldlework
ldleworkover 7 years ago
Hmm, do I understand that you're saying that the root modules are really "root" with regards to individual substantial LAYERS of your environment's architecture?
ldlework
ldleworkover 7 years ago
The environment isn't defined by calling a SINGLE module, but rather, it builds the environment out of a few major "layer-level root" modules?
ldlework
ldleworkover 7 years ago
Which you can pin to different tags, etc?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
consider them as a collection/catalog of everything you need in all your environments
ldlework
ldleworkover 7 years ago
Yes, but there's different conceptual abstraction levels of "everything you need in all your environments"
ldlework
ldleworkover 7 years ago
You could imagine a SINGLE remote module being the sole invocation that defines everything for an environment.
ldlework
ldleworkover 7 years ago
Or you could imagine a handful of remote modules which define the major layers of the architecture. VPC/Networking, ECS/ALB for each server, ECR, CloudWatch etc
ldlework
ldleworkover 7 years ago
I dunno.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
that’s your call how you configure them. We use the second approach
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
The thing is you never ever want one module that does everything
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
yep
ldlework
ldleworkover 7 years ago
@Erik Osterman (Cloud Posse) see the thread above
ldlework
ldleworkover 7 years ago
I was only thinking about one module because we were aiming at minimizing "cross environment merging".
ldlework
ldleworkover 7 years ago
One module, while silly, would allow each environment to simply pin to the exact implementation of the God Module relevant to that environment.
ldlework
ldleworkover 7 years ago
That one module could in-turn compose Layer Modules
ldlework
ldleworkover 7 years ago
Which in-turn compose Component Modules
ldlework
ldleworkover 7 years ago
w/e
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
A) terraform module composition has a lot of problems as you discovered
ldlework
ldleworkover 7 years ago
But yeah overall I think I'm clearer on the approach you were all suggesting.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
B) mono modules have huge state. We ran into this 2 years ago where a module was so big it took 20 minutes to run a plan
ldlework
ldleworkover 7 years ago
It's just another level similar to the way your "root" (I read "layer/stack") modules compose component modules.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
C) huge state means huge blast radius. You cannot easily roll out small incremental changes
ldlework
ldleworkover 7 years ago
You could.
ldlework
ldleworkover 7 years ago
You would change the god module to compose the layer modules differently.
ldlework
ldleworkover 7 years ago
Then you would update a given environment to the new pin of the god module
ldlework
ldleworkover 7 years ago
Exactly how layer modules work with component modules.
ldlework
ldleworkover 7 years ago
It's just another level of abstraction.
ldlework
ldleworkover 7 years ago
I'm not saying it's useful.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
no one God Module, we use separate folders for each service or a group of services (if that makes sense), so there is no one big single state
ldlework
ldleworkover 7 years ago
(but just from a thought experiment of how to absolutely minimize the cross-environment maintanence of merging the per-environment HCL)
ldlework
ldleworkover 7 years ago
(if one is to avoid branching strategy)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
there is no merging as in git branches, you just make changes to the root modules and apply them in each env separately as needed
ldlework
ldleworkover 7 years ago
@Andriy Knysh (Cloud Posse) there is
ldlework
ldleworkover 7 years ago
If each environment needs HCL to describe how it composes the high-level layer modules to describe the whole infrastructure for the environment
ldlework
ldleworkover 7 years ago
then you need to manually adopt those changes from one environment to the next, since they're all in a single branch
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Sorry, on my phone and don’t have the keyboard speed to keep up with this
ldlework
ldleworkover 7 years ago
if each environment simple makes one module { } call, then the amount you have to manually copy-paste over as you move enhancements/changes down the environment pipeline is the smallest amount possible
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
That’s true but that’s not what blast radius refers to
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
we don’t; apply everything at once
ldlework
ldleworkover 7 years ago
No, I'm just explaining why we're talking about this in the first place
ldlework
ldleworkover 7 years ago
to @Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
just the modules we need to change
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
@Andriy Knysh (Cloud Posse) he is doing a thought experiment
ldlework
ldleworkover 7 years ago
@Andriy Knysh (Cloud Posse) right
ldlework
ldleworkover 7 years ago
you typically apply just major layers
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
He is not asking what we do :-)
loren
lorenover 7 years ago
-target is hugely powerful
ldlework
ldleworkover 7 years ago
yeah
ldlework
ldleworkover 7 years ago
I'm going to refactor my stuff into layers like this tonight
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
He wants to understand why he should/shouldn’t do what he is proposing
ldlework
ldleworkover 7 years ago
VPC/Networking, ECS/ALB/CodePipeline for a given Container, Database, Caching
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
i get it 🙂
ldlework
ldleworkover 7 years ago
Make "root modules" for each of those
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
just saying that don’t apply everything and don’t merge anything - just make changes somewhere and apply step by step one by one
ldlework
ldleworkover 7 years ago
Does it make sense to have a root module covering CodePipeline/ECS/ALB-target-and-listeners for a given container?
ldlework
ldleworkover 7 years ago
Like everything needed to deploy a given container-layer of our overall product infrastructure
ldlework
ldleworkover 7 years ago
Pass in the VPC/ALB info, etc
ldlework
ldleworkover 7 years ago
Have it provision literally everything needed to build and run that container
ldlework
ldleworkover 7 years ago
Have a module we can reuse for that for each container we have
ldlework
ldleworkover 7 years ago
Or should there be a root module for each container?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
So if you have one module that tried to do all of that (beyond terraform not supporting it due to count of cannot be computed errors) the problem is the amount of infrastructure it touches is massive. So if you just wanted to change an auto scale group max size, you will incidentally be risking changes to every other part of the infrastructure because everything is in one module.
ldlework
ldleworkover 7 years ago
Sure, but it would be all the infrastructure having to do with a logical component of the overall microservice architecture.
ldlework
ldleworkover 7 years ago
If anything related to that microserivice gets boofed, the whole microservice is boofed anyway.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
All I can say is you have to experience it first hand :-)
ldlework
ldleworkover 7 years ago
That top-level module could of course be compose of other modules for doing just the ECS, just the CodePipeline, just the ALB
ldlework
ldleworkover 7 years ago
infact, those are already written
ldlework
ldleworkover 7 years ago
they're yours
loren
lorenover 7 years ago
It's hugely composable and there is no one right answer
ldlework
ldleworkover 7 years ago
I run two container-layers in our microservice architecture with this module: https://gist.github.com/dustinlacewell/c89421595a20577f1394251e99d51dd8
loren
lorenover 7 years ago
Just tradeoffs
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
exactly
ldlework
ldleworkover 7 years ago
It does ECR/CloudWatch/Container/Task/ALB-target-listener/CodePipeline
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
we, for example, have big and small root modules
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
ldlework
ldleworkover 7 years ago
lmao
ldlework
ldleworkover 7 years ago
wish i found that first link three days ago
ldlework
ldleworkover 7 years ago
i literally just built all that, but not as good, and there's more there like dns
loren
lorenover 7 years ago
Isn't it Friday night? Here we all are on slack 😋
ldlework
ldleworkover 7 years ago
I work for a broke startup so I work all the time for the foreseeable future
ldlework
ldleworkover 7 years ago
lol
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
let’s get some beer 🙂
ldlework
ldleworkover 7 years ago
OK
ldlework
ldleworkover 7 years ago
🙂
loren
lorenover 7 years ago
🍻
ldlework
ldleworkover 7 years ago
Here's a question I haven't been able to answer for myself yet. Is a single ECR registry capable of holding multiple images? Or is a single ECR only for a single Docker image?
ldlework
ldleworkover 7 years ago
I know that a Docker image can have multiple tags. I don't mean that. I mean can you store multiple images in a single ECR. Do I need an ECR for each of my containers, or just one for the region?
ldlework
ldleworkover 7 years ago
Cuz, Docker Registry obviously does multiple images, so I'm confused on this point.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
one ECR can have just one image
ldlework
ldleworkover 7 years ago
Got it thank you
ldlework
ldleworkover 7 years ago
Make sense since the ECR name is set as the image name. Thanks for confirming.
loren
lorenover 7 years ago
Gotta sign off, catch some zzzzzzs. Good discussion! Night folks!
ldlework
ldleworkover 7 years ago
o/
ldlework
ldleworkover 7 years ago
@Andriy Knysh (Cloud Posse) I must admit, I poured a cup of coffee instead.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
haha, maybe tomorrow, good night
ldlework
ldleworkover 7 years ago
Another thing I worry about, is that I have a container to containerize soon that hosts multiple ports.
ldlework
ldleworkover 7 years ago
I think I will need to place this Fargate container into multiple target-groups, one for each of the ports.
ldlework
ldleworkover 7 years ago
But the posse module for tasks only takes a single ARN
ldlework
ldleworkover 7 years ago
Oh maybe you have just one target group for the container, but you add multiple listeners.
ldlework
ldleworkover 7 years ago
That must be it.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
also, if you’re using our terraform-aws-ecs-web-app module, you might want to fork it for your needs since it’s rather opinionated. it was designed to show how to use all of our other ecs modules together.
ldlework
ldleworkover 7 years ago
I'm now pondering whether each top-level layer module should have it's own remote state. And whether the environment HCL should pass values to layers by utilizing the remote_state data source. That's basically essential if you're going to use -target right? Like how else does the auroradb layer get the right private DNS zone id? It has a variable on it's module for that. But how does the environment HCL pass it to it, if the private zone isn't being provisioned due to only the auroradb layer being targetted? It has to use the remote data source to access the other layer module's outputs right?
ldlework
ldleworkover 7 years ago
@Erik Osterman (Cloud Posse) thoughts on that?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
we do use separate states for each root module
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
and we use both 1) remote state 2) data source lookup to communicate values b/w them
ldlework
ldleworkover 7 years ago
I see.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
but again, that depends
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
we also use 3) write values from a module into SSM param store or AWS Secret Manager, then read those values from other modules
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
ldlework
ldleworkover 7 years ago
Never even heard of that before
ldlework
ldleworkover 7 years ago
@Andriy Knysh (Cloud Posse) I'm trying to imagine how all the state management would look on the environment/configuration repo that calls the various root modules.
ldlework
ldleworkover 7 years ago
Right now, I just have a little folder with a tiny bit of HCL that sets up an S3 bucket and DynamoDB table, and then those details are used when I apply my actual HCL
ldlework
ldleworkover 7 years ago
Would I have to have little side modules on the environment side for each root module the enviornment uses?
ldlework
ldleworkover 7 years ago
I bet you guys have some crazy complex infrastructures
ldlework
ldleworkover 7 years ago
I love all your abstractions, it's great.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
so, this is opinionated, but here’s what we have/use (in simple terms, regardless of complex infrastructure or not):
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
1. TF modules https://github.com/cloudposse?utf8=%E2%9C%93&q=terraform-&type=&language= - this is logic and also just the definition w/o specifying the invocation
ldlework
ldleworkover 7 years ago
I've been calling those Asset/Component modules. Individual AWS assets. Got it.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
2. Then we describe the invocation of those modules (which ones to use deppens on your use case) in the catalog of module invocation https://github.com/cloudposse/terraform-root-modules
ldlework
ldleworkover 7 years ago
I've been calling those Layer Modules. They define major aspects/facets of an environments overall infrastructure. Got it.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
those invocations are identity-less, they don’t care where and how they are deployed, this is just logic
ldlework
ldleworkover 7 years ago
Right, like you said some are big, some are small, but they compose the lower-level Asset modules.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
3. Then for each environment (prod, staging, dev, test), we have a GitHub repo with a Dockerfile that does two things:
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago(edited)
1) Copies the invocation of the root modules from the catalog into the container (geodesic in our case) - this is logic
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
2) Defines ENV vars to configure the modules (the ENV vars could come from many diff places including Dockerfile, SSM param store, HashiCorp Vault, direnv, etc.) - this is config
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
then we start the container and it has all the code and config required to run a particular environment
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
so only the final container has identity and knows what it will be deploying and where and how (environment, AWS region, AWS account. etc.)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
the final container could be run from your computer or from a CI/CD system (e.g #codefresh )
ldlework
ldleworkover 7 years ago
OK I got confused along the way
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
ldlework
ldleworkover 7 years ago
You said "Copies the invocation of the root modules from the catalog into the container". I thought the "catalog" was a repo with all the root/layer modules inside of it. And that this environment repo had some HCL that called root/layer modules out of the catalog to compose an entire environment architecture.
ldlework
ldleworkover 7 years ago
Where did I go wrong? Is the catalog something different than the repo containing the root/layer modules?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
each environment (prod, staging, dev) copies the entire catalog, OR only those modules that are needed for the particular environment
ldlework
ldleworkover 7 years ago
Sure, and it also has some HCL of its own for calling the root/layer modules from the catalog right?
ldlework
ldleworkover 7 years ago
The environments.
ldlework
ldleworkover 7 years ago
looks at the links.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
it can have (if it’s very specific to the environment AND is not re-used in other environments)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
but usually you want to re-use modules across diff environment (with different params of cause)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago(edited)
that’s why we put those module invocations in the common catalog (root modules as we call it)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
so in other words, the catalog is a collection of reusable module invocations (not definitions) that you would reuse in your environments. But any of those environments could have some specific module definitions and invocations nott from the catalog (if that makes sense)
ldlework
ldleworkover 7 years ago
Yes, reuse modules, but in an experimental environment, you might be trying to save costs and switch from one DB to another DB, and so a different root/layer module would be called in that environment's HCL right?
ldlework
ldleworkover 7 years ago
Like, you probably use a half-dozen or so root/layer modules to create a moderate environment for a reasonable multi-layered service right?
ldlework
ldleworkover 7 years ago
So there's got to be something in the environment, HCL, which calls all those modules.
ldlework
ldleworkover 7 years ago
Remember, there's no God Module
ldlework
ldleworkover 7 years ago
Like an environment doesn't invoke just one God Module from the catalogue right? Because the catalog contains modules which only cover a single "layer" of the environment. ECS, Networking, Database, etc right?
ldlework
ldleworkover 7 years ago
So each environment must have a bit of HCL which are the invocations of the various layer modules which define the major sections of the infrastructure.
ldlework
ldleworkover 7 years ago
lol I'm gonna make sense of this eventually
ldlework
ldleworkover 7 years ago
I see in your Dockerfile you're copying the various root modules which create various layers of the environment. account-dns, acm, cloudtrail, etc
ldlework
ldleworkover 7 years ago
Which reaffirms my idea that root modules only cover a facet of the environment, so you need multiple root modules to define an environment.
ldlework
ldleworkover 7 years ago
So where is the per-environment HCL that invokes those modules? in conf/ perhaps.
ldlework
ldleworkover 7 years ago
There's nothing in conf/!
ldlework
ldleworkover 7 years ago
Are you mounting the environment's HCL into the container via volume at conf/ ?
ldlework
ldleworkover 7 years ago
The stuff that calls the root modules?
ldlework
ldleworkover 7 years ago
OHH you're literally CD'ing into the root modules
ldlework
ldleworkover 7 years ago
and running terraform apply from inside the root modules themselves
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
If we need something special or new, we add it to the catalog and then can copy to the container of that environment
ldlework
ldleworkover 7 years ago
nothing calls the root modules
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
If we need a smaller db instance for dev, all those settings are config and provided from ENV vars
ldlework
ldleworkover 7 years ago
So let's say you needed two "instances" of what a root module provides. Like say it provides all the ECS and related infrastructure for running a microservice web-container.
ldlework
ldleworkover 7 years ago
Do you cd into the same root module and call it with different args?
ldlework
ldleworkover 7 years ago
Or would you end up making two root modules, one for each container in the microservice architecture?
ldlework
ldleworkover 7 years ago
I'm guessing the latter?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
In our case, we would create a “big” root module combining the smaller ones, and put it into the catalog
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Then configure it from ENV vars from the container
ldlework
ldleworkover 7 years ago
Let me give an example.
ldlework
ldleworkover 7 years ago
Let's say you have a single webservice container that does server side rendering. Like a Django application serving it's own static media. You have an Asset module like the CloudPosse ecs-codepipeline module, and other's like the Task and Container module. You create a Root Module which ties these together so you can deploy your Django application and everything it needs as a single Root Module. You might also have Root Module for the VPC/Networking for that application. So, you have an environment dockerfile, and you start by CD'ing into the VPC Root Module, and you deploy it. Then you CD into the Django App's Root Module, and you deploy that too. OK so now you have a Django web-service deployed.
ldlework
ldleworkover 7 years ago
Now lets say your dev team refactors the app so that Django only serves the backend, and you have an Nginx container serving the static assets for the frontend. So now you need another ECS-CodePipeline module, Task and Container modules. Just like for the Django container.
ldlework
ldleworkover 7 years ago
Do you create another Root Module for the Nginx Frontend container, which calls the CodePipeline, Task and Container modules again?
ldlework
ldleworkover 7 years ago
So then you'd have 3 Root Modules you'd be able to deploy independently? How would you resolve the duplication?
ldlework
ldleworkover 7 years ago
(the calling of ecs-codepipeline, task, and container modules the same way in two root modules representing the two container services in your environment)
ldlework
ldleworkover 7 years ago
BTW thanks for all the charity.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
so yea, good example 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
we mostly use k8s for that kind of things
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
and did not have ECS infra such as that
ldlework
ldleworkover 7 years ago
Sure but you can imagine someone using your modules for that (I am 😅)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
but i guess we would create one module with all that stuff but with many params to be able to parameterize all the things and even switch some things on/off
ldlework
ldleworkover 7 years ago
Sure, a root module which combines ecs-codepipeline, task and container, etc to deploy a single containerized microservice
ldlework
ldleworkover 7 years ago
But what if you needed two? Two different containers but deployed essentially the same way.
ldlework
ldleworkover 7 years ago
You already have on Root Module for deploying the unified container.
ldlework
ldleworkover 7 years ago
But now you need two, so do you just copy paste the Root Module for the second container, as it is basically exactly the same minus some port mappings and source repo, and stuff?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
that’s config
ldlework
ldleworkover 7 years ago
Or would you create a high-level NON-root-module which expressed how to deploy a ecs-codepipeline, task and container together
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
can be parmeterized
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
so yes, one module
ldlework
ldleworkover 7 years ago
And then two root modules which just called the non-root-module with different settings?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
ldlework
ldleworkover 7 years ago
one non-root module though?
ldlework
ldleworkover 7 years ago
But like
ldlework
ldleworkover 7 years ago
OK so you have one generalized root module for deploying ecs services great.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
those are just names (root or not) 🙂
ldlework
ldleworkover 7 years ago
So where are the different configs for the different containers?
ldlework
ldleworkover 7 years ago
Like in your prod.posse.com example.
ldlework
ldleworkover 7 years ago
Because it seems the only config that's available is environmental config
ldlework
ldleworkover 7 years ago
Like the difference between dev and prod
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
if you want to re-use the module (which can combine smaller modules), parametirize it and out into the catalog
ldlework
ldleworkover 7 years ago
But where would the difference between frontend and backend be, for calling the ecs root module twice?
ldlework
ldleworkover 7 years ago
The catalog is the thing that holds root modules right?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
it holds modules, big and small, that can be re-used
ldlework
ldleworkover 7 years ago
Or is the catalog the repo of modules that your root modules use?
ldlework
ldleworkover 7 years ago
OK, so you already have the ECS module right?
ldlework
ldleworkover 7 years ago
It is a general parameterized module that can deploy any ECS service.
ldlework
ldleworkover 7 years ago
You want to call it twice, with different parameters.
ldlework
ldleworkover 7 years ago
Where do you store the different paremters for calling that module twice?
ldlework
ldleworkover 7 years ago
Because the only environment variables that change between your environment repos is environment specific stuff.
ldlework
ldleworkover 7 years ago
Like between dev and prod.
ldlework
ldleworkover 7 years ago
So you're in dev, you copy in the ECS module into the Docker image
ldlework
ldleworkover 7 years ago
You're loaded into the geodesic shell
ldlework
ldleworkover 7 years ago
You're ready to deploy root modules by CD'ing into them and running terraform apply
ldlework
ldleworkover 7 years ago
You CD into the ecs root module
ldlework
ldleworkover 7 years ago
How do you go about deploying the frontend
ldlework
ldleworkover 7 years ago
And then how do you go about deploying the backend?
ldlework
ldleworkover 7 years ago
They both depend on this generalized ECS root module we copied into the prod Docker image.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
well, make a “bigger” module with frontend and backend in it, and put into the catalog into diff folder
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
that all depends on many diff factors
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
we are talking about ideas and patterns here
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
how do you implement it, your choice
ldlework
ldleworkover 7 years ago
Sure, I'm totally just fishing to understand how the posse does it 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
yes I get it 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
and thanks for those questions
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
no solution is perfect
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
some of your use-cases could be not covered, or covered better by some other solutions
ldlework
ldleworkover 7 years ago
Since root modules are literally terraform apply contexts... how do they boot strap their remote state?
ldlework
ldleworkover 7 years ago
You CD into one and terrafrom apply, didn't I need to bootstrap its remote state first somehow?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
it uses the S3 backend provisioned separately before
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
the backend is configured from ENV vars in the container
ldlework
ldleworkover 7 years ago
Do you guys provision like just one backend bucket, and then use varying names to store the various states in that bucket in different files? So you only have to provision the backend once?
ldlework
ldleworkover 7 years ago
looks
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
one backend per environment (AWS account)
ldlework
ldleworkover 7 years ago
oh I see OK
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
then each projects is in separate folder in the repo and in the backend S3 bucket
ldlework
ldleworkover 7 years ago
by project you mean "root module where we run terraform apply" right
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
ldlework
ldleworkover 7 years ago
right
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
BTW, take a look at the docs
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
little bit outdated, but can give you some ideas
ldlework
ldleworkover 7 years ago
I really appreciate all the advice. It was way above and beyond. Thanks for sticking through all that. Really!
ldlework
ldleworkover 7 years ago
Wow atlantis looks cool
ldlework
ldleworkover 7 years ago
phew there is still so much to learn
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Let’s talk about atlantis later :)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Just in case you want to follow, #atlantis
ldlework
ldleworkover 7 years ago
Oh dang, you can't refer to modules in a remote git repo that are not at the root?
ldlework
ldleworkover 7 years ago
rough
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
try this:
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago

terraform init -from-module=git::<https://github.com/cloudposse/terraform-root-modules.git//aws/users?ref=tags/0.53.3>"
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago(edited)
that initializes the aws/users module
ldlework
ldleworkover 7 years ago
I mean in a module { source = } block
ldlework
ldleworkover 7 years ago
Or maybe that double slash works there too?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
yes, that works in a source block too
ldlework
ldleworkover 7 years ago
Nice
xluffy
xluffyover 7 years ago
Hey, I have a string variable 10.20.30.111, I want to get last element of this string, expecting output is 111, I can use


value = "${element(split(".", var.private_ip), length(split(".", var.private_ip)) - 1 )}"


But too complex, any suggestion?
mmuehlberger
mmuehlbergerover 7 years ago
You can also just do element(split(".", var.private_ip), 3) since IP addresses have a predefined syntax (always 4 blocks).
Nikola Velkovski
Nikola Velkovskiover 7 years ago(edited)
usually these kind of errors are network/internet related
albttx
albttxover 7 years ago
Hello, just posted this https://github.com/cloudposse/terraform-aws-acm-request-certificate/issues/16
just wan’t to be sure it’s an error from the module… any idea ?
Bharat
Bharatover 7 years ago
Terraform is marking task-def as inactive, when ever we update the task-def. We need those old task-def’s to do roll-back. We are deploying ECS-Service as part of CI. Any work around on how to retain the older versions of task-def’s ?
deftunix
deftunixover 7 years ago
hi everyone
deftunix
deftunixover 7 years ago
I am using terraform to provision consul cluster on aws using the https://github.com/hashicorp/terraform-aws-consul module. do you have any
deftunix
deftunixover 7 years ago
suggestion to terminate instance one by one when launch configuration change?
Igor
Igorover 7 years ago
I am suddenly having trouble passing the route 53 zone nameservers as records to the NS record
Igor
Igorover 7 years ago
The error I get is records.0 must be a single value, not a list
Igor
Igorover 7 years ago
I am passing in ["${aws_route53_zone.hostedzone.*.name_servers}"]
antonbabenko
antonbabenkoover 7 years ago
Try: ["${flatten(aws_route53_zone.hostedzone.*.name_servers)}"]
Igor
Igorover 7 years ago
Thanks, I just stumbled on that
antonbabenko
antonbabenkoover 7 years ago
Or actually try: ["${aws_route53_zone.hostedzone.name_servers[0]}"]
Igor
Igorover 7 years ago
flatten worked for me
Igor
Igorover 7 years ago
element(..., 0) didn't ("element() may only be used with flat lists")
antonbabenko
antonbabenkoover 7 years ago
right, combine with flatten, if you need to get just one element
Igor
Igorover 7 years ago
Thanks
tallu
talluover 7 years ago
given values="m1.xlarge,c4.xlarge,c3.xlarge,c5.xlarge,t2.xlarge,r3.xlarge" can I use jsonencode or something similar to get the following
{
      "InstanceType": "m1.xlarge"
    },
    {
      "InstanceType": "c4.xlarge"
    },
    {
      "InstanceType": "c3.xlarge"
    },
    {
      "InstanceType": "c5.xlarge"
    },
    {
      "InstanceType": "t2.xlarge"
    },
    {
      "InstanceType": "r3.xlarge"
    }
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
or use null_data_source to construct anything you want, e.g. https://github.com/cloudposse/terraform-aws-ec2-autoscale-group/blob/master/main.tf#L63
tallu
talluover 7 years ago
thanks I will try it out
tallu
talluover 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
no
tallu
talluover 7 years ago
> formatlist("Hello, %s!", ["Valentina", "Ander", "Olivia", "Sam"])
parse error at 1:28: expected expression but found "["
tallu
talluover 7 years ago
in terraform console
tallu
talluover 7 years ago(edited)
nevermind formatlist("Hello, %s!", list("Valentina", "Ander", "Olivia", "Sam")) worked
ldlework
ldleworkover 7 years ago
What would be the cause of this failure when deploying ecs-codepipeline


* module.backend.module.ecs_codepipeline.module.github_webhooks.github_repository_webhook.default: 1 error(s) occurred:

• github_repository_webhook.default: POST <https://api.github.com/repos/dustinlacewell/roboduels-frontend/hooks>: 404 Not Found []
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
did you create a correct GitHub token?
ldlework
ldleworkover 7 years ago
Hmm...
ldlework
ldleworkover 7 years ago
Oh I know what happened. Thanks @Andriy Knysh (Cloud Posse) lol.
ldlework
ldleworkover 7 years ago
OK, does anyone ahve an idea about this one? I'm chalking this up to my inexperience with the interpolation details:

* output.cache_hostname: element: element() may only be used with flat lists, this list contains elements of type map in:

${element(module.elasticache.nodes, 0)}
* module.elasticache.aws_route53_record.cache: 1 error(s) occurred:

* module.elasticache.aws_route53_record.cache: At column 3, line 1: element: argument 1 should be type list, got type string in:

${element(aws_elasticache_cluster.cache.cache_nodes.*.address, 0)}
ldlework
ldleworkover 7 years ago
oh that's two errors
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
aws_elasticache_cluster.cache.cache_nodes is a map
ldlework
ldleworkover 7 years ago
isn't it a list of maps?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago(edited)
lookup(element(aws_elasticache_cluster.cache.cache_nodes, 0), "address") - try something like this
ldlework
ldleworkover 7 years ago
Interesting.
ldlework
ldleworkover 7 years ago
ew I think I have to escape the inner quotes
ldlework
ldleworkover 7 years ago
does a raindance for 0.12
ldlework
ldleworkover 7 years ago
escaping doesn't work either...
ldlework
ldleworkover 7 years ago
  value = "${lookup(element(module.elasticache.nodes, 0), \"address\")}"
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
what are you escaping?
ldlework
ldleworkover 7 years ago
Error: Error loading /home/ldlework/roboduels/infra/stages/qa/outputs.tf: Error reading config for output cache_hostname: parse error at 1:48: expected expression but found invalid sequence "\\"
loren
lorenover 7 years ago
a miracle of hcl is that you do not need to escape inner quotes like that 🙂
ldlework
ldleworkover 7 years ago
😂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
value = "${element(module.elasticache.nodes, 0)["address"]}" - or this
loren
lorenover 7 years ago

element() may only be used with flat lists, this list contains elements of type map in ...
loren
lorenover 7 years ago
so, no element() for you...
ldlework
ldleworkover 7 years ago
...
ldlework
ldleworkover 7 years ago
lol what, how do I report the address of the primary elasticache node? can I just output the whole list?
loren
lorenover 7 years ago
aws_elasticache_cluster.cache.cache_nodes[0]
ldlework
ldleworkover 7 years ago
that will output the whole map right?
ldlework
ldleworkover 7 years ago
i guess i don't know why I'm trying to reduce all my outputs to single values
ldlework
ldleworkover 7 years ago
stops doing that.
loren
lorenover 7 years ago
i think i also mixed up the two errors there, oops
ldlework
ldleworkover 7 years ago
Oh ok, so I guess I still have a problem
ldlework
ldleworkover 7 years ago
I want to give internal simple DNS to the first ip in the elasticache cluster
ldlework
ldleworkover 7 years ago
I so had something like:
  records = ["${element(aws_elasticache_cluster.cache.cache_nodes.*.address, 0)}"]
ldlework
ldleworkover 7 years ago
But it sounds like none of the options we just discussed are going to work to extract the address of the first item?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Element will not work on list of maps
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
You can input and output anything
ldlework
ldleworkover 7 years ago
Really? How will I extract the address of the first map of the lists?
ldlework
ldleworkover 7 years ago
Wont I have the same exact problem there?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
You can output a list of values
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
And then use element
ldlework
ldleworkover 7 years ago
Genius.
ldlework
ldleworkover 7 years ago
So you mean, output all the addresses as a list using splat. Then take the first. I'll try it.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Something like that
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
here is an example of working with list of maps and getting the first element https://github.com/cloudposse/terraform-aws-eks-cluster/blob/master/main.tf#L114
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
[0] works with maps
ldlework
ldleworkover 7 years ago
I have:


data "null_data_source" "cache_addresses" {
  inputs = {
    addresses = ["${aws_elasticache_cluster.cache.cache_nodes.*.address}"]
  }
}

resource "aws_route53_record" "cache" {
  zone_id = "${var.zone_id}"
  name    = "${local.name}"
  type    = "CNAME"
  records = ["${data.null_data_source.cache_addresses.outputs["addresses"]}"]
  ttl     = "300"
}


and get:

Error: module.elasticache.aws_route53_record.cache: records: should be a list
Error: module.elasticache.data.null_data_source.cache_addresses: inputs (addresses): '' expected type 'string', got unconvertible type '[]interface {}'
ldlework
ldleworkover 7 years ago
this is a pretty confusing DSL all things considered 😂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
just try [0] for now
ldlework
ldleworkover 7 years ago
@Andriy Knysh (Cloud Posse)

resource "aws_route53_record" "cache" {
  zone_id = "${var.zone_id}"
  name    = "${local.name}"
  type    = "CNAME"
  records = ["${aws_elasticache_cluster.cache.cache_nodes[0].address}"]
  ttl     = "300"
}

Error downloading modules: Error loading modules: module elasticache: Error loading .terraform/modules/a86e58cdab02f33e0c2a0f76c4ae3934/stacks/elasticache/main.tf: Error reading config for aws_route53_record[cache]: parse error at 1:47: expected "}" but found "."
ldlework
ldleworkover 7 years ago
Is that what you meant?
ldlework
ldleworkover 7 years ago
Starting to feel a little dumb here lol.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago(edited)
[“${aws_elasticache_cluster.cache.cache_nodes[0][“address”]}“]
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago(edited)
if [0]["address"] together does not work, use locals as in the example (we had the same issues)
ldlework
ldleworkover 7 years ago
yeah that doesn't work
ldlework
ldleworkover 7 years ago
OK trying coalescelist
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
coalescelist has nothing to do with that 🙂
ldlework
ldleworkover 7 years ago
what am i looking at then hehe
ldlework
ldleworkover 7 years ago
the splat?
ldlework
ldleworkover 7 years ago
ohhhh
ldlework
ldleworkover 7 years ago
the next few lines
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
i mean use locals to first get [0] from the list, then another local to get [“data”] from the map
ldlework
ldleworkover 7 years ago
tries
ldlework
ldleworkover 7 years ago
@Andriy Knysh (Cloud Posse) thanks homie
ldlework
ldleworkover 7 years ago
Why is the ECS container definition for the environment a list of maps?
ldlework
ldleworkover 7 years ago
oh i probably need to do name/value
GFox)(AWSDevSecOps
GFox)(AWSDevSecOpsover 7 years ago(edited)
Hello, anyone have any Terraform modules code 🙂 to automate cis benchmarks in azure subscriptions / tenants ??? 🙂
GFox)(AWSDevSecOps
GFox)(AWSDevSecOpsover 7 years ago
if so, you're going to be very popular!
ldlework
ldleworkover 7 years ago
A dependency graph encompassing all the CloudPosse projects would be awesome
ldlework
ldleworkover 7 years ago
I'm changing the environment setting of a https://github.com/cloudposse/terraform-aws-ecs-container-definition module and it isn't updating the container defnition.
ldlework
ldleworkover 7 years ago
Not sure how to get it to change.
ldlework
ldleworkover 7 years ago
TIL how to taint
ldlework
ldleworkover 7 years ago
not clear how to taint the container definition though...
ldlework
ldleworkover 7 years ago
just destroys the whole module
ldlework
ldleworkover 7 years ago
😞
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
it just generates JSON output
ldlework
ldleworkover 7 years ago
yes
ldlework
ldleworkover 7 years ago
and with ecs-codepipeline
ldlework
ldleworkover 7 years ago
what am I missing?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
you need to taint https://github.com/cloudposse/terraform-aws-ecs-alb-service-task/blob/master/main.tf#L41, the resources that uses the generated json
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
but this will prevent from updating the entire task definition including the container definition
ldlework
ldleworkover 7 years ago
@Andriy Knysh (Cloud Posse) I ran terraform destroy on the whole module and it says:


* module.backend.module.ecs_codepipeline.aws_s3_bucket.default (destroy): 1 error(s) occurred:
* aws_s3_bucket.default: error deleting S3 Bucket (us-west-1-qa-backend-codepipeline): BucketNotEmpty: The bucket you tried to delete is not empty
ldlework
ldleworkover 7 years ago
I guess you have to manually go in and delete the bucket data?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
yes
ldlework
ldleworkover 7 years ago
lame 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
we will consider updating the module to add a var for force destroy on the bucket
ldlework
ldleworkover 7 years ago
@Andriy Knysh (Cloud Posse) why are the container definitions ignored in the lifecycle??
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
don’t know, did not implement that part, maybe there was a good reason, need to take a look
ldlework
ldleworkover 7 years ago
np
jaykm
jaykmover 7 years ago(edited)
Hello everyone,
I Just joined this channel.

I want to ask one thing related to API gateway, I'm able to create API gateway, methods, and integration with lambda and I also configured all the headers, origins, methods, in the response parameters but CORS has not been configured.
Can someone help me to get in, I can also send the terraform script how I'm implementing.
@sahil FYI
jaykm
jaykmover 7 years ago
@UFC8P2WJG No, I'm already given the permission of lambda to API gateway and I can also access the lambda from API gateway but If I do a request from the browser (Client) then It gives me CORS error and you can also see the error.
jaykm
jaykmover 7 years ago
jaykm
jaykmover 7 years ago
@UFC8P2WJG
midacts
midactsover 7 years ago(edited)
https://github.com/cloudposse/terraform-aws-ec2-instance
Would i need to fork this to add a provisioner? We'd need to join Windows machines to the domain and run post provisioning steps on Windows and Linux.
ldlework
ldleworkover 7 years ago
@oscarsullivan_old only the root module's outputs are printed right?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
you need to add the outputs from the child module(s) to the outputs of the parent
antonbabenko
antonbabenkoover 7 years ago
Just discovered https://codeherent.tech/ - new kid in town, or am I the last one to find it? 🙂
Igor
Igorover 7 years ago
Looks cool ^^
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
They have been reaching out to us
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Guess I should check it out
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
I suppose this appeals to a certain mode of developer and probably not the one using vim as their primary IDE ;)
Igor
Igorover 7 years ago
It might make it easier to do code reviews
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
yes, as usual something like that looks nice and helpful at first, but then always gets in the way
johncblandii
johncblandiiover 7 years ago
Does this example actually work? https://github.com/cloudposse/terraform-aws-ecs-container-definition/blob/master/examples/multi_port_mappings/main.tf#L25-L36

I got this error when using a similar approach, but with 8080/80 (container/host):


* aws_ecs_task_definition.default: ClientException: When networkMode=awsvpc, the host ports and container ports in port mappings must match.


So I sleuthed and found https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_PortMapping.html:

If you are using containers in a task with the awsvpc or host network mode, exposed ports should be specified using containerPort. The hostPort can be left blank or it must be the same value as the containerPort.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
we're holding an office hours
Alex Siegman
Alex Siegmanover 7 years ago(edited)
@Erik Osterman (Cloud Posse) Yeah I'm just listening in to see what's up
Igor
Igorover 7 years ago(edited)
Has anyone ever gotten a diffs didn't match during apply. This is a bug with Terraform and should be reported as a GitHub Issue. message from Terraform?
Tim Malone
Tim Maloneover 7 years ago
i’ve seen it a couple of times. usually just another apply fixed it…
… but i would make sure your state is backed up - if you don’t already have it versioned - just in case
Igor
Igorover 7 years ago
Thanks, it looks like it didn't harm anything on my side either
ldlework
ldleworkover 7 years ago
Maybe consider making https://github.com/cloudposse/terraform-aws-ecs-alb-service-task.git support building the alb conditionally
ldlework
ldleworkover 7 years ago
the target_group / listener I mean
ldlework
ldleworkover 7 years ago
It's annoying to do in 0.11 I know - but that's exactly why I'd prefer if you maintained the HCL
ldlework
ldleworkover 7 years ago
🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
we’ll consider that 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
inactive
inactiveover 7 years ago
Guys, I have a question regarding team collaboration. What do you guys use to work in teams when running terraform? I know that terraform provides a solution to this on their paid tiers, but they are cost prohibitive for us. Currently, our workaround has been to share our terraform templates via a cloud drive which is replicated locally to each developer workstation. It sorta works, but there are running into several limitations and issues. Any suggestions?
Nikola Velkovski
Nikola Velkovskiover 7 years ago
Hey people,
I just noticed some interesting behavior when using the ECR module
https://github.com/cloudposse/terraform-aws-ecr/blob/master/main.tf#L34
makes apply fail if it's empty.
I use the label module but I do not have stage set, so I am wondering if
setting a simple conditional here:
https://github.com/cloudposse/terraform-aws-ecr/blob/master/main.tf#L11
would be good enough ?
Nikola Velkovski
Nikola Velkovskiover 7 years ago
The error is

aws_ecr_lifecycle_policy.default: InvalidParameterException: Invalid parameter at 'LifecyclePolicyText' failed to satisfy constraint: 'Lifecycle policy validation failure:
string "" is too short (length: 0, required minimum: 1)
Nikola Velkovski
Nikola Velkovskiover 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
hmm… why don’t you use something for the stage, e.g. test or testing? what’s the reason to not use it (all the modules were designed with the assumption to have namespace, stage and name )
Nikola Velkovski
Nikola Velkovskiover 7 years ago
no, but that's just because I Am used to environment rather than stage
Nikola Velkovski
Nikola Velkovskiover 7 years ago
nvm I can pass env in place of stage
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
yep, those are just labels 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
(environment and stage are the same in all cases we use the modules)
Nikola Velkovski
Nikola Velkovskiover 7 years ago
thanks for the reply though 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
You can even change the order if using terraform-null-label
Nikola Velkovski
Nikola Velkovskiover 7 years ago
oh yeah, I really dig that module 🙂
Nikola Velkovski
Nikola Velkovskiover 7 years ago
kudos for that one
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
hi 😃
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
hi @Julio Tain Sueiras
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
just wanted to ask the opinion for a terraform LSP?
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
(though right now mostly focusing on adding provider/provisioner completion and adding terraform v0.12 support once is in GA to my plugin)
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
also any vim + terraform users here?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
what is terraform LSP? https://langserver.org ?
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
correct
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
if you are talking about autocompletion/syntax highlighting/error highlighting, a lot of people are using VScode (with terraform plugin) or JetBrains IDEA (with terraform plugin)
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
I know that part(also a lot of people use my plugin for vim as well)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
nice plugin @Julio Tain Sueiras
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
since my thinking is, a LSP approach would allow any editor, and also can just focus on the new features rather then editor specific implementation
loren
lorenover 7 years ago
i like the LSP idea
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
for ex. vlad's plugin is very good
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
but because is a standard jetbrains plugin
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
meaning that update time is quite far apart
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
but terraform provider get updated very frequently
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
(for that issue, I implemented a bot that auto update ,and do version base completion, incase you want to use data from older version of the provider)
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
also LSP would allow even more nonstandard editor
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
to have terraform features
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
like Atom(RIP)
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
P.S. @Andriy Knysh (Cloud Posse) for the vscode terraform plugin, right now, it only support aws/gcp/azure
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
he is implementing a new feature that would load the completion data from my plugin
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
oh nice
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
if you click on aws for example
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
it have the data for every version of the provider
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
since my thinking was, what if you want to lock to a version
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
and only want the editor have completion data of that version
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
great, looks like you did a lot of work for that
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago(edited)
there is also extra small stuff like, autocomplete module's attribute and argument , autocomplete modules list from registry.terraform.com, evaluate interpolation, open documentation of the current parameter in a module, etc
Maarten van der Hoef
Maarten van der Hoefover 7 years ago
@Julio Tain Sueiras cool stuff!
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
the entire reason I did the plugin is, I don't want to use another editor 😃
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
but if there is no autocompletion, then terraform is quite annoyed to work with
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
(yea, that’s what it says on your GitHub profile 🙂 )
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
but yeah, once TF0.12 is on GA
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
then I will work a LSP implementation
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
especially because Golang have the AST for HCL
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
are you going to use Golang or Ruby?
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
go
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
even better 🙂
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
I did several provider for terraform & added new feature to official terraform providers(vsphere) and packer(vsphere as well)
Maarten van der Hoef
Maarten van der Hoefover 7 years ago
@Julio Tain Sueiras Can you add a Microsoft Clippy in case a user is writing iam_role_policy_attachment ? 😉
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
also 🤔 how would that work
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
?
Maarten van der Hoef
Maarten van der Hoefover 7 years ago
Clippy:
It looks like you want to use iam_role_policy_attachment, are you sure about that ? 😉
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
XD
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
I actually use that, since I don't like dealing with json
Maarten van der Hoef
Maarten van der Hoefover 7 years ago
do you also correct formatting like hclfmt does ?
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
my plugin have a dependencies for vim-terraform
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
which have auto format
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
using terraform format
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
also any of you guys use vault?
Maarten van der Hoef
Maarten van der Hoefover 7 years ago
I'm working with it at the moment
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
since I had a meeting with the Hashicorp's people(Toronto division) around 1 month ago
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
and I mention that I wish there is a similar thing for vault like aws_iam_policy_document data source
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
and they have it now
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
but...............
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
is not on the terraform docs
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
why not?
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
not sure
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
but is part of the official release
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
not a beta/alpha
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
so you can write all the vault policy
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
without using heredoc
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
that’s nice
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
P.S. the biggest best feeling I did was for kubernetes pod terraform
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
@Andriy Knysh (Cloud Posse) https://asciinema.org/a/158264
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
kube provider is nightmare to work with, without autocompletion
antonbabenko
antonbabenkoover 7 years ago
You can always use json2hcl (https://github.com/kvz/json2hcl) to write packer.json as hcl and then just convert it to valid json. I use it pretty often for various cases.
antonbabenko
antonbabenkoover 7 years ago
Though more often I use yaml<>json using https://github.com/dbohdan/remarshal
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
the use case I did for packer is because I want to reference terraform resources in packer
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
and json2hcl is a hard convert
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
but doesn't account for several things
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
like terraform doesn't understand the ideal of sequential
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
(so provisioners is going to have problem)
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
I use my provider with local_file resource
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
then null_resource with packer build
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
or validate
antonbabenko
antonbabenkoover 7 years ago(edited)
@Julio Tain Sueiras I used your documentation generators back in the days (2+ years ago in my project - https://github.com/antonbabenko/terrapin). I mentioned you there in the bottom of README 🙂 Thanks for that! It saved me some time. I will probably contact you in the future. 🙂
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
no problem 😃
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
one of the most funniest I did (with my plugin)
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
was doing terraform with my android phone
antonbabenko
antonbabenkoover 7 years ago
hehe, this one is pretty amazing usage of Terraform - https://www.youtube.com/watch?v=--iS_LH-5ls
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
nice!!
Bruce
Bruceover 7 years ago
Hey guys, I'm looking for a module to create a client VPN server to connect to our aws private network (home of our vault server). Any suggestions?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
hi @Bruce we have this module to create a vpn connection on AWS https://github.com/cloudposse/terraform-aws-vpn-connection
Maxim Tishchenko
Maxim Tishchenkoover 7 years ago(edited)
hey guys, I’m trying to fetch users from group via TF.
but seems for me it is impossible…
it is correct?

I was tried to fetch like that

data "aws_iam_group" "admin_group" {
  group_name = "Admins"
}

but I can’t get user list from this data…

I was tried to fetch user with group like this

data "aws_iam_user" "admins" {
}

but it doesn’t have such filter.

can anybody help me ?
ldlework
ldleworkover 7 years ago
Why would count = "${var.alb_target_group_arn == "" ? 1 : 0}" produce: * module.backend-worker.module.task.aws_ecs_service.default: aws_ecs_service.default: value of 'count' cannot be computed
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
rohit
rohitover 7 years ago
@Erik Osterman (Cloud Posse) I don’t understand where tfbundle would be useful. could you please elaborate the usecase(s) ?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
complex lambdas e.g. that npm install a lot of deps.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
using artifacts is nice b/c those artifacts are immutable
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
also, anyone can then deploy that lambda even if they don't have npm installed locally
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
typically, in terraform modules that you see deploy an advanced lambda, they expect the end user to have a full local dev environment with all build tools required to build the lambda and zip it up. if that lambda is instead distributed as a zip, it mitigates that problem.
rohit
rohitover 7 years ago
makes sense now
rohit
rohitover 7 years ago
Thanks for elaborating
rohit
rohitover 7 years ago
What is the best way to version your own terraform modules ? So that you use a particular version of your module(well tested) in prod and can also actively work on it
ldlework
ldleworkover 7 years ago
By splitting your sub-modules from your root-modules and using a git-tag as the source value on the module invocation.
rohit
rohitover 7 years ago
I was thinking in the same lines but not exactly sure on how to do it. Does you have examples that i can look at ?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
those are all of our examples of invoking modules
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
everytime we merge to master, we cut a new release
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
then in prod or staging, we do something like terraform init -from-module=git::<https://github.com/cloudposse/terraform-root-modules.git//aws/chamber?ref=tags/0.35.1>
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago(edited)
(technically, we use cloudposse/tfenv to make this easier)
rohit
rohitover 7 years ago
I have my root module that invokes my submodules(compute,storage,networking) and each of my submodules uses different resourses/modules, what would be the best way to version in this scenario ?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
You version your root modules
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
then the root modules are what you promote to each account/stage
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
are you familiar with terraform init -from-module=?
ldlework
ldleworkover 7 years ago
Why would I be getting:

 module.frontend.module.codepipeline.module.ecs_codepipeline.module.github_webhooks.provider.github: 1:3: unknown variable accessed: var.github_organization in:
ldlework
ldleworkover 7 years ago
It's a variable problem inside of ecs-codepipeline?
ldlework
ldleworkover 7 years ago
scratches his head.
ldlework
ldleworkover 7 years ago
"Error: Error asking for user input: 1 error(s) occurred:"
ldlework
ldleworkover 7 years ago
oh man i have no idea what this is
ldlework
ldleworkover 7 years ago

2019-03-30T17:24:05.054-0500 [DEBUG] plugin.terraform-provider-github_v1.3.0_x4: plugin address: timestamp=2019-03-30T17:24:05.054-0500 network=unix address=/tmp/plugin407023942
2019/03/30 17:24:05 [ERROR] root.frontend.codepipeline.ecs_codepipeline.github_webhooks: eval: *terraform.EvalOpFilter, err: 1:3: unknown variable accessed: var.github_organization in:
ldlework
ldleworkover 7 years ago
ah ok fixing some other errors fixed it
ldlework
ldleworkover 7 years ago
I have two ecs-codepipelines working great. When I tried to deploy a third, I got:

Action execution failed
Error calling startBuild: User: arn:aws:sts::607643753933:assumed-role/us-west-1-qa-backend-worker-codepipeline-assume/1553989516226 is not authorized to perform: codebuild:StartBuild on resource: arn:aws:codebuild:us-west-1:607643753933:project/us-west-1-qa-backend-worker-build (Service: AWSCodeBuild; Status Code: 400; Error Code: AccessDeniedException; Request ID: deba2268-5345-11e9-a5ef-d15213ce18a0)

I'm confused because the ecs-codepipeline module does not take any IAM information as variables...
ldlework
ldleworkover 7 years ago
Looks like the ARNs mentions are unique to that third service "backend-worker"
ldlework
ldleworkover 7 years ago
So like there shouldn't be any collision with the others. In which case why wouldn't the IAM stuff that the module creates for itself be working?
ldlework
ldleworkover 7 years ago
Can anyone please help me reason why ecs-codebuild module would suffer this permission error when trying to build the container?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
hrmmmm that's odd... you were able to deploy (2) without any problems, but the (3)rd one errors?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
this opens up interesting possibilities
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
anyone ever kick the tires on this?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
...basically execute any command as part of the apply or destroy phase; this is different from local-exec on null_resource
ldlework
ldleworkover 7 years ago
lol I think my issue is coming from the fact that my codepipeline is getting created before the service/task
ldlework
ldleworkover 7 years ago
the service uses the json from the container, the container uses the ecr registry url from the codepipeline and so yeah
ldlework
ldleworkover 7 years ago
the world makes sense again
ldlework
ldleworkover 7 years ago
phew!
ldlework
ldleworkover 7 years ago
guess I have to break out the ECR from my codepipeline abstracting module
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
i love that feeling