#terraform - May 2019

Tony
Tonyover 7 years ago
What are the pros and cons of using data lookup vs remote state file ?
Tony
Tonyover 7 years ago
If in an large organization where different group does different part and the remote state file is not accessible the I would say use data lookup. But if the same time that does all the setup then is it better to use either data lookup or remote state file ?
Andy Litzinger
Andy Litzingerover 7 years ago
does anyone know if the terraform-aws-vpc-peering module can be used to peer inter-region vpcs? I don't see any way to specify a region for the requestor or acceptor
Andy Litzinger
Andy Litzingerover 7 years ago
I'm trying to create a mesh of vpc peering between about 6 vpcs each in a different region, all part of the same account
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago(edited)
right now, we don't support that in our vpc peering module b/c it requires passing the provider and there's no way to make it optional
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
we have a cross account peering module, but no cross region
Andy Litzinger
Andy Litzingerover 7 years ago
ok, thanks. I have seen the cross-account module, was wondering if i could still use it but with a single account
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
I think it would work actually
mrwacky
mrwackyover 7 years ago
TIL
When the environment variable TF_IN_AUTOMATION is set to any non-empty value, Terraform makes some minor adjustments to its output to de-emphasize specific commands to run. The specific changes made will vary over time, but generally-speaking Terraform will consider this variable to indicate that there is some wrapping application that will help the user with the next step.
mrwacky
mrwackyover 7 years ago
Am I the only one who can't get terraform init -force-copy to behave as indicated? It still prompts "do you want to copy local state to remote backend".
mrwacky
mrwackyover 7 years ago
yes you bastards, YES
mrwacky
mrwackyover 7 years ago(edited)
mrwacky
mrwackyover 7 years ago
This is almost a year old fix ;(
Lee Skillen
Lee Skillenover 7 years ago(edited)
Not really a suggestion, but ... time to move to remote state 100% of the time? 🙂 (we use terragrunt to help us with this, although necessity for it will be diminished a bit in 0.12+).
mrwacky
mrwackyover 7 years ago
Ha. I'm doing a bunch of terraform state mv to rename things, and it's waaay faster to pull it locally, do the moves, then re-enable remote state
mrwacky
mrwackyover 7 years ago
Otherwise, we are all remote
Tim Malone
Tim Maloneover 7 years ago
^ haha yeah i do the same thing often. oh, and when mv’ing things between states, it’s kinda required to pull it locally / push it back afterwards
Tim Malone
Tim Maloneover 7 years ago
but i always get scared that someone else will modify in the meantime… i think i needa write a quick ‘lock this state in dynamodb’ aws cli alias
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
So got golang lsp skeleton working, and the lsp will have both static schema and provider plugin based , should able to show a demo this week
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
=) if everything goes well, I might expand to include nomad and other hcl based tool
Andy Litzinger
Andy Litzingerover 7 years ago
while using the aws-vpc-peering-multi-account module I get an error on the first run like so Error modifying VPC Peering Connection Options: OperationNotPermitted: Peering pcx-xxxxx is not active. Peering options can be added only to active peering which basically seems to be an object creation/modification race condition. I can re-run the apply and it gets past that error
Andy Litzinger
Andy Litzingerover 7 years ago
my question is, is there a common way to avoid having to do this or is it just something I have to accept?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
hrmmm probably an underlying terraformism
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
You need to apply twice
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
There is no way in TF to wait for the connection to become active
Andy Litzinger
Andy Litzingerover 7 years ago
ok, thanks!
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
There are two ways to fix it: separate connection options into a separate module, or apply with —target
Andy Litzinger
Andy Litzingerover 7 years ago
after I run a second time I'm getting a different error:
* aws_vpc_peering_connection_options.requester: Error modifying VPC Peering Connection Options: OperationNotPermitted: Modifying VPC peering connection options AllowEgressFromLocalClassicLinkToRemoteVpc, AllowEgressFromLocalVpcToRemoteClassicLink is not supported for cross-region VPC peering connections
Andy Litzinger
Andy Litzingerover 7 years ago
same error for the accepter
Andy Litzinger
Andy Litzingerover 7 years ago
the plan clearly shows it is attempting to set that property:
~ module.vpc_peering_cross_account.aws_vpc_peering_connection_options.accepter
      accepter.1102046665.allow_classic_link_to_remote_vpc:  "" => "false"
Andy Litzinger
Andy Litzingerover 7 years ago
but I can't figure out where the module is getting the variable for accepter.xxxxxxx.allow_calassic_link_to_remote_vpc
Andy Litzinger
Andy Litzingerover 7 years ago
or more specifically, how I can keep my terraform setup from trying to send/set that variable
Andy Litzinger
Andy Litzingerover 7 years ago
hmm, seems the error messages were misleading. it turns out the one parameter that was being explicitly set in the module, allow_remote_vpc_dns_resolution is actually not allowed for cross-region peering. once I set this to false the errors disappeared
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Nice find. We did it for cross account but in the same region, so the flag was set to true and it worked
xluffy
xluffyover 7 years ago
hey, anybody try to work around depends_on module? look like https://github.com/hashicorp/terraform/issues/1178#issuecomment-449158607?
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
@Andriy Knysh (Cloud Posse) https://asciinema.org/a/NBjkPvXsqnTqARnWHiEOl7o6s 😃 what you think?
johncblandii
johncblandiiover 7 years ago
quick PR to note it before I forget the details over the weekend 🙂 https://github.com/cloudposse/terraform-aws-eks-workers/pull/14
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
https://asciinema.org/a/Ey0Tt3zlveGWoSv71tBCrK7kX next stage is full recursive completion
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
(so you can then complete any level of module variable)
Richy de la cuadra
Richy de la cuadraover 7 years ago
hi, i get “ERROR: Job failed: exit code 1” when a run terraform init at gitlab-ci, somebody knows why?
Tim Malone
Tim Maloneover 7 years ago
could be anything 🙂 there should be additional logs available with a more specific error message
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
got resource completion, I will be releasing the first version(github) within next week

the first version should have these:

- Variable Completion(complete completion, infinite nesting type, including mix of list and map)
- Resource Completion(including nesting blocks)
- Data Source Completion(including nesting blocks)
- Functions Completion(including signatures)
- Provider config completion
- Backend completion
- Module Completion(including infinite nesting input)
- Error checking


Note: the resource and data source will talk to the terraform provider binary using grpc, so A) it will provide completion data of the version you specify, B) it will not require wait for update
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
any feedback is welcome
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
after the first version, I will need to figure out how to provide completion for scope for loop and dynamic block
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
oh forgot to mention Provisioner completion
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
I will need to see if it can talk to grpc based provisioner
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
(hence providing completion for ansible provisioner)
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
@Andriy Knysh (Cloud Posse) for you, since you use intellij 😃
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
(this is using intellij-lsp)
C
Cloud Posseover 7 years ago
Join us for "Office Hours" every Wednesday 11:30AM (PST, GMT-7).

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Mar 20, 2019 11:30AM.

👉️ Add it to your calendar

:zoom: https://zoom.us/j/684901853

:slack: #office-hours (our channel)
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
turn for VSCode 😃
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
Terraform LSP Client for Atom
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
question, what do you guys consider a must require to have for a editor plugin for terrafor
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
terraform*
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
(features-wise)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
1. highlight errors (wrong vars, missing vars, wrong resources, etc.)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago(edited)
2. Show ref count, when clicking on it shows all references/usages
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
3. Autocompletion 🙂
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
k, np, those two are in my list of next thing to do after releasing tomorrow
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
third one np, 😉 I am also working providing dynamic block completion
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
so completion with the context within dynamic block
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
4. Go to implementation
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
I most likely will have 1,2,4 done around friday or Saturday this week
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
since the funniest thing about HCL2 ¯\(ツ)/¯
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
is that every(and I mean every) syntax tree object have a range
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
function, and associated range var(like type range, declare range, open range, close range, etc)
johncblandii
johncblandiiover 7 years ago
is there a clean way to get the output of a local CLI call from a null_resource? My googling isn’t pulling up any solid results
Adam Barnwell
Adam Barnwellover 7 years ago
Hey all, quick question. I’m using some awesome Cloud Posse modules in terraform but unfortunately it doesn’t support all of the attributes I need for the resulting resources. What’s the best way of updating the resources created from the modules after the fact?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
@Adam Barnwell you can open a PR to add the missing attributes, then we merge it to master, then you apply again
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
I already created a vscode extension for it
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
The languageclient extension for it, already did the same for atom
Joe Presley
Joe Presleyover 7 years ago
@Julio Tain Sueiras Do you plan to create vim plugin as well?
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
XD
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
Vim , IntelliJ, and few others(Emacs,etc) is support natively
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
Atom and Vscode need the proxy client extension for lsp
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
Vim and the others doesn’t need
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
So the only thing you need is the general lsp client and the terraform-lsp
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
I will put the per editor instruction
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
For setup
rohit
rohitover 7 years ago
when using aws_cloudfront_distribution resource, if i want to enable Forward all, cache based on all value for Query String Forwarding and Caching what all parameters do i have to enable ?
rohit
rohitover 7 years ago(edited)
forwarded_values {
      query_string = true
 }
rohit
rohitover 7 years ago
is this good enough ?
rohit
rohitover 7 years ago
because in the under cloudfront distribution console, i see another option called Forward all, cache based on whitelist
rohit
rohitover 7 years ago
so i am not sure what option will be enabled
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
there is an extra option under that block call query_string_cache_keys
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
which is a list of string
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
from the tf docs

When specified, along with a value of true for query_string, all query strings are forwarded, however only the query string keys listed in this argument are cached. When omitted with a value of true for query_string, all query string keys are cached.
rohit
rohitover 7 years ago
makes sense
rohit
rohitover 7 years ago
thanks
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
getting there
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
doing the release in 1-2 hour, need to work out the issue with goreleaser
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
FYI: of course there is a lot of improvement and features still need to be implement
David Nolan
David Nolanover 7 years ago
I want to add a feature to https://github.com/cloudposse/terraform-aws-rds to pass additional security groups in to the module which will be added to the RDS instance. I have it working a fork, before I push a PR upstream I was wondering what the best name for the new module parameter is. For now I chose server_security_group_ids but I'm open to suggestions or pointers to naming conventions from other modules.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
hey @David Nolan thanks for doing it
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
we called it allowed_security_groups before
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
David Nolan
David Nolanover 7 years ago
this module already has security_group_ids which implements that functionality, but that is different from what I'm looking to add.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
but I think the modules has it already with diff name https://github.com/cloudposse/terraform-aws-rds/blob/master/variables.tf#L33
David Nolan
David Nolanover 7 years ago
right now it creates an SG which applies ingress rules to the RDS instance. I want to join the RDS instance to an existing SG which is used as a target on egress rules in other security groups.
David Nolan
David Nolanover 7 years ago
i.e. I have RDS clients which have a SG ("client SG") with a rule allowing port 3306 to an existing "server SG" security group, and I want that "server SG" group added to the RDS instance
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
it allows ingress from the existing SGs to the instance
David Nolan
David Nolanover 7 years ago
that a list of allowed clients, where I want to also add the RDS to a list of allowed servers on those clients
David Nolan
David Nolanover 7 years ago
The result is the RDS instance ends up with multiple SGs applied, which is a powerful usecase
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
i think it’s what we have now, no? 🙂
David Nolan
David Nolanover 7 years ago
No, what exists now is "create a new SG which allows ingress from this list of security groups"
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
ah sorry, you want to add the instance to those SGs
David Nolan
David Nolanover 7 years ago
Right, I also want to add the instance to a list of SGs
David Nolan
David Nolanover 7 years ago
that allows other SGs to target this RDS instance (and potentially dozens of other instances) with a single rule targetting that server group
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
so we did not do it before b/c it’s easier to create a SG for the instance9s) and then connect that SG to any other SGs you have
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
placing the instance in a list of SGs will have the same result
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
but more complicated to manage
David Nolan
David Nolanover 7 years ago
In this case I've got a use case for a single SG, "Managed RDS instances" and then a vault server which has a rule which allows it to talk to only those known RDS instances. The way to add new RDS instances to that known list is by joining the instance to that SG
David Nolan
David Nolanover 7 years ago
The result (which works in my testing) is the RDS instance ends up in two SGs. One exists to defined the ingress rules specifically for this instance, the other exists to provide the grouping of servers for targetting by the separate service.
David Nolan
David Nolanover 7 years ago
I would expect the behavior of "I already have an SG, add it to the RDS instance" to be pretty common, as it allows for centralized management of SG rules. (In my case the existing SG is managed by our legacy CloudFormation, so I have to import that data and apply it to the RDS)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
i understand your use-case, but will not you achieve the same result by connecting “Managed RDS instances” SG to the ingress rule of the created SG for the instance?
David Nolan
David Nolanover 7 years ago
That doesn't change the Egress rules on my other service. (I tried that first before having the AHA moment and realizing what was missing)
David Nolan
David Nolanover 7 years ago
In order to make this RDS instance a matching target in the egress rules in the my mysql client instance's SG rules, the RDS instance has to be added to that SG.
David Nolan
David Nolanover 7 years ago
If the existing parameter was named client_security_group_ids then naming this one server_security_group_ids might be more obvious in intent. Other names I contemplate are like extra_server_security_group_ids or join_security_group_ids.
David Nolan
David Nolanover 7 years ago
As with many thing in AWS, there's more than one way to do the SG setup... adding support here makes the module more flexible for working with existing setups.
David Nolan
David Nolanover 7 years ago
I'd be happy to open the PR and move discussion there if that makes sense.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
yes please
David Nolan
David Nolanover 7 years ago
FWIW, I'm doing a bunch of work on building new infra w/ terraform, and I blame @sarkis for getting me pointed at the cloudposse module suite as a strong starting point. But I'm deploying in an environment that is heavily CloudFormation based now. So I expect I'll continue to find edge cases where I want to add more flexibility like this. So I'm trying to understand your preferred flow on proposed feature additions. 😉
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
yes thanks for using the modules
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
and thanks @sarkis 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
btw, in cases like this, we did the following: provide an existing SG to the module in a var. If it’s empty, then create a new SG and all the rules. If it’s not empty, use it and don’t create SG and the rules
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
but what you are proposing will work too
David Nolan
David Nolanover 7 years ago
I think its nice and flexible with both options. You can define known clients specific to this database by passing in the client security group list security_group_ids, and now you can add common rules (for management, monitoring, etc) that are defined elsewhere by adding the instance to an existing SG as well.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
@Andriy Knysh (Cloud Posse) question, right now I am mostly focusing on non-intrusive features for the LSP(code completion, goto reference, error check, etc), do you think CodeAction would be useful ?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
@Julio Tain Sueiras quick fixes would be great, without them even if you highlight the issues, developers lose time reading and understanding the message, and then inventing a way to fix it manually. Not sure how easy for you to implement all of that 🙂
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
Quick easily actually, only part is to figure out what are the common issue pattern in developing terraform
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
will be adding functions error checks(next is checking for attributes and the others)
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
an important question @Andriy Knysh (Cloud Posse) when you have some time
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
what you think 😃
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Wow super
David Nolan
David Nolanover 7 years ago
Is there a policy on version compatibility with the AWS provider? I'm adding support for a flag that has been supported in the provider since version 1.39.0 (last October).
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
added Basic Dynamic Block Completion
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
also data source & reosurce attribute completion
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
onward to locals, and for each
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
and now welcome for inferred type completion in for reach https://asciinema.org/a/245663
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
😃
C
Cloud Posseover 7 years ago
Join us for "Office Hours" every Wednesday 11:30AM (PST, GMT-7).

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Mar 20, 2019 11:30AM.

👉️ Add it to your calendar

:zoom: https://zoom.us/j/684901853

:slack: #office-hours (our channel)
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
added a mechanism for dealing with google-beta
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
(it will read if your resource is using provider = google.beta)
Bruce
Bruceover 7 years ago
Any advice on transitioning ECS CloudFormation over to Terraform?
shaiss
shaissover 7 years ago
@Bruce I’ve seen some cft to tf converters, but honestly they never worked for me. I just ended up using in this case, the ECS tf module in the registry or one of the community ones
shaiss
shaissover 7 years ago
Bruce
Bruceover 7 years ago
Thanks @shaiss I'll be looking at writing the current ECS deployments in Terraform so this helps alot! Thanks.
shaiss
shaissover 7 years ago
does anyone know if there’s a way to make bucket/iam policies dynamic based on TF vars? IE https://www.terraform.io/docs/providers/aws/r/s3_bucket_policy.html. Instead of the resource stating the bucket name, it would be handy to have tf replace that value at runtime with the computed/coded bucket name.
shaiss
shaissover 7 years ago(edited)
any suggestions on making this more readable? It works, just fugly!
shaiss
shaissover 7 years ago

data "template_file" "init" {
  template = "${replace(replace(file("${var.bucket_policy}"),"[log_prefix]","${var.log_prefix}"),"[bucket_name]","${var.bucket_name}")}"
}
shaiss
shaissover 7 years ago
looks like using vars might work

  vars    = {
    bucket_name = "${var.bucket_name}"
    log_prefix  = "${var.log_prefix}"
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
@Andriy Knysh (Cloud Posse) I know is a bit off-topic, but would a lsp for nomad be useful?
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
nomad jobspe
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
jobspec*
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
and 😃 I finished it(the nomad-lsp), will release it in one hour, right now the only thing missing is the full schema, which I will put it in this week, mostly going to focus on terraform-lsp though
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
https://twitter.com/juliosueiras/status/1128502456932081664 , nomad is a lot easier to tackle compare to terraform
Bogdan
Bogdanover 7 years ago
In my providers.tf I've defined 4 aliased providers which assume different roles in different regions. However, there's also an invisible non-aliased provider which although not defined anywhere keeps prompting me for entering a value for provider.aws.region on every terraform apply and when trying to run apply it doesn't work unless I set my access/secret keys in an env var or as [default] profile in /.aws/credentials.
Bogdan
Bogdanover 7 years ago
terraform providers lists it, but I can't find its config...
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
or use the credentials from the default profile if not configured directly on the provider
Bogdan
Bogdanover 7 years ago
@Andriy Knysh (Cloud Posse) so you think that it can be (defined) in a module that is external/on the registry and thus while not visible in my cfgs still picked-up when running apply?
For now, I'll have to use the default profile on aws
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
it can be defined in any module, but it’s better not to define it in low-level modules that are used in top-level modules
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
let the top level modules (or examples) define everything they need to run
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
and provide all vars, settings, and providers with regions and credentials/roles
Bogdan
Bogdanover 7 years ago
ironically among the modules I'm using is cloudposse/terraform-aws-iam-user but searched any mention of a provider and there's nothing
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
yea, you need to define the provider in your top-level module. If not defined, TF will ask you for the region
Bogdan
Bogdanover 7 years ago
but i already defined 4 (aliased) providers which I'm passing to each terraform-aws-iam-user instantiation:
...
  providers = {
    aws = "aws.users.frankfurt"
  }
Bogdan
Bogdanover 7 years ago
that's what I find surprising...
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
yea, but TF is asking you for the region for the main provider for your main module which instantiates the 4 terraform-aws-iam-user modules
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
in your main module, add something like this
provider "aws" {
  region = "${var.region}"
}
S
SweetOpsover 7 years ago
Are you using some of our terraform-modules in your projects? Maybe you could leave us a testimonial! It means a lot to us to hear from people like you.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
office hours starting now: https://zoom.us/j/684901853
johncblandii
johncblandiiover 7 years ago
is there a hard requirement for this default security group to be added? https://github.com/cloudposse/terraform-aws-alb/blob/master/main.tf#L11
dalekurt
dalekurtover 7 years ago
@Erik Osterman (Cloud Posse) I will need to jump in on the next office hours. I’ve been running issues deploying the reference architecture on my AWS account.
Josh Larsen
Josh Larsenover 7 years ago
i'm curious if anyone ever had an issue with aws_route53_zone changing the order of the nameservers on you... specifically for creating the SOA record (which is usually set to the first name server in the array). i looked in the terraform source and they are doing a SORT() on the nameserver list for some odd reason, sorting them in alphabet order, which makes no sense.
Josh Larsen
Josh Larsenover 7 years ago
so we cannot set the first parameter of the SOA record to the correct value because terraform is sorting them... but i'm unsure if this matters or not. RFC for SOA record states that first arg is the primary master name server... but unsure how aws graphs their dns replication, but it just seems unsafe to be choosing one seeming at random since we don't have the original order in the name_servers output of aws_route53_zone. any thoughts?
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
for nomad lsp, I will have a nice little surprise today 😃
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
Stephen Lawrence
Stephen Lawrenceover 7 years ago(edited)
What is the SweetOps preferred way of standing up a cloud-agnostic kubernetes cluster via Terraform?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
There's no "cloud agnostic" way to setup a kubernetes cluster; doing so would preclude taking advantage of the best capabilities of that cloud provider.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Even terraform is not cloud agnostic. The way you terraform for AWS is different from GKE, Azure etc.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
For AWS we're still predominantly using #kops due to it's support for managing the full lifecycle of the kubernetes cluster including rolling updates/upgrades, which is not well supported by the other options. I think eksctl (by weaveworks) has recently adding some support for rolling updates.
evgmoskalenko
evgmoskalenkoover 7 years ago
@Andriy Knysh (Cloud Posse) , Hello, what do you think about this - https://github.com/cloudposse/terraform-aws-ec2-autoscale-group/issues/9#issuecomment-493318731
Bogdan
Bogdanover 7 years ago
Anyone knows how to manage multiple providers which assume an assumed role (i.e. crossaccount assumes)?
I have the following setup:

provider "aws" {
  region  = "eu-central-1"
  profile = "${var.aws_profile}"
}

provider "aws" {
  version = ">= 2.10"
  region  = "eu-central-1"

  assume_role {
    role_arn = "arn:aws:iam::${var.custom_account_id}:role/${var.custom_role_name}"
  }

  profile = "${var.aws_profile}"

  allowed_account_ids = [
    "${var.custom_account_id}",
  ]

  alias = "custom.frankfurt"
}

provider "aws" {
  version = ">= 2.10"
  region  = "eu-central-1"

  assume_role {
    role_arn = "arn:aws:iam::12345:role/12345-admin"
  }

  allowed_account_ids = [
    "12345",
  ]

  profile = "${var.aws_profile}"

  alias = "12345.frankfurt"
}

provider "aws" {
  version = ">= 2.10"
  region  = "eu-central-1"

  assume_role {
    role_arn = "arn:aws:iam::12354:role/12354-admin"
  }

  allowed_account_ids = [
    "12354",
  ]

  alias = "12354.frankfurt"
}
Bogdan
Bogdanover 7 years ago
which results in the following error:
Bogdan
Bogdanover 7 years ago
the profiles is defined in ~/.aws/config and work just fine with aws sts assume or other awscli commands
Bogdan
Bogdanover 7 years ago
I want to be able to provision resources across accounts and regions by different providers which assume a different role for each account essentially
Steven
Stevenover 7 years ago
You need to create an alias for each provider, then you reference the alias. With alias euc1 you'd reference with aws.euc1
Bogdan
Bogdanover 7 years ago
@Steven thanks! I'm already doing that, but pseudonimized it due to confidentiality reasons
Bogdan
Bogdanover 7 years ago
btw, I tried having the aws profiles in both ~/.aws/configure and ~/.aws/credentials but with no luck 😞
Steven
Stevenover 7 years ago
OH, different error. Sorry. You have 2 different errors. 1 not finding any credentials for some providers and 1 credential needing MFA, which is a different setup
Steven
Stevenover 7 years ago
1 sec. I'll grab an example
Bogdan
Bogdanover 7 years ago
all providers have different aliases
Steven
Stevenover 7 years ago
provider "aws" {
profile = "appzen-admin"
region = "us-east-1"
skip_credentials_validation = true
skip_get_ec2_platforms = true
skip_region_validation = true
}

# Provider for each account: dev, qa, shared,
provider "aws" {
alias = "dev"
profile = "appzen-admin"
region = "us-east-1"

assume_role {
role_arn = "arn:aws:iam::XXXXXXXXXXXX:role/OrganizationAccountAccessRole"
}
}

provider "aws" {
alias = "infra"
profile = "appzen-admin"
region = "us-east-1"

assume_role {
role_arn = "arn:aws:iam::XXXXXXXXXXXX:role/OrganizationAccountAccessRole"
}
}
Steven
Stevenover 7 years ago
This references a single profile in ./aws/config
Bogdan
Bogdanover 7 years ago
thanks @Steven but I don't have only 1 profile in ~/.aws/config I have 1 profile with static creds in ~/.aws/credentials then approx 5 dynamic profiles (which use source_profile to reference the static one in ~/.aws/credentials or another profile which is as assumed role)
For example, bob@account-users =assume=>developer@acccount-dev =assume=> prod-admin@account-prod can be defined in a dynamic profile called account-prod which refs the developer assumed role in the source_profile while dynamic profile account-dev refs the static profile account-users in its own source_profile = account-users
The profile I use across the providers is an assumed role which can assume all the roles in the other providers (at least via awscli)
loren
lorenover 7 years ago
i think there is a bug in terraform and the aws provider, where if you use something like source_profile in your aws config, then assume_role does not work in the provider config
loren
lorenover 7 years ago
they currently have two different logic paths for resolving credentials, and one works with assume_role in the provider, and one does not
loren
lorenover 7 years ago
we've submitted a patch that uses the same logic path to resolve credentials, waiting on review.... https://github.com/hashicorp/aws-sdk-go-base/pull/5
Steven
Stevenover 7 years ago
So, far I've been doing the assume role in either terraform provider or aws profile. But have not combined them as you're trying to do. I'd try simplifing the setup then adding the additional layers one at time. This will let you prove if it is a current limitation
rohit
rohitover 7 years ago
Is it possible to check if S3 bucket exist before creating one ?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
erik
erikover 7 years ago
Igor
Igorover 7 years ago
^^ Any initial thoughts? State diff is nice, I guess, but not sure if it's compelling enough at this point
shaiss
shaissover 7 years ago
I’m experiencing an interesting issue with the lambda resource. when I provide the kms_key_arn hardcoded (or any other method), on the first terraform apply the lambda function is created without the kms key. when I run tf apply again, it now has the right cmk key. Is this some bug or am I missing something?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
terraform 0.12 checklist (punch list for everything you need to do to migrate to 0.12 #terraform-0_12 https://github.com/hashicorp/terraform/pull/21241
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago

The output of this tool is in GitHub-flavored Markdown format so it can easily be pasted into a Markdown-capable issue tracker, like GitHub issues. Here is an example of output from a tailored configuration I wrote to show off some of the different checklist item types:
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
neat idea
erik
erikover 7 years ago
loren
lorenover 7 years ago
nice tks!
sarkis
sarkisover 7 years ago
is there an eta for 0.12 yet?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
apparently ☝️ is the "last" release of 0.11.x
btai
btaiover 7 years ago
anyone here terraform eks and installing a chart w/ the helm provider?
btai
btaiover 7 years ago
btai
btaiover 7 years ago
part of me thinks i’m configuring it wrong since i’m the only one that has 👍️ that github issue in half a year
btai
btaiover 7 years ago
my helm provider config:
provider "helm" {
  install_tiller  = true
  tiller_image    = "<http://gcr.io/kubernetes-helm/tiller:v2.11.0|gcr.io/kubernetes-helm/tiller:v2.11.0>"
  service_account = "${kubernetes_service_account.tiller.metadata.0.name}"
  namespace       = "${kubernetes_service_account.tiller.metadata.0.namespace}"

  kubernetes {
    host = "${module.eks_cluster.endpoint}"
    cluster_ca_certificate = "${base64decode(module.eks_cluster.certificate_authority_data)}"
    token            = "${data.aws_eks_cluster_auth.eks.token}"
    load_config_file = false
  }
}
btai
btaiover 7 years ago
I’ve attempted to comment out the token, but then i get an Unauthorized error
dalekurt
dalekurtover 7 years ago
Will doing a make clean on the reference-architecture break things. I’m having an issue completing a make children where it fails on the Security account.
C
Cloud Posseover 7 years ago
Join us for "Office Hours" every Wednesday 11:30AM (PST, GMT-7).

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Mar 20, 2019 11:30AM.

👉️ Add it to your calendar

:zoom: https://zoom.us/j/684901853

:slack: #office-hours (our channel)
antonbabenko
antonbabenkoover 7 years ago
Hi guys! 28th of May, I will speak about Terraform AWS modules and some of best-practices on AWS meetup in Mountain View. RSVP here https://www.meetup.com/awsgurus/events/261055503/ . Please share and join. If you are local around SF Bay Area, use Terraform, and want to meet and (maybe) like coffee as much as I do - we have to meet 26-28th of May 🙂
johncblandii
johncblandiiover 7 years ago
Anyone have a good resource for blue/green deployments with CodeDeploy?
GiriVardhanKumar
GiriVardhanKumarover 7 years ago
Hi , I want to create Logic App using terraform. The script should create "blank logic app" and should mimic the "Logic apps designer" . Is this possible using Terraform
cabrinha
cabrinhaover 7 years ago
Hello all, I'm getting into the "random_string" resource and reading the random provider docs here: https://www.terraform.io/docs/providers/random/index.html#resource-quot-keepers-quot-
cabrinha
cabrinhaover 7 years ago
I'm using "keepers" in order to decide when to generate a new string ... but I'm getting some weird errors:
* module.ecs-service.random_string.ecs-service-suffix: keepers (ordered_placement_strategy): '' expected type 'string', got unconvertible type '[]interface {}'
cabrinha
cabrinhaover 7 years ago
Do all the values in the "keepers" map need to be of type string?
loren
lorenover 7 years ago
yes
cabrinha
cabrinhaover 7 years ago
damn
cabrinha
cabrinhaover 7 years ago
My issue is that ecs_service resources are idempotent
cabrinha
cabrinhaover 7 years ago
I want to be able to create_before_destroy on an ecs_service ... but two ecs services can't have the same name
cabrinha
cabrinhaover 7 years ago
I'm solving this by appending a random string to the end of the ecs service name
loren
lorenover 7 years ago
the doc does say "arbitrary" keys/values, but that error is definitely indicating it must be a string
cabrinha
cabrinhaover 7 years ago
I'd like to use random_string.keepers to watch all the attributes that would force the recreation of the ecs service
cabrinha
cabrinhaover 7 years ago(edited)
ordered_placement_strategy is a list of map
loren
lorenover 7 years ago
if you have a list, you can try join(" ", <list>) or somesuch
loren
lorenover 7 years ago
oi
loren
lorenover 7 years ago
list of maps
cabrinha
cabrinhaover 7 years ago
yeah, so i need to convert that list of maps to a string? then back into a list of maps when I'm actually using it?
loren
lorenover 7 years ago
well no need to convert back as you'd still have the original
loren
lorenover 7 years ago
but i think otherwise yes
cabrinha
cabrinhaover 7 years ago
I need to convert it back because it needs to be passed through the random resource
cabrinha
cabrinhaover 7 years ago(edited)

                               # ["${var.ordered-placement-strategy}"]
ordered_placement_strategy =  "${random_string.default.keepers.ordered-placement-strat}" 
cabrinha
cabrinhaover 7 years ago
if that makes sense
cabrinha
cabrinhaover 7 years ago
not sure how to convert a list of maps to a string though
loren
lorenover 7 years ago
you could use depends_on instead to preserve the tree
loren
lorenover 7 years ago
yeah, checking the source and the schema looks like it's just a TypeMap... https://github.com/terraform-providers/terraform-provider-random/blob/master/random/resource_string.go#L21
loren
lorenover 7 years ago
which should allow nested lists...
loren
lorenover 7 years ago
anyway gotta run, good luck!
cabrinha
cabrinhaover 7 years ago
Yeah, so jsonencode() worked to convert the list of map to a string, but converting it back is going to be the tough part
cabrinha
cabrinhaover 7 years ago
still willing to entertain solutions 😅
loren
lorenover 7 years ago
maybe an external data source to do the conversion?
cabrinha
cabrinhaover 7 years ago
hm
cabrinha
cabrinhaover 7 years ago
does an update to "ordered_placement_strategy" force a new ecs_service resource?
cabrinha
cabrinhaover 7 years ago
I believe it does ...
cabrinha
cabrinhaover 7 years ago
The external data source docs seem to be a little ... confusing to me
cabrinha
cabrinhaover 7 years ago
well uh
cabrinha
cabrinhaover 7 years ago
I got it figured out
cabrinha
cabrinhaover 7 years ago
by not passing it through the random_string resource
Bogdan
Bogdanover 7 years ago
hey everyone! Is there an easy way to also store/export/save apply outputs to SSM Parameter Store? The main reason being so that they’re consumed by other tools frameworks which are non-Terraform?
Ayo Bami
Ayo Bamiover 7 years ago
HI All has anyone being able to use https://github.com/cloudposse/terraform-aws-key-pair module
Ayo Bami
Ayo Bamiover 7 years ago
I would like to see an example of how you created the keypair resource from the module.
Ayo Bami
Ayo Bamiover 7 years ago
This is how I used it but it fails because the key isn’t generated yet.
Dan
Danover 7 years ago
Is anyone doing blue/green deployments in AWS/GCP etc with Packer and Terraform?
loren
lorenover 7 years ago
any ideas on how to jsonencode every value in a list, when the values may not be strings?

* local.encoded: local.encoded: formatlist: list has non-string element (string) in:

${formatlist("${jsonencode("%v")}", local.values)}
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Public/Free Office Hours with Cloud Posse starting now!!

https://zoom.us/j/684901853
loren
lorenover 7 years ago
loren
lorenover 7 years ago
oh, and 0.12 has a jsondecode function, nice
Stephen Lawrence
Stephen Lawrenceover 7 years ago
Trying to stand up an eks cluster. Everything seems ok but I see this in the events:
Stephen Lawrence
Stephen Lawrenceover 7 years ago
kube-system 0s Warning FailedScheduling pod/coredns-7f66c6c4b9-v5g7h no nodes available to schedule pods
nutellinoit
nutellinoitover 7 years ago
terraform 0.12 is here
mrwacky
mrwackyover 7 years ago
I look forward to never being able to upgrade because we use >=3 seemingly unsupported providers that may never be upgraded ;(
PePe Amengual
PePe Amengualover 7 years ago
Hi guys, we have been using some of your modules and first of all Thanks you for all the hard work
PePe Amengual
PePe Amengualover 7 years ago(edited)
PePe Amengual
PePe Amengualover 7 years ago
what is the purpose of it ? I get the naming convention idea but having to call the module for every instance seems weird
PePe Amengual
PePe Amengualover 7 years ago
I think I fail to understand the use case
mrwacky
mrwackyover 7 years ago
my team uses (a forked version of) this to ensure that we have consistent tags on AWS resources
PePe Amengual
PePe Amengualover 7 years ago
ohhh ok, we use a map variable and add var.project-name to the Name tag
PePe Amengual
PePe Amengualover 7 years ago
like so :
tags      = "${merge(var.resource_tags, map("Name", "support-iq"))}"
PePe Amengual
PePe Amengualover 7 years ago
where support-iq could be var.project-name too
mrwacky
mrwackyover 7 years ago
yep: tags = "${module.label.tags}"
mrwacky
mrwackyover 7 years ago
But our version adds in a couple extra tags we want on our resources as well
mrwacky
mrwackyover 7 years ago
If we were smarter, we probably could have just wrapped it in another module that adds our extra mojo and turtles all the way down...
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
the label module is for 1) consistent and unique resource ID generation; 2) consistent tagging
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago(edited)
if you want to name your resources (across regions, accounts, companies) consistently and uniquely, you need to come up with something similar to namespace(org)-stage-name-additionalAttributes
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
- is the delimiter, also configurable
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
instead of doing it manually for every module, we invoke the label module to do it
PePe Amengual
PePe Amengualover 7 years ago
ohhhh I c ok, so is like I created my own module to create the map I use in all my modules
PePe Amengual
PePe Amengualover 7 years ago
to avoid duplication of that code and sanitize etc
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
yes
PePe Amengual
PePe Amengualover 7 years ago
I see now
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago(edited)
second, uniqueness across companies and accounts, especially for global resources like S3 buckets
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
so we can name our buckets in diff environments: cp-prod-app, cp-staging-app, cp-dev-app
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
you can name your buckets using the same modules: pepe-prod-app, pepe-staging-app, pepe-dev-app
PePe Amengual
PePe Amengualover 7 years ago
exactly
PePe Amengual
PePe Amengualover 7 years ago
cool, I get it
PePe Amengual
PePe Amengualover 7 years ago
I have another question: We have many aws accounts per team each account will have a number of vpc peered to the monitoring account and in different regions too, we have been thinking on creating accounts automatically and create vpcs and peer them automatically so in that case the S3/dynamo state file will be owned by the automation system atlantis/jenkins ( or whatever) but then each team have admin access to their accounts and start creating whatever they need ( ECS, EKS, fargate etc) on top of those pre-created vpcs so we are trying to find a good aproach to share/query the statefile that created the account to be able to find the vpcs IDs, subnets etc but I was thinking that is using the label module and adding consistent naming we could just use a data resource and just find the vpc base on the names, what do you guys do in this cases ?
PePe Amengual
PePe Amengualover 7 years ago
I have seen examples where everything is created in the same main.tf vpc+infra+policies etc but in our case we want to layer that so that the user just create the stuff they need for their app to run
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
will work on the rest of the new features for terraform-lsp on the weekend
PePe Amengual
PePe Amengualover 7 years ago
we were thinking to use remote states but is a bit more complicated for us since we all assumerole to different accounts and each team own and support their infra but not the vpc or role settings so if we use the “management” account to create the remote state s3 bucket then that will mean we will have to create an read only IAM role for each team to be able to read the state file
PePe Amengual
PePe Amengualover 7 years ago
but maybe that might not such a bad idea
PePe Amengual
PePe Amengualover 7 years ago
any of you have examples of a main.tf that can read state from one s3 bucket and save the state in another one ?
PePe Amengual
PePe Amengualover 7 years ago
we do not want people to be able to modify the state file that was created by us “SRE” team
loren
lorenover 7 years ago
in the terraform config that you manage, you can also push values into an s3 bucket or ssm path, which you then grant their roles access to based on key/path. then they do not need access to your tfstate
loren
lorenover 7 years ago
that can be ssm/s3 in their account, or in yours, with granular resource-based policies
David Nolan
David Nolanover 7 years ago
What about an "exports" terraform configuration, managed by SRE, which can read your main state files and defines outputs to map the important bits from those files into its own state. Then that state file is stored in a bucket you grant everyone who needs it read-only access to. That way you're not exposing any secrets that might be stored in the actual state files, only a small set of chosen outputs which you propagate through to where others can read them.
PePe Amengual
PePe Amengualover 7 years ago
Two very good ideas
PePe Amengual
PePe Amengualover 7 years ago
When I worked at EA we used Consul to store all this so then it was very easy to query the K/V for those values
David Nolan
David Nolanover 7 years ago
You could even populate that with terraform directly
PePe Amengual
PePe Amengualover 7 years ago
exactly, but here we do not use Consul so SSM/KMS is a pretty good option
PePe Amengual
PePe Amengualover 7 years ago
@David Nolan so your idea is to use outputs to create another state file ?
PePe Amengual
PePe Amengualover 7 years ago
it will be cool if you can export object values to something
PePe Amengual
PePe Amengualover 7 years ago
like module.vpc
PePe Amengual
PePe Amengualover 7 years ago
so all attributes are available
PePe Amengual
PePe Amengualover 7 years ago
without having to do one by one
David Nolan
David Nolanover 7 years ago
If there is risk in either granting "everyone" read access to the existing state files, or other contents in the same buckets, then an intermediary tf configuration to essentially export those values that matter should work
PePe Amengual
PePe Amengualover 7 years ago
but how do you export part of an state file ?
David Nolan
David Nolanover 7 years ago
output "foo" { value = "${data.terraform_remote_state.some_output_name}" }
PePe Amengual
PePe Amengualover 7 years ago
and then
terraform output foo
?
PePe Amengual
PePe Amengualover 7 years ago
to json or something ?
David Nolan
David Nolanover 7 years ago
and then terraform apply, and your state file will contain that output.
David Nolan
David Nolanover 7 years ago(edited)
then in the other group's terraform configs they also use a remote state file to read from the imtermediary state file and all they will be able to see is the values you chose to output to them.
David Nolan
David Nolanover 7 years ago
whereas if they can read the original state file they could fetch it directly and read everything.
PePe Amengual
PePe Amengualover 7 years ago
and the resource
terraform_remote_state
is just a file on s3 ?
David Nolan
David Nolanover 7 years ago
yes, in a bucket you grant the other teams access to
PePe Amengual
PePe Amengualover 7 years ago
sorry I never done this before, so excuse the stupid questions
David Nolan
David Nolanover 7 years ago
no worries.
David Nolan
David Nolanover 7 years ago
its a bit convoluted for sure
David Nolan
David Nolanover 7 years ago
There would end up being 3 tiers of terraform configs.
- A) Your central SRE configs that manage the VPC, etc.
- B) A second tier of "exported" configs which merely expose values from tier A to tier C
- C) The configs managed by other teams which pull from B as a data source
PePe Amengual
PePe Amengualover 7 years ago(edited)
ok if I understand this correctly : my SRE main.tf will have a remote state that is the one I do not want to share and I will create in the team’s owned bucket a output file that will be populated by
output "foo" { value = "${data.terraform_remote_state.some_output_name}" }
type of outputs and then the team’s project tf file will read that state file as an a intermediate file from which they will be able to read the objects attributes needed
David Nolan
David Nolanover 7 years ago
Replace 'output file' with 'a second statefile"
PePe Amengual
PePe Amengualover 7 years ago
correct
David Nolan
David Nolanover 7 years ago
I think you're getting it
David Nolan
David Nolanover 7 years ago
If your team has write permissions to their bucket this is even easier, You just write the export module and store the state in their bucket.
PePe Amengual
PePe Amengualover 7 years ago
I did not know you can output a object
David Nolan
David Nolanover 7 years ago
I was assuming one global "exported state" bucket which you would grant every team read access to
PePe Amengual
PePe Amengualover 7 years ago
I though where only individual attributes
David Nolan
David Nolanover 7 years ago
I believe you can output strings, lists and maps. Anything you can store in a variable basically.
David Nolan
David Nolanover 7 years ago
This isn't outputting the entire object, just copying an output from A to B so it can be read by C
PePe Amengual
PePe Amengualover 7 years ago
ohhh I c ok
David Nolan
David Nolanover 7 years ago
In TF 0.12 this is more clear because what you see in the datasource structure is ${data.terraform_remote_state.outputs.some_output_name}
PePe Amengual
PePe Amengualover 7 years ago
I was just reading that
PePe Amengual
PePe Amengualover 7 years ago
but that needs to read the SRE state file anyways
PePe Amengual
PePe Amengualover 7 years ago
data "terraform_remote_state" "vpc" {
  backend = "atlas"
  config {
    name = "hashicorp/vpc-prod"
  }
}
David Nolan
David Nolanover 7 years ago
The change in the data source object naming I think is it make it clear that only explicit outputs are available. But it also means they could add additional functionality side by side later.
PePe Amengual
PePe Amengualover 7 years ago
this looks like a regression to me :
PePe Amengual
PePe Amengualover 7 years ago
# Terraform >= 0.12
resource "aws_instance" "foo" {
  # ...
  subnet_id = "${data.terraform_remote_state.vpc.outputs.subnet_id}"
}

# Terraform <= 0.11
resource "aws_instance" "foo" {
  # ...
  subnet_id = "${data.terraform_remote_state.vpc.subnet_id}"
}
PePe Amengual
PePe Amengualover 7 years ago
in 0.11 you could use the object
PePe Amengual
PePe Amengualover 7 years ago
in 0.12 only outputs
PePe Amengual
PePe Amengualover 7 years ago
weird
David Nolan
David Nolanover 7 years ago
no it was always just the outputs, but the naming made it look like it was the object
David Nolan
David Nolanover 7 years ago
the object would have been data.terraform_remote_state.aws_vpc.some_name.id or data.terraform_remote_state.aws_subnet.some_name.id
David Nolan
David Nolanover 7 years ago
subnet_id is already a defined output from the terraform config, but in 0.12 they changed where that is exposed in a terraform_remote_state data source in order to make that clear.
PePe Amengual
PePe Amengualover 7 years ago
ohhh I see ok
David Nolan
David Nolanover 7 years ago
I think the confusion you had is the primary reason they changed it, but I also think it allows them to in the future expose the entire object.
PePe Amengual
PePe Amengualover 7 years ago
exactly
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
@Andriy Knysh (Cloud Posse) question, I am planning out what else is needed for lsp, and want to ask, do you guys care about completion + inspection of remote state?
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
(using terraform_remote_state)
Aleksandar
Aleksandarover 7 years ago
Steven
Stevenover 7 years ago
@Aleksandar you're using v0.12. I don't think any of the cloudposse modules have been upgraded for that yet
David Nolan
David Nolanover 7 years ago
Should a version marker be added to the cloudposse modules? And maybe start creating branches for 0.12 support
Aleksandar
Aleksandarover 7 years ago
Aleksandar
Aleksandarover 7 years ago
is it possible to configure option Restrict Bucket Access On in cloudfront origin?
PePe Amengual
PePe Amengualover 7 years ago
@Aleksandar AFAIK is not possible but it could have changed recently , when we tried AWS said we need to open up the origin to pretty much every single aws ip that very well could be an attacker so we never mode to cloudfront because of that
C
Cloud Posseover 7 years ago
Join us for "Office Hours" every Wednesday 11:30AM (PST, GMT-7).

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Mar 20, 2019 11:30AM.

👉️ Add it to your calendar

:zoom: https://zoom.us/j/684901853

:slack: #office-hours (our channel)
Tim Malone
Tim Maloneover 7 years ago
Posted this in another Slack last night but didn’t receive any answers, so thought I’d ask here too... is there any way to disable marking certain attributes as sensitive in plan output?
i want to see what this is going to be changed to 😛

Terraform will perform the following actions:

  ~ module.server_ini.aws_ssm_parameter.main
      value: <sensitive> => <sensitive> (attribute changed)

Plan: 0 to add, 1 to change, 0 to destroy.
PePe Amengual
PePe Amengualover 7 years ago
I think I found a bug in the docs for this module and some examples
PePe Amengual
PePe Amengualover 7 years ago
the readme says
PePe Amengual
PePe Amengualover 7 years ago
private_subnet_ids        = ["xxxxx", "yyyyy", "zzzzz"]
PePe Amengual
PePe Amengualover 7 years ago(edited)
but the variable input name is subnet_ids
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
@PePe Amengual please open an issue
PePe Amengual
PePe Amengualover 7 years ago
no problem
PePe Amengual
PePe Amengualover 7 years ago
done
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
thanks!
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
I've integrated our test-harness into terraform-null-label as well as added a basic terratest implementation.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
the test-harness is using bats
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
this is one step in the direction for supporting 0.12. basically, as we undertake this massive effort, I want to introduce better testing.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
next step, open a PR to port terraform-null-label 😃
evgmoskalenko
evgmoskalenkoover 7 years ago(edited)
Hello, could you tell me please. Wthat I doing wrong? Where my mistake? 🙂 My code looks like
evgmoskalenko
evgmoskalenkoover 7 years ago
evgmoskalenko
evgmoskalenkoover 7 years ago(edited)
- database is created
- using the ReadWrite user, I can create a table and I can execute INSERT query into it

CREATE TABLE employee(phone VARCHAR(32), firstname VARCHAR(32), lastname VARCHAR(32), address VARCHAR(64), company VARCHAR(32));


INSERT INTO employee(phone, firstname, lastname, address, company) VALUES(\'+10000000000\', \'John\', \'Doe\', \'Country\', \'Company Ltd.\');


But using the user ReadOnly, I can’t execute a SELECT query from the database. Error: permission denied for relation employee

SELECT * FROM employee ORDER BY lastname;


The logic is as follows:
1. create an owner role
2. Create a database with this owner
3. Create the group role ReadWrite, ReadOnly (login = false)
4. Give rights for the database for the sequence and table of these roles from item #3
5. Create the user ReadWrite, ReadOnly and add to the groups role from item #3

Try on AWS RDS (PostgreSQL)
Ayo Bami
Ayo Bamiover 7 years ago
Anyone know of the new format couldn’t wrap my head around the documentation.
Ayo Bami
Ayo Bamiover 7 years ago
something to do with dynamic
chris
chrisover 7 years ago(edited)
Hi guys, can anyone enlighten me as to why cloudposse / terraform-aws-cloudtrail doesn’t implement an enabled flag ? turns out we have a need to selectively enable a cloudtrail in certain accounts, and i thought the enabled flag was kind of the pattern here….?
PePe Amengual
PePe Amengualover 7 years ago
Hi Guys, I was playing with and I ended up forking since I plan to use CodeDeploy but even before that I have a problem with the ALB resource, since it is not done creating before the target group
aws_ecs_service.bluegreen: InvalidParameterException: The target group with targetGroupArn arn:aws:elasticloadbalancing:us-east-1:4444444:targetgroup/dev-fargateecs-app/07be25bc125ab2a0 does not have an associated load balancer.
so i was wonder if could do something to hack a depends_on at the module level ?
PePe Amengual
PePe Amengualover 7 years ago(edited)
Another question : https://github.com/cloudposse/terraform-aws-ecs-alb-service-task/blob/master/main.tf#L192 I can see you guys define only egress and icmp ingress for the service-task but and you pass a list of SGs defined from : var.security_group_ids that in here : https://github.com/cloudposse/terraform-aws-ecs-web-app/blob/master/examples/without_authentication/main.tf#L116 it comes from the defaul vpc security group
PePe Amengual
PePe Amengualover 7 years ago
But why to use the default vpc security group instead of :
resource "aws_security_group_rule" "ecs-servicetask-allow-alb-ingress" {
  security_group_id = "${aws_security_group.ecs_service.id}"

  type = "ingress"
  from_port = 0
  to_port = 0
  protocol = "-1"

  source_security_group_id = "${var.security_group_ids.[0]}"
PePe Amengual
PePe Amengualover 7 years ago
where the var.security_group_ids.[0] is = to the aws_alb SG id
PePe Amengual
PePe Amengualover 7 years ago
isn’t that more secure ?
PePe Amengual
PePe Amengualover 7 years ago
we usually delete the default VPC SG always
PePe Amengual
PePe Amengualover 7 years ago
but I wonder you guys deploy a VPC per app and that is why prefer to use the default vpc sg instead
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
@Andriy Knysh (Cloud Posse): @Daren was wondering if there's an easy fix for https://github.com/cloudposse/terraform-aws-s3-bucket/issues/11
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
(not urgent)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
public #office-hours starting now! join us here: https://zoom.us/j/684901853
Julio Tain Sueiras
Julio Tain Sueirasover 7 years ago
@Andriy Knysh (Cloud Posse) so intellij-hcl released 0.7.0 for HCL2, but I tested it, and it look like it does not support for each completion, dynamic content completion, nested variables completion, etc
Vidhi Virmani
Vidhi Virmaniover 7 years ago
Hi everyone,

I was using this module terraform-aws-modules/vpc/aws with terraform version 0.12 so it gave the error
Error parsing .terraform/modules/359629d31c12c09f870d559d03898da7/terraform-aws-modules-terraform-aws-vpc-e99089a/main.tf: At 2:23: Unknown token: 2:23 IDENT max
but then I read the issue https://github.com/terraform-aws-modules/terraform-aws-vpc/issues/267 so I changes the module version to ~>v1.66 after that I have started getting some different errors descibed in the snippet.
Vidhi Virmani
Vidhi Virmaniover 7 years ago
Vidhi Virmani
Vidhi Virmaniover 7 years ago
I am using terraform enterprise and tried downgrading the terraform version but it didn't work.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
@Vidhi Virmani ask in #terraform-aws-modules
Bogdan
Bogdanover 7 years ago
I’m getting both terragrunt and terraform stuck at random_string.bucket_prefix: Refreshing state... (ID: none) when trying to destroy a bunch of terraform-aws-codebuild modules - https://github.com/cloudposse/terraform-aws-codebuild
Bogdan
Bogdanover 7 years ago
@Andriy Knysh (Cloud Posse) @Erik Osterman (Cloud Posse) @Igor Rodionov @Jamie Nelson (BitFlight) do any of you know anything about this?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Can be a network issue or something similar. Try again. What terraform plan shows?
Bogdan
Bogdanover 7 years ago(edited)
@Andriy Knysh (Cloud Posse) solved it! I was call your modules from another/my wrapper module that used providers with profiles that required MFA. I spotted it in the logs
Bogdan
Bogdanover 7 years ago
thanks!