#terraform - November 2018

November 2018
onzyone
onzyoneabout 8 years ago
I have a few more questions ... how do I ref the output of a module in another module ...
onzyone
onzyoneabout 8 years ago
ie create kms key and then pass the arn to my postgres module ... then the postgres module pass the endpoint to my rout53 module
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago(edited)
onzyone
onzyoneabout 8 years ago
nice! got it!
onzyone
onzyoneabout 8 years ago
Thank you @Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
@johncblandii your PR looks good, merged to master, thanks
johncblandii
johncblandiiabout 8 years ago
thx
johncblandii
johncblandiiabout 8 years ago
releasing a version or going to stack a few?
johncblandii
johncblandiiabout 8 years ago
💥
onzyone
onzyoneabout 8 years ago(edited)
do you guys have an option on terragrunt?
joshmyers
joshmyersabout 8 years ago
AFAIK cloudposse/geodesic doesn't support terragrunt ootb, however you could easily open a PR to change the terraform path to that of terragrunt. The question is why would you want to? After using terragrunt before, and being happy with using in moderation, I'm unsure what you would want that geodesic doesn't give you. With the added fact that terraform 0.12 is soon landing (not withstanding the beta which is already out) that no doubt will have some backwards compatibility issues with terragrunt. Terragrunt was initially created back when TF was lacking in some of its orchestration. It has moved on since then with alot of this stuff now built in. Yes it still has some nice features that aren't in mainstream TF, but TF is diverging quicker than that value add IMO
joshmyers
joshmyersabout 8 years ago
Unpicking your terragrunt when it comes time to will likely be a headache
onzyone
onzyoneabout 8 years ago(edited)
we have just started our TF journey ... so I am playing around with a number of different approaches
joshmyers
joshmyersabout 8 years ago
The gruntwork TF testing tooling doesn't support Terragrunt (yet at least), and they wrote it.
onzyone
onzyoneabout 8 years ago(edited)
so what is the best way to do orchestration of multiple modules in todays landscape?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
@joshmyers nice points, thanks
joshmyers
joshmyersabout 8 years ago
If starting a new project id certainly look at using geodesic over terragrunt
joshmyers
joshmyersabout 8 years ago
Not taking anything away from the gruntwork folks, it has helped me loads in the past keeping code dry
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
terragrunt is a nice tool, and we use it sometimes
joshmyers
joshmyersabout 8 years ago
But geodesic does similar, in a cleaner way IMO
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
there are a few patterns of doing it
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
joshmyers
joshmyersabout 8 years ago
And only using terraform itself is a huge plus. Trust me saying you don't want to refactor the state machine for your infra.
joshmyers
joshmyersabout 8 years ago
What @Andriy Knysh (Cloud Posse) said
onzyone
onzyoneabout 8 years ago
@joshmyers @Andriy Knysh (Cloud Posse) these are really good points
onzyone
onzyoneabout 8 years ago(edited)
I like #6 we will need to look into this as well
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago(edited)
yea, as we mentioned, terragrunt is very nice, but it’s another wrapper on top of TF, and wrappers, although solve many problems, always have their own issues
onzyone
onzyoneabout 8 years ago
ya 😄 one can say the same thing about TF and CNF ...
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
you can
joshmyers
joshmyersabout 8 years ago(edited)
I'm not sure how I feel about modules of modules as it is rabbit holes all the way down and finding where you are and passing in more vars then you want is a pain
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
but HashiCorp is big, well funded, and is in business for a long time
joshmyers
joshmyersabout 8 years ago
But that is more of a terraform shortfall than the tooling around it
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
one day they will be bought
onzyone
onzyoneabout 8 years ago
I am not going against HashiCorp or TF, from what I have seen I really like it.
onzyone
onzyoneabout 8 years ago
and it is the tool that we are going to be using moving forward ...
onzyone
onzyoneabout 8 years ago
I am just trying to understand the best way to setup my modules
joshmyers
joshmyersabout 8 years ago
See how far something like geodesic gets you and go from there. I've been using TF from the early days, orchestrated many ways and as of now, geodesic and what the cloudposse folk are doing is the best I've seen, especially around orchestration and reusable modules
joshmyers
joshmyersabout 8 years ago(edited)
You probably only need to write your higher level modules if you look at what you can pick off their shelf
onzyone
onzyoneabout 8 years ago
😂 roles-profiles-modules ...brings be back a few years to my puppet days
joshmyers
joshmyersabout 8 years ago
Terragrunt gives you a slice of the pie, geodesic gives you the whole thing as far as orchestration, managing secrets, account separation, auditability
joshmyers
joshmyersabout 8 years ago
And if you have to unpick it, it's just using normal stuff, no terragrunt magic
joshmyers
joshmyersabout 8 years ago
Kinda.
onzyone
onzyoneabout 8 years ago
ya i am reading up on geodesic now
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
yea, we use geodesic everywhere, for many clients, and we support it
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
it’s not about our implementation per se, it’s about tooling and workflows, and not only for TF deployments - it supports tens of other tools including k8s, helm, helmfile etc.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
and security (assume IAM roles to access AWS)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago(edited)
and account/stage/env separation (as @joshmyers mentioned)
joshmyers
joshmyersabout 8 years ago
@Andriy Knysh (Cloud Posse) are ya'll using it with kops? Looks like it. How does TF generation fit in if at all?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Yea using it with kops
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
And EKS
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Give me 5 minutes
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Will show
onzyone
onzyoneabout 8 years ago
oh it has eks support!!!
onzyone
onzyoneabout 8 years ago
it is getting better now 😄
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
joshmyers
joshmyersabout 8 years ago
Yup, seen those. Look good. Was specifically wondering more about the kops generating your terraform resources etc in particular, if that was a thing you had looked into.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
geodesic does not include EKS support, you can use geodesic container to provision those
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
^ high level overview
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
if anyone interested, we can provide more details
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
more details 🙂 :
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago(edited)
Provision the Kops cluster

We create a kops cluster from a manifest.

The manifest template is located in [/templates/kops/default.yaml](https://github.com/cloudposse/geodesic/blob/master/rootfs/templates/kops/default.yaml)
and is compiled by running build-kops-manifest in the [Dockerfile](Dockerfile).

Provisioning a kops cluster takes three steps:

1. Provision the kops backend (config S3 bucket, cluster DNS zone, and SSH keypair to access the k8s masters and nodes) in Terraform
2. Update the [Dockerfile](Dockerfile) and rebuild/restart the geodesic shell to generate a kops manifest file
3. Execute the kops manifest file to create the kops cluster


Change directory to kops folder

cd /conf/kops


Run Terraform to provision the kops backend (S3 bucket, DNS zone, and SSH keypair)

init-terraform
terraform plan
terraform apply


From the Terraform outputs, copy the zone_name and bucket_name into the ENV vars KOPS_CLUSTER_NAME and KOPS_STATE_STORE in the [Dockerfile](Dockerfile).

The Dockerfile kops config should look like this:

docker
# kops config
ENV KOPS_CLUSTER_NAME="<http://us-west-2.staging.example.net|us-west-2.staging.example.net>"
ENV KOPS_DNS_ZONE=${KOPS_CLUSTER_NAME}
ENV KOPS_STATE_STORE="<s3://jexample-staging-kops-state>"
ENV KOPS_STATE_STORE_REGION="us-west-2"
ENV KOPS_AVAILABILITY_ZONES="us-west-2a,us-west-2b,us-west-2c"
ENV KOPS_BASTION_PUBLIC_NAME="bastion"
ENV BASTION_MACHINE_TYPE="t2.medium"
ENV MASTER_MACHINE_TYPE="t2.medium"
ENV NODE_MACHINE_TYPE="t2.medium"
ENV NODE_MAX_SIZE="2"
ENV NODE_MIN_SIZE="2"


Change directory to kops folder, init Terraform, and list files

cd /conf/kops
init-terraform


You will see the kops manifest file manifest.yaml generated.

Run kops create -f manifest.yaml to create the cluster (this will just create the cluster state and store it in the S3 bucket, but not the AWS resources for the cluster).

Run kops create secret sshpublickey admin -i /secrets/tf/ssh/example-staging-kops-us-west-2.pub --name $KOPS_CLUSTER_NAME to add the SSH public key to the cluster.

Run kops update cluster --yes to provision the AWS resources for the cluster.

All done. The kops cluster is now up and running.


_NOTE:_ If you want to change kops cluster settings (e.g. number of nodes, instance types, etc.):

1. Modify the kops settings in the [Dockerfile](Dockerfile)
2. Rebuild Docker image (make docker/build)
3. Run geodesic shell (<http://staging.example.net|staging.example.net>), assume role (assume-role) and change directory to kops folder
4. Run kops replace -f manifest.yaml to replace the cluster resources (update state)
5. Run kops update cluster --yes to modify the AWS resources for the cluster


_NOTE:_ To force a rolling update (replace the EC2 instances), run kops rolling-update cluster --yes --force


_NOTE:_ To use kops and kubectl commands (e.g. kubectl get nodes, kubectl get pods), you need to export the kubecfg configuration settings from the cluster.

https://github.com/kubernetes/kops/blob/master/docs/kubectl.md

Run kops export kubecfg $KOPS_CLUSTER_NAME to export kubecfg settings.

You need to do it every time before you work with the cluster (run kubectl or kops commands, validate cluster kops validate cluster, etc.) after it has been created.
btai
btaiabout 8 years ago
hi is anyone using terraform to deploy EKS/AKS clusters?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
moring @btai! yep, we are.... have you seen the cloudposse modules?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
btw, let's move to #terraform
btai
btaiabout 8 years ago
isnt this #terraform
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
hahaha
btai
btaiabout 8 years ago
😄
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
😃
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
See above
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
We posted links to the modules
btai
btaiabout 8 years ago
so i dont have a problem creating the modules
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
And working example which we provisioned
btai
btaiabout 8 years ago
I was wondering how you are managing upgrades (kubernetes version, node sizes, etc) with zero downtime. the only solution i know of is redeploying a new cluster and doing a cutover
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
You should not have any problem :)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Ask questions if you have
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
That’s is not easy yet with EKS
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
With kops, no problem
btai
btaiabout 8 years ago
yeah, so im not only on AWS but also in azure as well
btai
btaiabout 8 years ago
and kops doesn't have an azure solution (yet)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
yea, terraform is not well suited for managing the lifecycle
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
we still use kops for managing k8s on aws
btai
btaiabout 8 years ago
i see
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
there’s a tool by weaveworks attempting to be the kops of eks, but it’s not there yet
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
it doesn’t handle drain/cordon rolling updates like kops
btai
btaiabout 8 years ago
so kops upgrades the cluster with zero downtime
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
yup
btai
btaiabout 8 years ago
so i have a tentative solution that I can think of to automate the upgrade for me using just terraform
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
kops export kubecfg
kops replace -f manifest.yaml
kops update cluster
kops update cluster --yes
kops rolling-update cluster
kops rolling-update cluster --yes
btai
btaiabout 8 years ago
mind if i run it by you guys?
btai
btaiabout 8 years ago
just to get some feedback
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
yea, for sure - would like to hear
btai
btaiabout 8 years ago
cool
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
the approach I'm thinking of would be to use blue/green node pools
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
how does Azure AKS handle that?
btai
btaiabout 8 years ago(edited)
this is just a sample solution

# current running version
terraform workspace new kube-1.9.0
terraform apply -var 'kube-version=1.9.0'

# upgraded version
terraform workspace new kube-1.10.0
terraform  apply -var 'kube-version=1.10.0'

# cutover process + testing

# destroy old cluster
terraform workspace select kube-1.9.0
terraform destroy
clean up old state file (kube-1.9.0)
btai
btaiabout 8 years ago
the more robust solution would be to maintain a changelog of the cluster workspace so workspace: kube-v1 could be initial, workspace:kube-v2 could be upgrading node size (i.e. c4.large -> c4.xlarge), workspace:kube-v3 upgrades kube version from 1.9.0 to 1.10.0
btai
btaiabout 8 years ago
let me know if that makes sense
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
(on the phone - will be back in ~20)
btai
btaiabout 8 years ago
ok thanks!
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
hrm…. that’s a pretty cool notion
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
need to let that sink in for a moment.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
basically, you’re suggesting using “workspaces” as colors for the worker nodes
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
on the surface that seems like a pretty elegant solution. there still should be a drain+cordon step, but in terms of provisioning the new worker pool this should work
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
1. provision terraform-aws-eks-cluster in default workspace
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
2. provision terraform-aws-eks-workers in blue workspace
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
… new release of kubernetes …
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
3. provision terraform-aws-eks-workers in green workspace
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
4. drain cordon nodes in blue workspace, then destroy node pool. Something like this could maybe be done: https://github.com/dmathieu/dice
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@btai ☝️
btai
btaiabout 8 years ago
yep, i've used workspaces to deploy in multi-environment and multi-region and its worked really well using only one set of terraform files (keeping things dry)
btai
btaiabout 8 years ago
the workspace pattern i usually use is {environment}_{region}
btai
btaiabout 8 years ago
and you can use map variables to switch on the configs
btai
btaiabout 8 years ago
for my aks cluster, it will have to be a special pattern something like {environment}_{region}_{kube-version}
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
How is that different of having two autoscaling groups inside one 'workspace' with specified seperated min/max controlled by a conditional ?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
@Maarten van der Hoef it's a similar strategy, but this is more fluid
btai
btaiabout 8 years ago
its similar
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
you don't need to maintain 2 of everything
btai
btaiabout 8 years ago
yep
btai
btaiabout 8 years ago
i went with that route initially
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
you might need to have 2 GPU node pools, 2 high mem node pools, 2 high network node pools, etc
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
with workspaces, you can just have 1 of each, then have as many "colors" as necessary
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
Ok i see
btai
btaiabout 8 years ago
the problem i ran into with the 2 autoscaling groups solution is when I tried to put it in a fully automated CI job, I realized i needed to maintain the state of which blue or green ASG was currently taking traffic
btai
btaiabout 8 years ago(edited)
thanks @Erik Osterman (Cloud Posse)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Dan
Danabout 8 years ago
If you have multiple ingress rules when creating an SG... for terraform, would I just created multiple ingress brackets
Dan
Danabout 8 years ago
example:
Dan
Danabout 8 years ago
Dan
Danabout 8 years ago
but it doesnt have a good example of multiple ingress rules
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
@Dan yes, you add multiple ingress rules
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
and you need to (re)create the egress rule since TF deletes the default one https://github.com/cloudposse/terraform-aws-eks-cluster/blob/master/main.tf#L52
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago(edited)
but don’t mix the two ways in one module, and the first one (standalone resource) is preferred
Dan
Danabout 8 years ago
yea i liked the inline cause it has it all laid out by the specific gorup
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
but standalone is better 🙂
Dan
Danabout 8 years ago
sell me
Dan
Danabout 8 years ago
sell me on the standalone
Dan
Danabout 8 years ago
lol
Dan
Danabout 8 years ago
maybe im misunderstanding somthing
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
b/c:
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
1. in many cases it’s useful to expose the created SG to outside the module so you or someone else could add rules if needed (not possible if used inline)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
2. You can use counts to enable/disable rules based on conditions (not possible inline) https://github.com/cloudposse/terraform-aws-eks-cluster/blob/master/main.tf#L64
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
3. Since it’s a separate resource, could be controlled separately from TF
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
but I agree, the inline case looks better 🙂
Dan
Danabout 8 years ago
ah that makes sense... like if I add a port or rule... itll have to delete the whole sg and then recreate it rather then just adding that one rule?
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
AWS implementation of security groups has been done in a way that it's very hard for Terraform to know if a security group rule reflects to an inline inside a security group resource or if it is a seperate security_group_rule resource. Everything works without problems as long as people don't mix this up in Terraform.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
@Dan exactly
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
@Maarten van der Hoef 💯
Dan
Danabout 8 years ago
yea it would be nice if terraform had some "I'm aware" capability
Dan
Danabout 8 years ago
is the security_group_id = "${join("", aws_security_group.default.*.id)}"
Dan
Danabout 8 years ago
joinining it to the the sg
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago(edited)
it's not about that, the problem is that most resources you create have unique identifiers, security group rules don't.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
it’s a splat + join pattern 🙂
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
since the SG uses count
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago(edited)
if the SG is disabled (count=0), then `security_group_id = “${aws_security_group.default.*.id)}” would produce errors
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
b/c TF could not find it (it’s disabled)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
so we join it with an empty string
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
if the SG is disabled, the whole expression just produces an empty string
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
if you don’t use count, you don’t need it
Dan
Danabout 8 years ago
hmm
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
since the SG uses count, it is a list, not a single resource
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 8 years ago
even if it has just one item (or it does not, if count=0)
Gabe
Gabeabout 8 years ago
does anyone here have a strong argument for using terraform remote state vs a data lookup to get information on resources in different accounts?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Good question. We do both. No specific "best practices" related to it right now.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
i suppose it comes down to RBAC
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
to use remote state bucket, you need to grant full read access to that state file which may/may not expose secrets stored therein
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago(edited)
while if you usse the data providers you can use IAM to gate access without exposing anything sensitive in the statefiles
mrwacky
mrwackyabout 8 years ago
yawp, we're working to avoid secrets stored in state. In the end, we realized there's little value generating IAM access keys with TF if it just puts secrets in the state, and they are hard^Wnon-trivial to rotate
joshmyers
joshmyersabout 8 years ago
Depending on situation, I'd opt for data sources rather than remote state. Use case where had different remote state for lots of things including a DR environment. If the main environment goes down due to AWS issues for example, you can't terraform your DR environment if it relies on pulling state from S3 in a failed region
joshmyers
joshmyersabout 8 years ago
Have found a data lookup to be more reliable. Then again if you are working in a massive account which is constantly hitting API rate limiting even after AWS increasing, you may want the one hit request Vs potentially lots of data lookups
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
Which S3 are you using @joshmyers 🙂
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
I like Remote State, but learned to not chain remote states too much. Especially with Terraform updates this can be a huge pain. Pro's of remote state is that the "master"- state only needs to have the information filled in ones and the "slave"- states are updated.

Using datasources one would still need to replicate data a bit as the datasources need to be addressed. For never changing information like vpc_id that is probably fine, but it works just as well with remote state. Matter of taste then.
joshmyers
joshmyersabout 8 years ago
Pulling in 5 different remote states during a run, mostly from different accounts, on top of the actual state file the main resources was a pain point.
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
I was joking a bit, but availability of S3 as argument for not using remote state is of course a bit far fetched no.
joshmyers
joshmyersabout 8 years ago
Not when you are building for the government and the entire point was for a DR scenario due to an entire regional outage.
joshmyers
joshmyersabout 8 years ago
Most situations, yes :)
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
ok wow, wouldn't think something like that would happen. I remember one s3 outage 2 years ago in us-east-1 i think, but it was solved quickly and most of it was available.
joshmyers
joshmyersabout 8 years ago
Agreed, extremely unlikely but it has happened before and it was a scenario the client wanted a working DR environment.
joshmyers
joshmyersabout 8 years ago
AFAICR IAM policies on the KMS keys across a ccounts was also a pain when using the default KMS key. You couldn't change the policy on the default key and we had strict policy limits like source IPs and MFA etx
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
oh yes, i think I've struggled with that once
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
also AWS did not have any good guide on that online
joshmyers
joshmyersabout 8 years ago
Yeah, lost a day or so to that one. Cloudtrail shows source IP being the correct one, so policy should have been OK. In fact the request was coming from another internal service was failing, despite cloudtrail showing otherwise
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
ah nice..
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
Still it's quite cool that the US has something like GovCloud. Here in Europe governments are not so quick on moving to Cloud Services which are US HQ'ed.
Maarten van der Hoef
Maarten van der Hoefabout 8 years ago
So a lot of those projects end up on super expensive private clouds.
joshmyers
joshmyersabout 8 years ago
This is UK government. Large chunks of UK gov is on public cloud like Amazon these days
joshmyers
joshmyersabout 8 years ago
But yeah, the shitty private vcloud private clouds are shitty expensive
btai
btaiabout 8 years ago
so you would resort to only using local state?
joshmyers
joshmyersabout 8 years ago
Some teams used git and git-crypt
btai
btaiabout 8 years ago
git hosted on a private server?
joshmyers
joshmyersabout 8 years ago
Yup, gitlab/GitHub enterprise
btai
btaiabout 8 years ago
is it still hosted in the cloud?
btai
btaiabout 8 years ago(edited)
artifactory or consul for remote state works too
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
oh, interesting - didn't know about the artifactory backend
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 8 years ago
have you used it? @btai
btai
btaiabout 8 years ago
for terraform backend, no
btai
btaiabout 8 years ago
but i wanted to throw that out there because remote state gives you state locking
Kene Nnamani
Kene Nnamaniabout 8 years ago
I am sure we have seen the integration of Terraform with Google Cloud Shell. I love this feature. You can read my blog about it. https://medium.com/@nnamani.kenechukwu/terraform-in-gcloud-shell-learning-infrastructure-automation-made-simple-7ef4a4300ec4
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
@Kene Nnamani thanks for sharing, interesting article
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
woohoo! @Andriy Knysh (Cloud Posse) added serverless support to https://github.com/cloudposse/terraform-aws-rds-cluster
johncblandii
johncblandiialmost 8 years ago
no clue why the commits bloated, but the files changed are legit
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
@johncblandii thanks, merged to master
johncblandii
johncblandiialmost 8 years ago
no prob
btai
btaialmost 8 years ago
what are your thoughts on coupling your EKS cluster with a VPC, so when you do an upgrade to the cluster (spinning up a new cluster + cutover) you are also spinning up a new VPC with it
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago(edited)
blue/green clusters+vpc?
btai
btaialmost 8 years ago
yeah
btai
btaialmost 8 years ago
that vs blue/green cluster within same VPC
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
my biggest concern with that would just be ensuring security groups are kept current and allow both VPCs
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
at a certain scale, would maybe be better to consider an active/active setup where you can take one cluster out of commission for upgrades
btai
btaialmost 8 years ago
maintain two identical clusters at all time?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
yes, but they don’t need to be scaled up equally
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
but i find that if something’s not in use, you don’t know until it’s too late what doesn’t work.
joshmyers
joshmyersalmost 8 years ago
Some larger orgs I've worked at give you a vpc to operate in, nevermind all vpc setup like peering etc. I'd rather have that separated out
joshmyers
joshmyersalmost 8 years ago
IMO coupling to VPCs sounds like a recipe for a headache
joshmyers
joshmyersalmost 8 years ago
Is there a specific reason you'd want to do that?
btai
btaialmost 8 years ago
@joshmyers not a really good reason, the org im at currently follows that pattern (done with kops) which has worked for them
btai
btaialmost 8 years ago
i personally gravitate towards how you feel which is why i brought up the question
btai
btaialmost 8 years ago
we are actually going multi-cloud so I'm terraforming my azure clusters and its dependencies (vnet, subnets, etc) and I was debating whether to keep the same pattern vs doing the cutover within the vnet
btai
btaialmost 8 years ago
vnet = azure's vpc if youre unaware
joshmyers
joshmyersalmost 8 years ago
How do you manage terraform codebases for the different providers?
joshmyers
joshmyersalmost 8 years ago
Different runs, code, state files etc?
btai
btaialmost 8 years ago
yeah
btai
btaialmost 8 years ago
terraform repo with an aws sub-directory and an azure sub-directory
btai
btaialmost 8 years ago
its different providers so the terraform code is completely different
btai
btaialmost 8 years ago(edited)
but regardless terraform will allow us to manage multi-cloud infrastructure much easier
inactive
inactivealmost 8 years ago
hey guys, i’m having an issue when creating an EMR cluster using terraform… I have tried multiple things, none of them seem to stick
inactive
inactivealmost 8 years ago
here is the error message:
* aws_emr_cluster.a: Error waiting for EMR Cluster state to be "WAITING" or "RUNNING": TERMINATED_WITH_ERRORS: VALIDATION_ERROR: The VPC/subnet configuration was invalid: No route to any external sources detected in Route Table for Subnet: subnet-05df0470906f7ce15 for VPC: vpc-046d66b0ac8496fcd
inactive
inactivealmost 8 years ago
the interesting thing is, that if i run ‘terraform apply’ again, then it works
joshmyers
joshmyersalmost 8 years ago
Race condition?
inactive
inactivealmost 8 years ago
exactly
inactive
inactivealmost 8 years ago
the networking dependencies have not yet been fully created
inactive
inactivealmost 8 years ago
then it attempts to launch emr cluster prematurely
inactive
inactivealmost 8 years ago
i have tried adding “depends on” statements
inactive
inactivealmost 8 years ago
but no joy
inactive
inactivealmost 8 years ago
tried moving the networking resources to the main.tf file, but that doesn’t seem to make any difference…
joshmyers
joshmyersalmost 8 years ago
Maybe?
inactive
inactivealmost 8 years ago
wow — exactly my issue 🙂
inactive
inactivealmost 8 years ago
but… i don’t see a solution
joshmyers
joshmyersalmost 8 years ago
Without more info, hard to help
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
Use terraform apply -target module.vpc -target module.subnets to first provision the network
joshmyers
joshmyersalmost 8 years ago(edited)
If these are disposable things you will be destroying and recreating often, it's probably gonna bite again.
inactive
inactivealmost 8 years ago
ok, thanks aknysh… that may work — i’ll give it a try
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
Or move the network files into a separate folder and provision it first. Then in EMR module look up the vpc and subnets, or use a remote state
inactive
inactivealmost 8 years ago
so, in general practice, is it normal to provision some items first? and then run terraform apply again for the rest?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
If you have those issues, yes
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
You can do it in three different ways: use target, lookup data source, or remote state
joshmyers
joshmyersalmost 8 years ago
It happens but is not ideal. If depends on doesn't fix it there is likely a race condition AWS end in creation of the resource from when you call create and it is ready. Generally resources like this need to support polling the API until a thing is ready
inactive
inactivealmost 8 years ago
right — makes sense
inactive
inactivealmost 8 years ago
i suppose there is not a way to add a “sleep” or increase timeout setting?
inactive
inactivealmost 8 years ago
because i find that it if would just wait a few more seconds, then it would work
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
Yea, but not a good practice :). What if next time you need to wait for 7 seconds, but you have timeout only for 3
joshmyers
joshmyersalmost 8 years ago
p.s. I have no idea if that is the case for your route table resources etc as I haven’t looked into it
joshmyers
joshmyersalmost 8 years ago
If this is a one off thing, @Andriy Knysh (Cloud Posse) suggestions are the way to go
joshmyers
joshmyersalmost 8 years ago
If you destroy and re run this 10 times a day, that will get tiresome.
inactive
inactivealmost 8 years ago
yes, agreed —
i just ran the -target vpc and it worked
inactive
inactivealmost 8 years ago
this is an OK workaround for now, i guess — but would prefer that i could run it just once
inactive
inactivealmost 8 years ago
i will look into the other 2 options…
inactive
inactivealmost 8 years ago
i can also share my emr.tf if you care to take a peek..
inactive
inactivealmost 8 years ago
no — the route table is simply updated
joshmyers
joshmyersalmost 8 years ago
ah sorry, I probably misread
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
agree, all those workarounds are not ideal (better to just do terraform apply once). We had a few situations like this with complex module dependencies. So instead of spending days on trying to find a solution which might not even exists, just use -target 🙂
inactive
inactivealmost 8 years ago
i use vpc module to create the initial artifacts, enable nat gateway, and then on my emr.tf I create a new subnet and add it to an existing route table
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
why not create a VPC and subnets first, and then in the EMR module just use them?
inactive
inactivealmost 8 years ago
well i thought i did that by moving all of the networking resources to main.tf
inactive
inactivealmost 8 years ago
but, yes i think creating the VPC first with a targeted approach makes sense
inactive
inactivealmost 8 years ago
then run terraform apply a 2nd time…
inactive
inactivealmost 8 years ago
works for me
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
yea i think that when you create a subnet and add it to the route table, it takes time. And then EMR wants to use the route, but it does not exist yet. If you create all subnets first and then use one of them for EMR, should work
inactive
inactivealmost 8 years ago
agreed — thank you both for your input
inactive
inactivealmost 8 years ago
separate topic:
inactive
inactivealmost 8 years ago
i’m still trying to provision an internal load balancer from one of your modules, but not sure if that is supported
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
what module?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
ah yea, not supported yet, we have an issue opened for that
inactive
inactivealmost 8 years ago
ok — that’s fine
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
if you know how to do it, can open a PR?
inactive
inactivealmost 8 years ago
uhm… not sure, i guess if i were to manually create the load balancer
inactive
inactivealmost 8 years ago
but I’m relying on elastic beanstalk because of all of the automation that is included
inactive
inactivealmost 8 years ago
but i’ll keep digging, maybe I can figure it out
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
need to add a few settings, one of them is
setting {
    namespace = "aws:ec2:vpc"
    name      = "ELBScheme"
    value     = "internal"
 }
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
value = "internal" should be a variable
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
so we have already 4 settings in the aws:ec2:vpc namespace
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
need to add just this one
setting {
    namespace = "aws:ec2:vpc"
    name      = "ELBScheme"
    value     = "${var.elb_scheme}"
 }
inactive
inactivealmost 8 years ago
ok, great — looks quite straightforward
inactive
inactivealmost 8 years ago
thanks again for your help
lvh
lvhalmost 8 years ago
OK, so, Terraform.

Context: I have some infra (really mostly scheduled ECS tasks) that needs to repeat for multiple IAM roles (it's auditing infra, each IAM role represents a role in an account with auditing privileges, each scheduled task is a kind of audit). It seems terraform won't let me do that because no count in modules. Since cloudposse uses modules extensively, I'm guessing I'm not the first person to run into this problem. How do you solve it?
lvh
lvhalmost 8 years ago
Right now I'm solving it by having another program generate terraform for me, which is the most go thing I've done this week.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
@lvh sounds like generating TF files in this case is a good idea. What module are you using? Over what settings do you want to iterate? Maybe we could improve the module, or you could just use plain TF resources with counts
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
Well, our modules might not address this particular use-case, but that doesn’t preclude using modules
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
you might need some more specialized module that supports what you want to do
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
i’d need to see some more concrete code to visualize it (i’m more of a visual person)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
i’d avoid the code generation for as long as necessary
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
is there one cloudposse module in particular you’re referring to?
btai
btaialmost 8 years ago
terraform workspaces
btai
btaialmost 8 years ago
if you want to use the same terraform module with different variables
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
@btai that could be used
lvh
lvhalmost 8 years ago
Nah, it's not a single module that's causing problems, it's an entire file with multiple resources + cloudposse modules in it
lvh
lvhalmost 8 years ago
It has ECS tasks, scheduling, iam roles, etc etc
lvh
lvhalmost 8 years ago
I also have an input set of IAM roles; I need to replicate the entire structure for multiple IAM roles -- the obvious way to do that IMO would be to wrap the entire thing in a module and then have a count?
lvh
lvhalmost 8 years ago
But I guess I can just write out the module calls explciitly
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
hrm… the approach @btai suggests might be interesting
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago(edited)
anyone know if this may be easier in terraform 0.12? (the universal hail mary)
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
I like the @btai option, I guess @lvh has different customers to audit with different roles respectively. A workspace per customer sounds like a nice idea and scales easily. Wit a big variable map with the iam roles populated per customer makes it easy for the calling code to do a simple lookup.
btai
btaialmost 8 years ago(edited)
@lvh i would use terraform workspaces with variable maps
btai
btaialmost 8 years ago

module "some_random_module" {
  source = "../../modules/some-random-module"

  name                 = "${var.name_map[terraform.workspace]}"
}



variable "name_map" {
  type = "map"

  default = {
    "customer_a" = "customer_a_name"
    "customer_b" = "customer_b_name"
  }
}



terraform workspace new customer_a
terraform workspace new customer_b
lvh
lvhalmost 8 years ago
I do have multiple customers but each customer has separate IAM roles, because each customer has multiple accounts
lvh
lvhalmost 8 years ago
i already have an AWS acct per customer though? so I guess I can do a ... workspace per customer account
lvh
lvhalmost 8 years ago
which seems annoying but ok it'll work
lvh
lvhalmost 8 years ago
thanks!
lvh
lvhalmost 8 years ago
@Erik Osterman (Cloud Posse) and no, 0.12 won't fix this because they've given up on making it work for resources and modules
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago(edited)
oh really! have you seen them say that "count" has no ETA of ever making it into tf modules?
lvh
lvhalmost 8 years ago
sorry I meant for 0.12; they claim to have laid a lot of the ground work for it? but it seems very very far away, and they already said 0.12 wont have it
lvh
lvhalmost 8 years ago
just the comprehensions (maps, lists)
lvh
lvhalmost 8 years ago
it might still happen eventually, but not any time soon
joshmyers
joshmyersalmost 8 years ago
You should totally have different state files per customer if not already
joshmyers
joshmyersalmost 8 years ago
Until 0.12 maps are good for what you suggest but only 1 level deep so no rich data there without nasty hacks
joshmyers
joshmyersalmost 8 years ago
So I'd probably do a workspace per customer per env?
btai
btaialmost 8 years ago
^ ++
btai
btaialmost 8 years ago
for the time being ive been using the workspace to switch on multiple
btai
btaialmost 8 years ago
{environment}_{region}
btai
btaialmost 8 years ago
and then split the workspace on _
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
Anyone having issues with security groups with v0.11.10 ?
Gonzalo Peci
Gonzalo Pecialmost 8 years ago
What sort of issues?
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
I think it's more related to anton's module, will deep dive later
jonboulle
jonboullealmost 8 years ago
is this a 0 -> -1 change?
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
yes
jonboulle
jonboullealmost 8 years ago
I wondered about that - an AWS API change? I had a similar one last week with something eles
loren
lorenalmost 8 years ago
Amos
Amosalmost 8 years ago
I asked a question over in #announcements concerning one of your terraform project, but am thinking this is probably the more correct place to do it. Should I move that question over here?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
nah, we can just link it here
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
we've not tried to set up inter-region tunnels with that module
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago(edited)
our use-case was to peer a backing services VPC (e.g. one with RDS, ElastiCache, etc) with a kops vpc
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
I suspect the reason it's not working is that the module will need to have multiple providers so that it can reference two different regions
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
I feel like someone submitted PR for another module that does this, but I can't find which repo that was in (it was not related to vpc peering)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
(also, we implemented the module before inter-region VPC peering was possible)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
(jumping on a call)
Amos
Amosalmost 8 years ago
I see! thanks for the info. I will attempt to fork it and try it out.
Amos
Amosalmost 8 years ago
This is similar to my use case as well, but I have additional VPCs, like a general infra VPC and a VPN VPC I need to peer as well (one of which is in a different region).
Suresh
Sureshalmost 8 years ago
Suresh
Sureshalmost 8 years ago
I am trying to use ELB health for the ASG instead of EC2 health check!
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
Any VI users in the house? Give me your thumbs up 🙂
https://github.com/scrooloose/nerdcommenter/pull/350
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
done
Nikola Velkovski
Nikola Velkovskialmost 8 years ago
Done, I've been using https://github.com/tpope/vim-commentary for a long time and haven't had any issues so far.
Gabe
Gabealmost 8 years ago
do you guys use any tools for autocompletion for terraform on vim?
mpmsimo
mpmsimoalmost 8 years ago
An honorable mention, not specifically for autocompletion but some other terraform nice to haves:
https://github.com/hashivim/vim-terraform
nutellinoit
nutellinoitalmost 8 years ago
Hi everyone, i’m using https://github.com/cloudposse/terraform-aws-vpc-peering to setup peering connection between two vpcs
nutellinoit
nutellinoitalmost 8 years ago
the first time i run apply, everything is created correctly
nutellinoit
nutellinoitalmost 8 years ago
the second time, terraform deletes routes on the two route tables of the two vpcs
nutellinoit
nutellinoitalmost 8 years ago
the third, the routes are recreated , and so on
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
@nutellinoit it’s been a few months since we tested our implementation here https://github.com/cloudposse/terraform-root-modules/blob/master/aws/kops-aws-platform/vpc-peering.tf
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
can you confirm that you use similar setup (we did not see the issue you described)
nutellinoit
nutellinoitalmost 8 years ago
uhm
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
but maybe something changed in TF or other things
nutellinoit
nutellinoitalmost 8 years ago
investigating the issue i think it’s related to another vpc module that i’m using, the one from hashicorp aws best practice
nutellinoit
nutellinoitalmost 8 years ago
i’ll get in touch if this is the case
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
yea thanks. you can try our vpc module and see what happens
nutellinoit
nutellinoitalmost 8 years ago
found it, the conflict is caused by inline base rule on the aws_route_table resource (like the 0.0.0.0/0 for the igw)
nutellinoit
nutellinoitalmost 8 years ago
removing the route inline and using aws_route resource the problem is gone!
Tee
Teealmost 8 years ago
Is it possible to create a private dns route 53 address and resolve it as static website in web browser internally behind vpn with internal elb using terraform or cloudformation
joshmyers
joshmyersalmost 8 years ago
@Tee what problem are you trying to solve? Terraform / CF are just tools which talk to the AWS API. Is this possible via the API?
Tee
Teealmost 8 years ago
Yes. So i want a simple webpage that is only available behind vpn using internal dns. https://internal.example.com so when i open link in web browser i can see content only when connected to vpn. As it just internal route 53 private domain resolved behind VPN
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
@Tee I've responded to you in #aws
Tee
Teealmost 8 years ago
Thanks
mmarseglia
mmarsegliaalmost 8 years ago
looks like my terraform state wasn't saved. resources were created but they don't exist in the state file. how do folks deal with a situation like this?
mmarseglia
mmarsegliaalmost 8 years ago
should i try an import?
mrwacky
mrwackyalmost 8 years ago
sure
joshmyers
joshmyersalmost 8 years ago
@mmarseglia There is also https://github.com/dtan4/terraforming for generating the state of those resources, which you can hack back into Terraform if you now have split state. Although this can mean hand mangling JSON, which you would rather avoid.
mmarseglia
mmarsegliaalmost 8 years ago
thanks.. i'll check that out and maybe try a terraform import
mmarseglia
mmarsegliaalmost 8 years ago
i've got the code written..it's just a matter of getting terraform to recognize what it already did
mmarseglia
mmarsegliaalmost 8 years ago
i'm still wondering why it didn't save.. if maybe i did an apply and then shut my system down before it could save
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
@mmarseglia Are you using remote state ?
mmarseglia
mmarsegliaalmost 8 years ago
i am
mmarseglia
mmarsegliaalmost 8 years ago
saving to an s3 bucket
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
Have you checked for a file "errored.tfstate"
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
in your root module folder
joshmyers
joshmyersalmost 8 years ago
It should obey SIGTERM/SIGINT in a sane way and try and get the state stored. Did you end up with half a state, with some resources?
mmarseglia
mmarsegliaalmost 8 years ago
unfortunately i didn't notice what was wrong, cleaned up by removing the .terraform directory, and tried applying again
mmarseglia
mmarsegliaalmost 8 years ago
so any errored.tfstate file is gone
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
errored.tfstate isn't stored in .terraform afaik
mmarseglia
mmarsegliaalmost 8 years ago
oh.. maybe it's still around then
mmarseglia
mmarsegliaalmost 8 years ago
i'll look
mmarseglia
mmarsegliaalmost 8 years ago
my clean step is a rm -rf terraform/.terraform terraform/*.tfstate output.json inspec.json bucket
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
ah yeah, that's quite rigorous 🙂
mmarseglia
mmarsegliaalmost 8 years ago
@joshmyers some resources are in the state file, there's an IAM policy in there that was created
mmarseglia
mmarsegliaalmost 8 years ago
but others are not
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
Sometimes simpler than importing is to just delete everything which isn't in the state and apply again.
mmarseglia
mmarsegliaalmost 8 years ago
i'll have to reconsider that clean step
mmarseglia
mmarsegliaalmost 8 years ago
sounds like it's too drastic
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
yep
joshmyers
joshmyersalmost 8 years ago
do you need this thing that failed to build?
joshmyers
joshmyersalmost 8 years ago
or can you destroy and re provision?
mmarseglia
mmarsegliaalmost 8 years ago
i think i can go through and delete the resources it created and didn't save state for
mmarseglia
mmarsegliaalmost 8 years ago
that might be easier than an import
joshmyers
joshmyersalmost 8 years ago
Have worked on codebases where a single TF run contains 500+ resources, not so easy to delete, hah. Filter by Terraform tag on resources and delete
mmarseglia
mmarsegliaalmost 8 years ago
this is why i need to tag things
joshmyers
joshmyersalmost 8 years ago
Also there is a tradeoff between large state files (the collateral damage that a bad TF run could do) and splitting into tiny but related state, where you find yourself then having to run multiple times to pick up new changes etc. How to draw those dependencies?
mmarseglia
mmarsegliaalmost 8 years ago
that's a tough problem.
joshmyers
joshmyersalmost 8 years ago
Terragrunt has a kinda neat way of dealing with it
mmarseglia
mmarsegliaalmost 8 years ago
i like it
mmarseglia
mmarsegliaalmost 8 years ago
didn't think about separate .tfvars for different environments. but that's a good idea
mmarseglia
mmarsegliaalmost 8 years ago
right now i just run two environments, dev and prod, in separate AWS accounts
joshmyers
joshmyersalmost 8 years ago
Isn’t ideal though as planning B doesn’t pick up changes that planning changes in A has done, because A’s haven’t actually been apply’d yet. AFAICR
johncblandii
johncblandiialmost 8 years ago
that’ll be resolved once .12 comes out; modules get depends_on
joshmyers
joshmyersalmost 8 years ago
Not things in different state though, no? Hence why it is important to try and split state on logical boundaries
loren
lorenalmost 8 years ago
maybe use something simple like Makefile to execute terragrunt/terraform in a required order?
joshmyers
joshmyersalmost 8 years ago
Aye, that fixes half of that problem. Other half is harder because of state.
joshmyers
joshmyersalmost 8 years ago
i5okie
i5okiealmost 8 years ago
hi all
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
@i5okie the first one can generate a key and import it into AWS
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
the second one just generates a key-pair (does not touch AWS at all)
i5okie
i5okiealmost 8 years ago
how does it import it into AWS?
i5okie
i5okiealmost 8 years ago
nvm
Gonzalo Peci
Gonzalo Pecialmost 8 years ago
Trying to trial ecs-codepipeline, but as far as I see there is no example of what generates this https://github.com/cloudposse/terraform-aws-ecs-codepipeline/blob/master/main.tf#L234
Gonzalo Peci
Gonzalo Pecialmost 8 years ago
any pointers?
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
Gonzalo Peci
Gonzalo Pecialmost 8 years ago
Yeah, i know that part
Gonzalo Peci
Gonzalo Pecialmost 8 years ago
I was checking aws-ecs-web-app
Gonzalo Peci
Gonzalo Pecialmost 8 years ago
but the pipeline says codebuild will output a task
Gonzalo Peci
Gonzalo Pecialmost 8 years ago
but I dont find any examples of in which format, if it expects the generated JSON or just a taskarn:version
Gonzalo Peci
Gonzalo Pecialmost 8 years ago
etc
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
@Gonzalo Peci I can provide examples in about 2 hours
Gonzalo Peci
Gonzalo Pecialmost 8 years ago
Yeah no rush! I was just curious/testing/playing around
antonbabenko
antonbabenkoalmost 8 years ago
I’ve just released it to the public! Now nice AWS diagrams created using cloudcraft.co can be exported to Terraform infrastructure code. Please try it and share with your circles. More details in my blog post - https://medium.com/@anton.babenko/modules-tf-convert-visual-aws-diagram-into-terraform-configurations-e61fb0574b10
loren
lorenalmost 8 years ago
anyone using terraform to apply tags on vpcs/subnets? it looks like terraform does this destructively, so if any tag is created outside terraform, the tag is removed on next apply... any workaround for that behavior?
nutellinoit
nutellinoitalmost 8 years ago
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
quick q. How can I count the size of multiple splitted values of a string , inside a map, inside a list.

[ { 
 env: "key1:val,key2:val"
 somethingelse: "hello"
},
{
 env: "key1:val,key2:val,key3:val"
 somethingelse: "hi there"
}]

count =5
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
dammit 🙂
joshmyers
joshmyersalmost 8 years ago
Terraform is not a proper language 😞
joshmyers
joshmyersalmost 8 years ago
There is probably a hack for it…dare I ask if 0.12 fixes this?
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
didn't deep dive yet
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
doesn't look like that nested loops work
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago(edited)
from: https://www.hashicorp.com/blog/hashicorp-terraform-0-12-preview-for-and-for-each

      for num in var.subnet_numbers:
      cidrsubnet(data.aws_vpc.example.cidr_block, 8, num)
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
so maybe it does
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
Looks interesting
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
State file for terraform project state
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
Wow! This is like Helmfile for Terraform: https://github.com/uber/astro/blob/master/README.md
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
Dig it
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
Has anyone attempted vendoring terraform modules?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
Templating terraform has come up a few times. Was going through my stats and found this https://github.com/cbroglie/terrastache/blob/master/README.md
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
This might be the tool we need to achieve some kind of vendoring with terraform
Duong
Duongalmost 8 years ago
hello, does anyone know how to edit an existing resource that belongs to another module (was created by another module) by terraform 🤔

I'm using github.com/cloudposse/terraform-aws-ec2-bastion-server and I want to edit the aws_security_group, I want to add the egress part to that.



# Main bastion
module "bastion" {
  source              = "<http://github.com/cloudposse/terraform-aws-ec2-bastion-server|github.com/cloudposse/terraform-aws-ec2-bastion-server>"

  name                = "${var.name}"
  ami                 = "${var.ami}"
  instance_type       = "${var.instance_type}"
  # This key needs to be created beforehand
  key_name            = "${var.deploy_key_name}-${terraform.workspace}"
  vpc_id              = "${var.vpc_id}"
  subnets             = "${var.subnets}"
  ssh_user            = "${var.ssh_user}"
  security_groups     = []
  namespace           = "${var.namespace}"
  stage               = "${terraform.workspace == "staging" ? "staging" : "production"}"
}

resource "aws_security_group" "default" {
  name        = "${var.name}"
  vpc_id      = "${var.vpc_id}"

  ingress {
    protocol  = "tcp"
    from_port = 22
    to_port   = 22

    cidr_blocks = ["xxx/32", "yyy/32"]
  }

  egress {
    protocol  = "tcp"
    from_port = 22
    to_port   = 22

    cidr_blocks = ["xxx/32", "yyy/32"]
  }

  depends_on = ["module.overblock_public_bastion"]
}


thanks.
nutellinoit
nutellinoitalmost 8 years ago
nutellinoit
nutellinoitalmost 8 years ago
i think you need only an additional egress rule on the bastion security group
Duong
Duongalmost 8 years ago
thanks @nutellinoit 🙏
antonbabenko
antonbabenkoalmost 8 years ago
@loren and other #terragrunt users, I’d love to hear your feedback on terragrunt structure I generate when generating code from cloudcraft.co diagrams.
loren
lorenalmost 8 years ago
got an example repo?
antonbabenko
antonbabenkoalmost 8 years ago
@loren This is an example which was generated from the complete diagram mentioned in my blogpost (includes VPC, SG, and real dependencies)
Jan
Janalmost 8 years ago
could you link to the post?
antonbabenko
antonbabenkoalmost 8 years ago
Good idea, maybe a bit later, I really need people to try it and I can see errors in logs (much fewer than I expected really) 🙂
Jan
Janalmost 8 years ago
ID be happy to try it
antonbabenko
antonbabenkoalmost 8 years ago
It is free for all types of cloudcraft users
Jan
Janalmost 8 years ago
train + ipv6 = fail
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
or just BAHN = fehl
Jan
Janalmost 8 years ago
hahaha
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
It's funny how it's possible to have more or less working WIFI on airplanes, but crappy signal the moment a train goes faster than 100mph.
Jan
Janalmost 8 years ago
its great in the tunnels
Jan
Janalmost 8 years ago
im gonna be doing this trip regularly
Jan
Janalmost 8 years ago
might start building packetloss ands signal noise heat map
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
which trip is it ?
Jan
Janalmost 8 years ago
Berlin to Freiberg and back
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
long one 🙂 cute city btw
Jan
Janalmost 8 years ago
Yea its a good 5-6 hour trip
Jan
Janalmost 8 years ago
beats the crap out of flying to Zurich and driving back in
Jan
Janalmost 8 years ago
at least on the train I can work
Jan
Janalmost 8 years ago
provided I dont need to download every thing
Jan
Janalmost 8 years ago
😛
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
running tf locally ?
Jan
Janalmost 8 years ago
I hadnt planned it well
Jan
Janalmost 8 years ago(edited)
😛
antonbabenko
antonbabenkoalmost 8 years ago
You can use terraform aws provider for plans locally, yes
Jan
Janalmost 8 years ago
cached to ~/.terraform?
Jan
Janalmost 8 years ago(edited)
Just hit Berlin, internet should get better
Jan
Janalmost 8 years ago
oh
Jan
Janalmost 8 years ago
in /localhost/
Jan
Janalmost 8 years ago
nice
antonbabenko
antonbabenkoalmost 8 years ago
Check my blogpost for arguments in provider - https://link.medium.com/zfM60Pfp5R
Maarten van der Hoef
Maarten van der Hoefalmost 8 years ago
so with skip_requesting_account_id to true, how does that affect say, aws_vpc ?
antonbabenko
antonbabenkoalmost 8 years ago
you can run “terraform plan -refresh=false” and terraform won’t perform API calls externally
Nikola Velkovski
Nikola Velkovskialmost 8 years ago
@antonbabenko I'll check them out with our state and see the improvements because we are sending the apply times to CW for measurement.
Nikola Velkovski
Nikola Velkovskialmost 8 years ago
Thanks!
antonbabenko
antonbabenkoalmost 8 years ago
Last time I checked it was reduced number of calls, but not speed improvement in first place. Let us know here, or leave a comment to the post and I will update it.
Nikola Velkovski
Nikola Velkovskialmost 8 years ago
will do
Andrew Jeffree
Andrew Jeffreealmost 8 years ago
never thought I'd be doing this. But I'm using terraform to create an obstacle course of a broken AWS environment for an interview process. But it's kind of fun.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
that’s cool
Andrew Jeffree
Andrew Jeffreealmost 8 years ago
Yeah. We're finally expanding the team and I wanted a way to be able to reproduce a broken environment.
Andrew Jeffree
Andrew Jeffreealmost 8 years ago
for testing purposes
Andrew Jeffree
Andrew Jeffreealmost 8 years ago
having to reference my modules quite a bit without actually using them, as they're all built with as much validation as I can put in them to prevent exactly what I'm building from happening
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
i’ve tried to write similar things for other languages. it can be tricky to come up with fair challenges.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
one thing i’ve noticed is that technical domain expertise related to triaging “bugs” can be highly tied to an organization’s style of development
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
we had a developer on our team who was a very senior terraform guy
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
yet when it came to developing on our modules, the kinds of errors he ran into we’re new to him
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago(edited)
not sure how to be express this. what i mean is that we see often the same kinds of errors for our stuff, which might not be the same kind of errors another company often sees. the “count of cannot be computed” is common for us. yet it seems (from my experience interviewing a lot of candidates) that there are sr terraform developers who are not familiar with this error, despite having written a lot of modules.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago(edited)
so… i am curious, what kinds of things do you want to test for?
Andrew Jeffree
Andrew Jeffreealmost 8 years ago
so we're not actually testing terraform
Andrew Jeffree
Andrew Jeffreealmost 8 years ago
we're testing AWS knowledge/troubleshooting
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
aha! i see what you mean
Andrew Jeffree
Andrew Jeffreealmost 8 years ago
I considered testing terraform, but I ran into the same thoughts you've mentioned above
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
gotcha, so using terraform to “stage” an environment in a “broken” state
Andrew Jeffree
Andrew Jeffreealmost 8 years ago
yep
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
are you creating a disposable org at the same time?
Andrew Jeffree
Andrew Jeffreealmost 8 years ago
not at this point in time. It's just a single account thing.
Andrew Jeffree
Andrew Jeffreealmost 8 years ago
The role is a juniorish one
Andrew Jeffree
Andrew Jeffreealmost 8 years ago
It's our first time hiring for this role externally so we'll probably adjust it as we go.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
yea, plant a stake and start somewhere
Andrew Jeffree
Andrew Jeffreealmost 8 years ago
But for now I've been told to avoid doing anything particularly nasty compared to some of the other obstacle courses I've made.
Andrew Jeffree
Andrew Jeffreealmost 8 years ago
I made a linux based one where it kicks off a shutdown timer for 60 seconds when you login the first time. We don't use it often, as we have tons of them, it's mostly used for the people who are smug and try and throw their weight around in the interview process.
joshmyers
joshmyersalmost 8 years ago
Doh, just about to open a PR for the Terraform org CloudTrail stuff and appear to have been pipped by https://github.com/terraform-providers/terraform-provider-aws/pull/6580
joshmyers
joshmyersalmost 8 years ago
Pretty much same implementation, ah well
joshmyers
joshmyersalmost 8 years ago
Pipped by 5 hours 🤣
joshmyers
joshmyersalmost 8 years ago
Half a day I’m never getting back
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
Argh! That’s annoying....
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
It’s pretty amazing (generally) how fast this stuff gets contributed by the community.
Ben Hecht
Ben Hechtalmost 8 years ago
Hi, is this the proper support channel for terraform-aws-kops-vpc-peering?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
hey Ben, yes it is
Ben Hecht
Ben Hechtalmost 8 years ago
Hi - We are looking to peer our Kops VPC to a VPC in another AWS account so pods can access services in the outside account. In looking for a way to codify the "routes" we had planned on adding manually, I came across your project. Few questions - why does namespace need to be specified. Where do we define the network ranges for which traffic to send over the peer? Why does a namespace need to be declared? Mulitple pods in multiple namespaces might be sending traffic to the peer. And what is stage?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
the namespace in the module is not a k8s namespace. namespace+stage+name is our naming convention (how we uniquely name all AWS resources
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
for example, if you company is CloudPosse, then you can select the namespace as cp (or any other name you want)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
stage could be prod, staging, dev, testing
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago(edited)
name is your app/solution name
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago(edited)
so, for production, you would select cp-prod-myapp to name all the AWS resources created by the modules
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
we use the label modules for that
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
Ben Hecht
Ben Hechtalmost 8 years ago
Got it, what is the purpose of bastion_name masters_name nodes_name is it for identifying existing SGs/Roles?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
here is an example on how we use it to peer the kops VPC with backing services VPC https://github.com/cloudposse/terraform-root-modules/tree/master/aws/kops-aws-platform
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
we did not test it cross-account
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
(default to how kops names the masters and the nodes)
Ben Hecht
Ben Hechtalmost 8 years ago(edited)
Does the module support peering with VPCs in different accounts? InvalidVpcID.NotFound
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
we did not test it, not sure what needs to be done for that (will have to take a look)
Ben Hecht
Ben Hechtalmost 8 years ago
Thanks for the help!
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
np
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
the module does not support cross-account b/c it was created before it was added by AWS
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
you need to use new resources for cross-account https://www.terraform.io/docs/providers/aws/r/vpc_peering.html
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
i recall with one of our clients we setup cross account VPCs. the downside was there was no easy way to "Autoapprove" (if i recall correctly)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
so there was a manual step necessary to establish the tunnels
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
i'm going off vague memory here
loren
lorenalmost 8 years ago
you can (now) autoapprove a vpc peering link across accounts
loren
lorenalmost 8 years ago
you need to define two providers, one for each end of the link
loren
lorenalmost 8 years ago

provider "aws" {}

provider "aws" {
  alias = "peer"
}

data "aws_vpc" "peer" {
  count = "${var.create_peering_connection ? 1 : 0}"

  provider = "aws.peer"

  id = "${var.peer_vpc_id}"
}

resource "aws_vpc_peering_connection" "this" {
  count = "${var.create_peering_connection ? 1 : 0}"

  peer_owner_id = "${var.peer_owner_id}"
  peer_vpc_id   = "${var.peer_vpc_id}"
  vpc_id        = "${var.vpc_id}"
  tags          = "${merge(var.tags, map("Name", "${var.name}-${var.vpc_cidr}<->${var.peer_alias}-${data.aws_vpc.peer.cidr_block}"))}"
}

resource "aws_vpc_peering_connection_accepter" "this" {
  count = "${var.create_peering_connection ? 1 : 0}"

  provider = "aws.peer"

  vpc_peering_connection_id = "${aws_vpc_peering_connection.this.id}"
  auto_accept               = true

  tags {
    Name = "${var.peer_alias}-${data.aws_vpc.peer.cidr_block}<->${var.name}-${var.vpc_cidr}"
  }
}
loren
lorenalmost 8 years ago
you can also create routes on both ends, using the multiple providers... (though now i'm seeing i biffed the route table association and just grabbed one instead of iterating over the list)

resource "aws_route" "public" {
  count = "${var.create_peering_connection ? 1 : 0}"

  route_table_id            = "${var.public_route_tables[0]}"
  destination_cidr_block    = "${data.aws_vpc.peer.cidr_block}"
  vpc_peering_connection_id = "${aws_vpc_peering_connection.this.id}"
}

resource "aws_route" "private" {
  count = "${var.create_peering_connection ? 1 : 0}"

  route_table_id            = "${var.private_route_tables[0]}"
  destination_cidr_block    = "${data.aws_vpc.peer.cidr_block}"
  vpc_peering_connection_id = "${aws_vpc_peering_connection.this.id}"
}

resource "aws_route" "peer" {
  count = "${var.create_peering_connection ? length(var.peer_route_tables) : 0}"

  provider = "aws.peer"

  route_table_id            = "${var.peer_route_tables[count.index]}"
  destination_cidr_block    = "${var.vpc_cidr}"
  vpc_peering_connection_id = "${aws_vpc_peering_connection_accepter.this.id}"

  depends_on = ["aws_vpc_peering_connection_accepter.this"]
}
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
thanks @loren
loren
lorenalmost 8 years ago
you pass providers in when instantiating the module, like so:

module "pcx" {
  source = "../pcx"

  providers = {
    aws      = "aws"
    aws.peer = "aws.peer"
  }

  // vars
}
loren
lorenalmost 8 years ago
oh, and now we have a transit gateway option, looks to make vpc peering mostly obsolete? https://aws.amazon.com/blogs/aws/new-use-an-aws-transit-gateway-to-simplify-your-network-architecture/
joshmyers
joshmyersalmost 8 years ago
Nice!
Jan
Janalmost 8 years ago
Im curious to see a cost comparison
i5okie
i5okiealmost 8 years ago
Hi
i5okie
i5okiealmost 8 years ago
so, the terraform-aws-elastic-beanstalk-environment module isn’t working for me right now. with load-balancer type set to “application” i get CREATE_FAILED
i5okie
i5okiealmost 8 years ago
Creating load balancer failed Reason: Property HealthyThreshold cannot be empty
i5okie
i5okiealmost 8 years ago
im testing it with adding the setting
joshmyers
joshmyersalmost 8 years ago
@i5okie Do you have a gist of output or anymore info?
i5okie
i5okiealmost 8 years ago
lost terraform’s ouput.
i5okie
i5okiealmost 8 years ago
gah now its stuck in “invalid state” lol grrreat
joshmyers
joshmyersalmost 8 years ago
Generally stayed away from EBS as it is a pain to debug. Black box.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
for some of the beanstalk @johncblandii is probably the most senior!
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
😃
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
he's taken it above and beyond
johncblandii
johncblandiialmost 8 years ago
hehe. More is coming too. 🙂
johncblandii
johncblandiialmost 8 years ago
Ping me directly, @i5okie. I’ll see if I can help.
i5okie
i5okiealmost 8 years ago
yeah i can’t figure it out. I thought adding that one setting would help, but it didnt. ended up switching back to classic elb
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
Andy
Andyalmost 8 years ago
Also, checkout https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/command-options-general.html specifically the elbv2 ones might help. I remember running into issues as well and stuck with Classic.
i5okie
i5okiealmost 8 years ago
i think i just figured out why my user data file isn’t getting pushed to bastion, with the aws_ec2_bastion_server module..
i5okie
i5okiealmost 8 years ago
there is a variable “user_data_file”. but it is not actually used anywhere else in the module lol.
rohit
rohitalmost 8 years ago(edited)
you know how we can add propagate_at_launch = "true" to some of the resources but not all of them
. i see that we can use cloudposse/terraform-null-label
additional_tag_map = {
    propagate_at_launch = "true"
  }

but wouldn't it add this to all the resources ?
rohit
rohitalmost 8 years ago
and i am not sure about the side effects of adding this to all the resources
rohit
rohitalmost 8 years ago
so i am trying to figure out how to add this tag only to the resources that need it
rohit
rohitalmost 8 years ago
any help ?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago(edited)
hrm.... "wouldn't it add this to all labels"
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago(edited)
can you clarify/elaborate
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
oh, i think i gotcha
rohit
rohitalmost 8 years ago
typo
rohit
rohitalmost 8 years ago
updated it
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago(edited)
so there's a tags_as_list_of_maps output which uses those additional tags
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago(edited)
this was something @Jamie Nelson (BitFlight) added to make it easier to add tag that only apply to certain resources (E.g. autoscale groups)
rohit
rohitalmost 8 years ago(edited)
so the tags defined in additional_tag_map will only be applied when i call "${module.my_app.tags_as_list_of_maps}" and will not be applied when i call "${module.my_app.tags}" , correct ?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
almost, but i think you got it reversed
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
so the tags defined in additional_tag_map will only be applied when i call "${module.my_app.tags_as_list_of_maps}" and will not be applied when i call "${module.my_app.tags}"
rohit
rohitalmost 8 years ago
lol yeah
rohit
rohitalmost 8 years ago
makes sense
rohit
rohitalmost 8 years ago
this is great
rohit
rohitalmost 8 years ago
i also see that there is another module called terraform-terraform-label
rohit
rohitalmost 8 years ago
what's the difference between these two modules ?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
hi @rohit
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
terraform-terraform-label is a simpler version of terraform-null-label (before a lot of features were added to it)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago(edited)
for simple use-cases you can use terraform-terraform-label
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago(edited)
@Andriy Knysh (Cloud Posse) i don't think it has the feature he wants: tags_as_list_of_maps
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
those tags as list of maps are only for some AWS resources
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
most of AWS does not require it
rohit
rohitalmost 8 years ago
@Andriy Knysh (Cloud Posse) makes sense. Thanks
rohit
rohitalmost 8 years ago
is it possible to use string interpolation in both these modules ?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
also, terraform-null-label supports a few additional features: 1) context; 2) var.environment
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
@rohit what do you mean by string interpolation? string interpolation can be used in any TF resource
rohit
rohitalmost 8 years ago
i mean when using terraform-null-label, can i use something like stage = "${terraform.workspace}" ?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
yes
rohit
rohitalmost 8 years ago
nice
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
(it’s not related to terraform-null-label, you can do it with any module or resource)
rohit
rohitalmost 8 years ago
i am fairly new to terraform and i noticed that there are some restrictions on where you could use string interpolation, count variable etc. so just wanted to confirm
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
aha
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
ah ok, yes you can use it as inputs to any module
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago(edited)
but be aware of "count cannot be computed"!
rohit
rohitalmost 8 years ago
good to know
rohit
rohitalmost 8 years ago
so if i do something like this
tag_specifications {
    resource_type = "instance"
    tags          = "${module.app.tags}"
  }

in aws_launch_template resource and tags = ["${module.app.tags_as_list_of_maps}"] in aws_autoscaling_group resource, will it apply the tags twice ?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
because of the “count cannot be computed” issues with terraform-null-label when it’s used in some complex module hierarchies (not all)
rohit
rohitalmost 8 years ago
it makes sense how you did it using terraform-terraform-label but if i use terraform-null-label, does my question above makes sense ?
rohit
rohitalmost 8 years ago
well, i don't think my question is related to using one vs other
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago(edited)
yea, your question is related to how the tags will be applied to the EC2 instances if they are specified in both launch template and auto-scaling group
rohit
rohitalmost 8 years ago
yup
rohit
rohitalmost 8 years ago
so in your case what happens ?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
we did not see duplicate tags
rohit
rohitalmost 8 years ago
ok
rohit
rohitalmost 8 years ago
how did you organize your modules ? was it based on apps or category ?
rohit
rohitalmost 8 years ago
i am planning to do it based on category
rohit
rohitalmost 8 years ago(edited)
modules/compute/app1,modules/storage,module/networking
rohit
rohitalmost 8 years ago
and i noticed that some people do it by apps
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
yea, we have a pretty novel way of doing things.
rohit
rohitalmost 8 years ago
just want to know how you did it
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
first as you see, we have a lot of modules. these follow the naming convention set forth by the terraform registry
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
then there are the invocations of these modules; basically, how we use these modules in concert with eachother
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
we call this our terraform-root-modules because they are the "root" module
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
then we pull in the root modules where we want to use them.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
notice we use docker for everything 😃
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
this eliminates almost all local (or native) dependencies since we can bundle it in the docker image
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
e.g. terraform, chamber, aws-vault, etc
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
there are certainly a lot of ways to do things
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago(edited)
here are our refrence architectures: http://cpco.io/reference-architectures
rohit
rohitalmost 8 years ago
we are not using docker and i am not sure which one to pick from two approaches i mentioned
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
Also google #terragrunt and check out their reference architectures
rohit
rohitalmost 8 years ago
heard a lot about terragrunt
rohit
rohitalmost 8 years ago
but terraform is new to me and i didn't want to learn a new tool on top of terraform
rohit
rohitalmost 8 years ago
is there a reason why you didn't use terragrunt ?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
there are two main patterns to assemble everything together and provide variables:
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
1. Template rendering (as done by terragrunt)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago(edited)
2. Using Docker containers + multi-stage images + ENV vars specified in Dockerfile (and in some secret storage as AWS SSM for secrets) - we do it with geodesic (https://github.com/cloudposse/geodesic)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
so we mostly use #2 as you can see in http://cpco.io/reference-architectures
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
@rohit to answer you question how did you organize your modules ? was it based on apps or category ?, it’s never that straightforward, we probably organize the modules by categories first https://github.com/cloudposse/terraform-root-modules/tree/master/aws
rohit
rohitalmost 8 years ago
@Andriy Knysh (Cloud Posse) i am planning to organize by category too
rohit
rohitalmost 8 years ago
Is it possible to download userdata template from S3 ?
rohit
rohitalmost 8 years ago(edited)
How do you overcome the issue that i mentioned here https://github.com/terraform-providers/terraform-provider-aws/issues/6019 ?
Jamie Nelson (BitFlight)
Jamie Nelson (BitFlight)almost 8 years ago
If anyone is getting ‘the value of count cannot be computed’ errors when using the null label module can you show me an example of where it’s used?
I have recently been troubleshooting this and I have found some new cases where that error shows up when it normally wouldn’t.
Jamie Nelson (BitFlight)
Jamie Nelson (BitFlight)almost 8 years ago(edited)
Short answer being: if upstream dependant resources have an error, that error isn’t displayed and instead downstream computed values like count are broken.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
@Jamie Nelson (BitFlight) here is an example of that
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
Stephen
Stephenalmost 8 years ago
Question: For terraform-aws-codebuild what is the buildspec variable? i have a buildspec.yml file in my repo. How do i configure my codebuild to use that?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
1 sec @Stephen
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
and it was throwing a lot of count can't be computed errors
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
that’s why we just added this https://github.com/cloudposse/terraform-aws-ec2-autoscale-group/blob/master/main.tf#L61 to calculate tags_as_list_of_maps separately
Stephen
Stephenalmost 8 years ago
@Andriy Knysh (Cloud Posse) i think i got the answer from an issue re: that variable. It picks it up in the root dir during build if not set.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
Stephen
Stephenalmost 8 years ago
@Andriy Knysh (Cloud Posse) It worked 🙂 It just pushed my git repo project up to dockerhub?
Stephen
Stephenalmost 8 years ago
Now to hook up the pipeline, and then webhooks i think. Then it will be EtoE
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
nice @Stephen
Stephen
Stephenalmost 8 years ago
Your cicd example is GitHub -> ECR (Docker image) -> Elastic Beanstalk (running Docker stack) but i will need to replace ECR w/ Dockerhub.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
let us know if you need any help
Stephen
Stephenalmost 8 years ago
It looks like maybe i should have just used terraform-aws-cic instead. It looks like it has the Build and Pipeline together?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
you can
Stephen
Stephenalmost 8 years ago
i am trying to get the aws-cicd example to match with my existing pipeline configuration. What are the available options for Stage.actions.output.Artifacts?
Stephen
Stephenalmost 8 years ago
as well as inputArtifacts
Stephen
Stephenalmost 8 years ago
my live pipeline:
 "outputArtifacts": [
                            {
                                "name": "BuildArtifact"
                            }
                        ],
                        "inputArtifacts": [
                            {
                                "name": "SourceArtifact"
                            }
                        ]
Stephen
Stephenalmost 8 years ago
my new build stage in aws-cicd:
input_artifacts  = ["code"]
      output_artifacts = ["package"]
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
@Stephen it’s just names you select for the stages’ input and output artifacts, you can assign any names as long as you use the same name from the stage output to the next stage input
Stephen
Stephenalmost 8 years ago
Ok, another question. i am pulling from a public GH repo but it seems like i am required to use a GITHUB_TOKEN for pipeline, even though the codebuild module worked fine without it.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
yes, currently the module requires GitHub token (we used it for private registries) https://github.com/cloudposse/terraform-aws-cicd/blob/master/variables.tf#L33
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
you can add a token (even if you use only public repos)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
Can we make the token optional?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
i don’t remember if making the token optional will work here https://github.com/cloudposse/terraform-aws-cicd/blob/master/main.tf#L197
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
needs to be tested
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
@Stephen if you test it and it works, can open a PR, we’ll review promptly, thanks
Stephen
Stephenalmost 8 years ago
Leaving it blank fails
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
can you test with any random string? 🙂
Stephen
Stephenalmost 8 years ago
tried that, it then tried to auth
Stephen
Stephenalmost 8 years ago
w/o string it complained about the config not correct.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
yea, that’s why it’s there, for private repos
Stephen
Stephenalmost 8 years ago
ya, it would be better if it were optional. Guess thats upstream issue?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
for just public, the module needs to be changed to support that
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
it’s AWS API ‘issue’: you don’t provide a tokem, can access only public repos. You provide a token, it will try to auth even if you don’t want to use it
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
you can create a token with limited permissions and use it for now
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
also can you open an issue in the repo and explain that? thanks
Stephen
Stephenalmost 8 years ago
How would i go about adding ENV vars to the pipeline->Build stage?
Stephen
Stephenalmost 8 years ago
In my case, in my buildspec.yml I do a docker login with my dockerhub u/p
Stephen
Stephenalmost 8 years ago
so i can push the docker image up
Stephen
Stephenalmost 8 years ago
pre_build:
    commands:
      - echo Logging in to Docker Hub...
      # Type the command to log in to your Docker Hub account here.
      - docker login -u $DOCKER_USERNAME -p $DOCKER_PASSWORD
Stephen
Stephenalmost 8 years ago
@Andriy Knysh (Cloud Posse) re: the token issue: https://github.com/terraform-providers/terraform-provider-aws/issues/6646
i5okie
i5okiealmost 8 years ago
you guys ever seen this before?
Stephen
Stephenalmost 8 years ago
@i5okie Ya, i think you added the wrong stack name
i5okie
i5okiealmost 8 years ago
I was originally using this solution_stack_name = "64bit Amazon Linux 2018.03 v2.8.6 running Ruby 2.3 (Passenger Standalone)"
Stephen
Stephenalmost 8 years ago
64bit Amazon Linux 2018.03 v2.8.6 running Ruby 2.3 (Passenger Standalone) is valid.
i5okie
i5okiealmost 8 years ago
and copied the config over for another account. and now its complaining wrong stack. so i changed the name to what is in the screenshot
i5okie
i5okiealmost 8 years ago
ooh wait.. i think i know whats wrong
Stephen
Stephenalmost 8 years ago
2.8.4 not found
i5okie
i5okiealmost 8 years ago
i’ve changefd to 2.8.4 but didn’t change the date
i5okie
i5okiealmost 8 years ago
hehe
Stephen
Stephenalmost 8 years ago
😉
i5okie
i5okiealmost 8 years ago
four eyes is better than two
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
@Stephen re: env vars in codebuild
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
unfortunately, not propagated to https://github.com/cloudposse/terraform-aws-cicd 😞
Stephen
Stephenalmost 8 years ago
ya, i just added it to my local version
Stephen
Stephenalmost 8 years ago
@Andriy Knysh (Cloud Posse) Getting this now during my EB deploy stage:
Deployment failed. The provided role does not have sufficient permissions: Failed to deploy application. Service:AWSLogs, Message:User: arn:aws:sts::xxxxx:assumed-role/xxxx-development-app/xxxxx is not authorized to perform: logs:PutRetentionPolicy on resource: arn:aws:logs:us-east-1:xxxx:log-group:/aws/elasticbeanstalk/xxxxx-dev-node-demo/var/log/nginx/error.log:log-stream:


Any ideas?
joshmyers
joshmyersalmost 8 years ago
It needs access to cloudwatchlogs in order to create log streams and put retention policies on them etc
joshmyers
joshmyersalmost 8 years ago
Did that resolve the issue for you?
joshmyers
joshmyersalmost 8 years ago
I think it would be nicer to split out those perms into a more scoped form, rather than applying on every resource.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
so yea, we did not use logs for our EB apps
Stephen
Stephenalmost 8 years ago
@joshmyers Yes, i was able to Source, Build, and Deploy once i added that new action
Stephen
Stephenalmost 8 years ago
Also had to add this as well:
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
thanks @Stephen will review and merge
Stephen
Stephenalmost 8 years ago
starting on your webhook module now…
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
@Stephen reviewed https://github.com/cloudposse/terraform-aws-cicd/pull/36, thanks, just a few comments
Stephen
Stephenalmost 8 years ago(edited)
@Andriy Knysh (Cloud Posse) all set
Stephen
Stephenalmost 8 years ago
How do i replicate the CodePipline UI where it connects to github and i tell it to use "webhooks" for the detection option?
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago
@Stephen terraform-aws-cicd does not use webhooks, it uses this https://github.com/cloudposse/terraform-aws-cicd/blob/master/main.tf#L203
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 8 years ago(edited)
if you know how to implement it, we’ll accept one more PR 🙂 (thanks again for the two PRs)
i5okie
i5okiealmost 8 years ago
nope still
i5okie
i5okiealmost 8 years ago
straight from > eb platform list -v
i5okie
i5okiealmost 8 years ago
oh crap. its not letting me create a new environment with older stack. argh well this screws me over
i5okie
i5okiealmost 8 years ago
lol. again.. stupid elastic beanstalk haha
joshmyers
joshmyersalmost 8 years ago
Step. Away. From. ElasticBeanstalk :D
rohit
rohitalmost 8 years ago
Is it possible to create an rds postgres instance with multiple databases ?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
Yes/no
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
You would use the Postgres terraform provider to provision the additional databases inside of the RDS instance
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
The AWS API for RDS does not enable additional databases (as in schemas) to be provisioned
rohit
rohitalmost 8 years ago
is there any example that you use ?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
No we don’t have an example
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 8 years ago
rohit
rohitalmost 8 years ago
thanks Erik
Max Moon
Max Moonalmost 8 years ago(edited)
I have used the postgres TF provider, it works okay
Max Moon
Max Moonalmost 8 years ago
I would up scrapping it, because I still needed to do enough work after applying changes with it that it wasn't worth using it in the first place