#airship - March 2019 | Slack Archive

90 messages

Home of Airship ECS Modules ( https://github.com/blinkist/terraform-aws-airship-ecs-service / https://github.com/blinkist/terraform-aws-airship-ecs-cluster )

Archive: https://archive.sweetops.com/airship/

Feb 2019

March 2019

Apr 2019

Maarten van der Hoefover 7 years ago

hey guys, I'll try to spend some more time again, started working somewhere new so last weeks were just flying and I needed to be away from my laptop in the evenings.

jaustinpageover 7 years ago

Hi Guys!

I was running through the airship demo, but, got stuck on some errors on the last step, creation of the fargate service itself.
I checked github issues, didnt see anything similar, checked github pr's, didnt see anything that looked the same.

in trying to debug, i bumped to the 0.8.8 tag, instead of 0.8.6 as listed in the docs

The errors are:

* module.fargate_service.module.iam.aws_iam_role_policy_attachment.ecs_tasks_execution_role: 1 error(s) occurred:
* module.fargate_service.module.iam.aws_iam_role_policy_attachment.ecs_tasks_execution_role: Resource 'aws_iam_role.ecs_task_execution_role' not found for variable 'aws_iam_role.ecs_task_execution_role.id'
* module.fargate_service.module.iam.output.ecs_task_execution_role_arn: Resource 'aws_iam_role.ecs_task_execution_role' does not have attribute 'arn' for variable 'aws_iam_role.ecs_task_execution_role.*.arn'
* module.fargate_service.module.iam.output.lambda_ecs_task_scheduler_role_arn: Resource 'aws_iam_role.lambda_ecs_task_scheduler' does not have attribute 'arn' for variable 'aws_iam_role.lambda_ecs_task_scheduler.*.arn'
* module.fargate_service.module.iam.aws_iam_role_policy.lambda_ecs_task_scheduler_policy: 1 error(s) occurred:
* module.fargate_service.module.iam.aws_iam_role_policy.lambda_ecs_task_scheduler_policy: Resource 'aws_iam_role.lambda_ecs_task_scheduler' not found for variable 'aws_iam_role.lambda_ecs_task_scheduler.name'

so it would seem like: <https://github.com/blinkist/terraform-aws-airship-ecs-service/blob/master/modules/iam/main.tf#L22> isnt getting created. This is weird, because I set create=true as well as fargate_enabled=true

jaustinpageover 7 years ago

i was curious if there was something simple that i missed, or a pointer of where to look next

Maarten van der Hoefover 7 years ago

Hi @jaustinpage do you have any other errors maybe regarding the creation of IAM resources ? Also, does your user for terraform has AdministratorAccess ?

jaustinpageover 7 years ago

@Maarten van der Hoef: no, no other errors. lemme make 100% certain i have admin...

jaustinpageover 7 years ago

yep, the assumerole i am using has AdministratorAccess

Maarten van der Hoefover 7 years ago

ok, so that's all good

jaustinpageover 7 years ago

im surprised that there isnt a resource creation error, it seems like it never got to the point where it could try to create resources

Maarten van der Hoefover 7 years ago

can you run terraform -v in your root module ?

jaustinpageover 7 years ago

one other potential red herring: when i tried to create the ecs cluster, i did get an error, but then manually applied https://github.com/blinkist/terraform-aws-airship-ecs-cluster/pull/10 to fix

jaustinpageover 7 years ago(edited)

terraform -v
Terraform v0.11.11
+ provider.aws v2.0.0
+ provider.null v2.1.0
+ provider.template v2.1.0

Your version of Terraform is out of date! The latest version
is 0.11.12. You can update by downloading from <http://www.terraform.io/downloads.html|www.terraform.io/downloads.html>

jaustinpageover 7 years ago

ill update tf

Maarten van der Hoefover 7 years ago

ok, let me go through the same loop as you, and see if the new aws provider breaks something

jaustinpageover 7 years ago

terraform -v
Terraform v0.11.13
+ provider.aws v2.0.0
+ provider.null v2.1.0
+ provider.template v2.1.0

jaustinpageover 7 years ago

updated terraform makes no difference, lemme double check my providers...

jaustinpageover 7 years ago

i was 1 version back on the aws provider, lets see if that fixes

jaustinpageover 7 years ago

ahh, gotta manually patch for the owners attribute again...

Maarten van der Hoefover 7 years ago

no wait 🙂

Maarten van der Hoefover 7 years ago

use 0.5.1

Maarten van der Hoefover 7 years ago


module "ecs" {
    source  = "blinkist/airship-ecs-cluster/aws"
    version = "0.5.1"

jaustinpageover 7 years ago

ahhh! nice 🙂

jaustinpageover 7 years ago

much appreciated!

Maarten van der Hoefover 7 years ago

ok, now I'm running ecs service adding

jaustinpageover 7 years ago

yep, no luck with provider aws v2.1.0

jaustinpageover 7 years ago

@Maarten van der Hoef: thank you so much for all the help, by the way, i really appreciate it

Maarten van der Hoefover 7 years ago

ok, i get some other errors regarding policies

jaustinpageover 7 years ago

a different set of errors?

Maarten van der Hoefover 7 years ago



* module.fargate_service.module.iam.aws_iam_role_policy.lambda_lookup_policy: 1 error(s) occurred:

* aws_iam_role_policy.lambda_lookup_policy: Error putting IAM role policy terraform-20190312184616302000000001: MalformedPolicyDocument: The policy failed legacy parsing
	status code: 400, request id: 1e46ae1b-44f7-11e9-ac46-ddd75d8a23d7
* module.fargate_service.module.iam.aws_iam_role_policy.lambda_ecs_task_scheduler_policy: 1 error(s) occurred:

* aws_iam_role_policy.lambda_ecs_task_scheduler_policy: Error putting IAM role policy terraform-20190312184616313900000002: MalformedPolicyDocument: The policy failed legacy parsing
	status code: 400, request id: 1e461169-44f7-11e9-8ea3-e7746315f9c9

jaustinpageover 7 years ago

hmmm, that would mean that it made it further than it did for me i think...

jaustinpageover 7 years ago

i dont know if this helps, but, i ran tf on 0.8.6, then 0.8.8, im not sure if that caused some of the resources to make it further or not...

Maarten van der Hoefover 7 years ago

it should just work, so let me fix this, not sure what's going on

jaustinpageover 7 years ago

ok, let me know what I can do to assist

rssover 7 years ago

0.5.1: Added owners attribute to ecs_ami (#10)
added owners attribute to ecs_ami
owner: self -> amazon

Maarten van der Hoefover 7 years ago

@jaustinpage what do you have at
region =

jaustinpageover 7 years ago

region = "${data.aws_region.current.name}"'

jaustinpageover 7 years ago

Line 0: data "aws_region" "current" {}

Maarten van der Hoefover 7 years ago

ok, that's good

jaustinpageover 7 years ago

the reason for that is im creating the same resources in 2 regions. technically i am calling the airship module from a homespun "region" module, so that i can make all the regions the same.

jaustinpageover 7 years ago

and, i may have doctored the above logs to hide this fact...

Maarten van der Hoefover 7 years ago

we all have our secrets 🙂

Maarten van der Hoefover 7 years ago

so are you using 2 different aws providers atm ?

jaustinpageover 7 years ago

3, but only 2 are having airship called at the moment

Maarten van der Hoefover 7 years ago

and you pass the provider block to both modules

jaustinpageover 7 years ago

the 3rd one is handling some aws credential stuff that is happening behind the scenes

jaustinpageover 7 years ago

i pass the provider block to the homespun region module

jaustinpageover 7 years ago

then use default inheritance from there

Maarten van der Hoefover 7 years ago

ok, but for me to understand

Maarten van der Hoefover 7 years ago

you have one root module, in which you create 3 different aws providers

Maarten van der Hoefover 7 years ago

then you create 2 ecs clusters

jaustinpageover 7 years ago

let me put together some excerpts

Maarten van der Hoefover 7 years ago

you pass both ecs clusters the provider block like:

module "example" {
source = "./example"
providers = {
aws = "aws.usw2"
}
}

Maarten van der Hoefover 7 years ago

and pass the same provider block to the relevant ecs service modules

jaustinpageover 7 years ago

and then the "modules/region" module looks just like a normal demo.

jaustinpageover 7 years ago

inside the modules/region module, because it was handed a single aws provider, we let the default provider inheritance in terraform do the rest of the work

Maarten van der Hoefover 7 years ago

that should cascade down

Maarten van der Hoefover 7 years ago

But i'm not interpolating region names in my resources. Could you interpolate the region name or a short version of it in the name of the ecs service ?

Maarten van der Hoefover 7 years ago

and test

jaustinpageover 7 years ago

sure thing, i would have hoped a name conflict to produce a better error message (which would be a tf/provider bug), but ill give it a shot

jaustinpageover 7 years ago

name = "${var.name}-${terraform.workspace}-${data.aws_region.current.name}"

Maarten van der Hoefover 7 years ago

how long are the names of your workspaces ?

jaustinpageover 7 years ago

'lower' and 'production'

Maarten van der Hoefover 7 years ago

maybe you can substr 0,3 them for now, the ecs cluster name will also be interpolated, and that can result in some issues unfortunately

Maarten van der Hoefover 7 years ago

lets first see if we can get it to work with short names

jaustinpageover 7 years ago

hmm, i am getting the 64char limit now

jaustinpageover 7 years ago

ill find another way to unique them

Maarten van der Hoefover 7 years ago

jaustinpageover 7 years ago

i ran into some other bugs, itl take a sec for me to fix them.

Maarten van der Hoefover 7 years ago

sure

jaustinpageover 7 years ago

yeah, got some name collisions with the change in name above, its just gonna take a sec to remove the existing resource and re-add to make sure that all the resources get recreated

jaustinpageover 7 years ago

ok, back to the initial state

jaustinpageover 7 years ago

of errors that were listed above

Maarten van der Hoefover 7 years ago

Is there a way for you to simplify the setup step-by-step to be able to start ruling out stuff.

jaustinpageover 7 years ago

i take it you are having difficulty reproducing?

Maarten van der Hoefover 7 years ago

the errors I had earlier were something else.

jaustinpageover 7 years ago

yep, ill simplify.

Maarten van der Hoefover 7 years ago

and was a human error on my side, by bluntly copying the ecs_service block without modifying the region 🙂

jaustinpageover 7 years ago

messing with the name of both the ecs and fargate_service modules appears to have moved this on to a new error. thanks so much for your help @Maarten van der Hoef, i think that was it

jaustinpageover 7 years ago

must have been some sort of weird naming collisions just not throwing the errors i would have expected

Maarten van der Hoefover 7 years ago

That happens sometimes, the real errors are hidden, takes some effort to figure those out.

Jamie Nelson (BitFlight)over 7 years ago

Those are the worst.

Jamie Nelson (BitFlight)over 7 years ago

It can take ages to debug resources that have maps or lists that have one wrong or missing value in them.

Jamie Nelson (BitFlight)over 7 years ago

Because the terraform parser that handles it says that the error is in a count, or in another component.

rssover 7 years ago

0.9.0: Load balancer map removal (#47)
Added an example of create = false
Load balancer map removal

johnbeansover 7 years ago

when i run terraform with the airship module it seems to want to try to overwrite the task definition of my service… is there anything i can do to make it skip trying to do that?


  ~ module.fargate_service.module.ecs_service.aws_ecs_service.app_with_lb_awsvpc
      task_definition: "internal-tools-admin-api:15" => "internal-tools-admin-api:12"

Maarten van der Hoefover 7 years ago(edited)

John, can you (privately) paste me your ecs module definition, and disclose a bit more of your plan. Normally the module takes the current taskdefinition unless it's changed. If it is changed it would create a new task definition with the image of the live definition.

rssover 7 years ago

0.9.1
Moving getting_started into this repository as well

Maarten van der Hoefover 7 years ago

rssover 7 years ago

0.9.1.1
fixes for the nlb examples (#56)

rssover 7 years ago

0.9.1.2
variables fix (#57)

rssover 7 years ago

0.9.2: Travis cli install + proper config build trigger (#55)
Travis install + proper config

Install travis gem only within the condition branch=master

February 2019 Browse by date April 2019