#aws - August 2020

Is it possible to use "PropagateTags": "TASK_DEFINITION" when triggering an ECS task with CloudWatch Event rule?

Hi! About https://github.com/cloudposse/terraform-aws-eks-cluster/ and https://github.com/cloudposse/terraform-aws-eks-node-group.
I am currently testing the bugfix implemented in 0.22.0 : https://github.com/cloudposse/terraform-aws-eks-cluster/releases/tag/0.22.0
In the example, https://github.com/cloudposse/terraform-aws-eks-cluster/blob/master/examples/complete/main.tf

data "null_data_source" "wait_for_cluster_and_kubernetes_configmap" {
module "eks_node_group" {
cluster_name = data.null_data_source.wait_for_cluster_and_kubernetes_configmap.outputs["cluster_name"]

I have this in my "main.tf", but when I apply Terraform I get the following error:
Error: Cycle: module.eks_cluster.kubernetes_config_map.aws_auth, module.eks_node_group.module.label.output.tags, module.eks_node_group.aws_iam_role.default, module.eks_node_group.output.eks_node_group_role_arn, module.eks_cluster.var.workers_role_arns, module.eks_cluster.local.map_worker_roles, module.eks_cluster.kubernetes_config_map.aws_auth_ignore_changes, module.eks_cluster.output.kubernetes_config_map_id, data.null_data_source.wait_for_cluster_and_kubernetes_configmap, module.eks_node_group.var.cluster_name, module.eks_node_group.local.tags, module.eks_node_group.module.label.var.tags, module.eks_node_group.module.label.local.tags

Has this been tested like in the example? Thanks!

Hello everyone
Do you mind taking a look into this PR please?
https://github.com/cloudposse/terraform-aws-elasticsearch/pull/63

Hi all,
Do you know if I can share Transit Gateway between regions with RAM in same account?

You should do tgw peering across region

should is hard word 😄 I wanted to avoid that

if i see it right there will be at least one additional tg-attachment in that case ( more if you want to connect more regions ) edit: actually its x2

I was looking at the ASG max instance lifetime setting … the units are seconds but it has a minimum value of 604800 🤔

hello all, The documentation of Application load balancer says SSL termination at LB level...if we configure https listeners for the target... How does the traffic flow from ALB to the target servers? is it not encrypted again from ALB to target servers?

Anybody play with the python 🐍 CDK

Or the CDK in general

I'm interested to see what the terraform CDK adoption is gonna be like

Why would one use the terraform CDK over aws one if only using aws

@Patrick Joyce may be we would want to migrate to different cloud down the line. we never know:)

i think its to write up the terraform code programatically without having to write up terraform manually

if i understand it correctly

1. write cdk in coding language of your choice like python (similar to pulumi)
2. run cdk to generate terraform
3. terraform apply

i've also had use cases where i needed to generate/template the terraform hcl, to workaround some limitation of terraform. that particular use case was addressed by for_each, but i expect other similar cases where generating the hcl from a more expressive language has advantages

maybe also as a different abstraction layer for vars/inputs, a wrapper that takes inputs in your form of choice, and writes the values into the hcl. something of a workaround for the annoying decision to warn (and maybe error) when a tfvars file has undeclared vars

Is there some cloudformation juju to lookup an existing aws resource that is not part of a stack?
ie I have a kms key alias and I need the arn

Yea I suspect that's most common reason, just limitations of hcl

is it possible to use Service Discovery with ECS+EC2 setups ?

Anyone here setup netflix's repokid or ardvaark? Would love to know your deployment, caveats, and ways to simplify getting it setup

Can I track EFS costs per Access Point? In other words when I set an Access Point tag will I be able to see the EFS cost for this tag in Cost Explorer?

Has anyone had issues cloning a git repo over SSH while connected to AWS VPN?

hey, what error are you getting ?

@jason einon I will have the get the exact error, but what happens is that once I’m connected to VPN I’m unable to git clone or git push over SSH

is this for any git repo? its very possible that the vpn connection does not have the correct port open for ssh tcp 22 usually

anyone know of an ssm command line tool where you can specify the command and list of instance ids to run the command ?

Looking like wafv2 doesn’t allow geoblocking for all the evil countries. Is the easiest way to fix this to apply geolocation routing in r53? Where would you route the bad traffic to? An S3 bucket that says “you are not allowed”? or what?

who agree the new EC2 console is ugly and really inconvenient?

@Juan Soto 127.0.0.1

If you send them to S3, it will cost you money. If you send themselves to 127.0.0.1 it will be free 😄

good idea, let me check it

Hello guys,

quick AWS query, I have a use case of hitting private hosted zone domain from public api gateway, the HTTP integration request of the API gateway is not happy with the private hosted zone domain name. Did anyone tried this before?

can i use the same security group in different vpcs ? or do i need to recreate the security group ?

if i have to recreate the security group per vpc, is there a cool aws way to reuse the security group rules (already reusing them at the moment using tf but wondering if there is a better way)

Anyone has some insights into how to appropriate configure an idle timeout for ALB? When a request comes in from a client, i have a rails api, with no nginx involved, that goes to rds and fetch whatever is needed and serves it back as a csv (a very typical workflow i assume...) Are there any downside to just bump up the idle timeout to say 10x default = 600s? Or should i really be looking at nginx of that sort or retweak my app to make it more async?

I'm currently going thru this blog and hope someone can chime in on this topic. 😃
https://sigopt.com/blog/the-case-of-the-mysterious-aws-elb-504-errors/

Hello all, I just wanted to understand the 2 options and how they differ in terms of usage as i'm just not able to differentiate them

1)kms:ViaService

2)kms:GrantIsForAWSResource

Problem: My initial thought of a policy required for user to start ec2 instance which had a CMK key encrypted volume was that i needed to provide decrypt permission with a condition statement for the ec2 instance service so that it can call kms to get the plain text data key on to the memory.

"Action": [
    "kms:Decrypt",

  "Condition": {
    "StringEquals": {
      "kms:ViaService": [
        "ec2.us-west-2.amazonaws.com",



The AWS documentation and a google search shows to use kms:CreateGrant and kms:GrantIsForAWSResource true to allow an user to start EC2 with KMS CMK encrypted volume

  "Action": [
    "kms:CreateGrant",
  ],
  "Resource": "*",
  "Condition": {"Bool": {"kms:GrantIsForAWSResource": true}}

Ello peoples, anyone faced an issue where their user_data script wasn't executed on startup ?

Via the console that is

fixed, forgot to include #!/bin/bash 😐️

classic problem, I have that happen so frequently

Hello, we have EKS workloads running in separate AWS accounts for non-prod and prod environments. I'm thinking of creating a "SharedServices" AWS account and setting up ECR repositories that can be used by both non-prod and prod environments. Any downsides with this approach? Other recommendations?

ello peoples, anyone know of any good resources on implementing ci/cd on aws with terraform. In particular best practices on managing the plan and apply commands in the build phase using codebuild and interacting with s3 state files?

We use an office security group to allow ingress into our vpc. We're approaching the 60 security group rule limit. What's a good way to scale past this limit ?

Does anyone use Aurora (postgres) and love their experiences with it? Considering migrating from RDS (postgres) to it. I know a few years ago there was reliability concerns, but not sure about in 2020 at scale.

MMMM I guess the question is more like, Do you like Aurora in general?

Aurora and aws have way better support for mysql than postgress

aurora storage has some limitations and if heavy write workloads you can easily kill a cluster by writing too fast

if you do not have any of those problems and you do not need to tune up mysql or postgres aurora is great

i think we're not write heavy

if you had Aurora you can check Performance Insights to answer that question lol

hahaha nice. got me there

basically i'm asking the super n00b question of is there a point to cut over to Aurora if RDS is working OK? 🤣

I guess it depends on what you want, if you are looking for automatic failover, updates, elastic storage and HA aurora is nice

and in your experience, in prod, has it been reliable for ya?

yes, no issues but again we did hit the underlaying storage problem I described earlier because we write a huge amount of stuff

apart from that issue it works just fine

so was it like, a lot of writes, caused replication lag, and caused requests to return slowly?

or like, data was lost?

no, we basically find a way to failover the cluster at will by doing specific operations

Is there a full fledged project like a terraform module or something I can use to establish a home account for IAM users + define groups/roles to assume for all users across my accounts? I see a lot of pieces in github, but before i mess around, was wondering if anyone/or other project/ has a “best practice complete layout for home account user provisioning” so I can implement a pull request driven workflow for users provisioning.

Again, I’ve seen pieces, but a full fledged “best practice” layout or service is what I’m wanting to explore tomorrow

I had some issues with lockdowns in Aurora in stress moments, leaving the database zombie. From the AWS perspective the Database is alive as their user (locally runned for monitoring) is able to do stuff, but the cluster stops answering connections until we reboot it. This happened 2 times in 6 months, but apart from that it’s pretty smooth.

(Running Aurora MySQL, for more information)

When using a Lambda to process SQS, are you always using batch size 1 or do you handle failures of messages individually? if the later, how?

for the people who are using https://github.com/Nike-Inc/gimme-aws-creds

I'm switching away from aws-okta and was wondering if anyone has thoughts on aws-okta-processor? I've used it and like it.

https://github.com/godaddy/aws-okta-processor

Does anyone know of a way to set up AWS-VAULT so CloudTrail recognizes that the login is with MFA?

^ similar question but I’ll fork off for gimme-aws-creds if anyone knows how to make CloudTrail recognize that I have an MFA in a session

anyone netflix' dispatch to work ?

https://netflixtechblog.com/introducing-dispatch-da4b8a2a8072

I was wondering if there were any jmespath gurus here, I’ve got the following command

 aws s3api list-buckets --query "Buckets[?starts_with(Name, \`tf-app-npd\`)]|[?contains(Name, \`state\`)].Name"

This works just fine, however it returns the following list

[
    "tf-app-npd-kmstest-state",
    "tf-app-npd-pr1-state",
    "tf-app-npd-shared-state",
    "tf-app-npd-stage-state",
    "tf-app-npd-state"
]

I’d like to exclude buckets such as tf-app-npd-shared-state or tf-app-npd-state, but I’m stuck - any ideas? 🤔

Hello -I would like to keep 100s of GBS of files in sync between to cross account same region S3 buckets with the ability to delete the files from destination bucket when I delete or replace the files in the source bucket ? The s3 replication feature does not solve this issue as S3 does not do replicate delete , the aws s3 sync also wont help here since it would not delete the files from the remote bucket ?

Do I need to build some kind of manifest to keep log of the files which will command what files remains in sync ?

Looks like RDS is down for those using AWS Europe - London Region https://downdetector.co.uk/status/aws-amazon-web-services/

AWS have yet to report on it , status checks still indicate all green

what's a good minimum_protocol_version to set a cloudfront distribution to if it has an acm cert for a static s3 site. I'm currently using TLSv1.1_2016 but I think I should go to TLSv1.2_2019

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudfront_distribution#minimum_protocol_version

Thoughts?

Do you know? Having an Imported certificate in ACM, assigned to some ALB listeners - when updating said imported certificate by uploading a new to ACM - are the load balancer listeners all supposed to propagate to use that new certificate?

Is there a way of using logical OR in IAMs instead of implementing it by writing separate blocks ?

anyone have experience using aws session manager with a .ssh/config, such that a git-over-ssh connection would utilize session manager? we have gitlab running in a private subnet, and would like to support an ssh remote without opening ssh via an ELB in a public subnet...

Does one know how to get codebuild to git clone my codecommit repo instead of zipping it up? I'm unable to execute git commands bc it isn't a git repo

Is it just me or is the EMR spot market in us-west-2 for the past week.. non existant? we’re having capacity trouble for all sorts of instance types

anyone done a cost benefit analysis of migrating ECS to EKS ?

Ok.... I'm done with AWS SSM as my long-term plan. Too slow to iterate and lots of edge bugs for my use.



I want to bring a company wide consistency to config tooling, no more Choco only for windows and Linux left out to dry :-)


Best in class for cross platform and ease of maintenance I'm leaning towards is AWS opsworks puppet enterprise. While we have some ansible already I want state to be checked + run through ssm when possible. Folks here don't use Ruby but lots have dabbled in python

The key requirement is simplify runs when possible by using AWS ssm associations and running through that. Winrm seems problematic in comparison for 200+ instances.

Puppet?

Hi anyone encountered this issue before: UnsupportedAvailabilityZoneException: Cannot create cluster 'eks-cluster-platform' because us-east-1e, the targeted availability zone, does not currently have sufficient capacity to support the cluster ??

I have, in my case I just tried again later and it worked

is there an easy way to see when a new ecs service is being deployed ? if there is an event, i'd like to be able to hit up a slack channel so we can keep track of production deployments

I asked this before.. I was told to use lambda... so looks like no events out-of-the-box

I have used cloudwatch events and lambdas for that in the past

I found this in the archive.. https://github.com/bitflight-public/terraform-aws-ecs-events

nooice

Help. 🙂 Just need to know how to get past this failed helm release. Brand new to this and using a docker release library for gitpod.

Error: cannot re-use a name that is still in use

  on modules/gitpod/main.tf line 9, in resource "helm_release" "gitpod":
   9: resource "helm_release" "gitpod" {

I have no idea how to get it removed or whatever as I don’t see anything successful yet in AWS EKS

#aws
6 tips that I apply to optimize our cost in AWS, check them out and let me know how many of them do you apply?
https://www.dailytask.co/task/6-tips-that-you-should-think-about-them-to-optimize-your-costs-in-aws-ahmed-zidan

what's a good way for the container to know if it has been deployed in fargate or ecs ?

Something I’d like to verify:
The public/private of an Aurora cluster is dependent on the public flag on the instances within the cluster (and of course, routing to the IGW).

Is that correct?

I’m looking here.