#aws - August 2022

I have an unusual situation with a client. They manage many remote sites and have physical devices (up to 20) at each location. Each device needs to send metrics to cloudwatch and upload files to S3 and they currently use static aws credentials (/.aws/credentials). I would like to move them to IAM anywhere to use temporary credentials. The ask is if a device gets compromised how can we disable access to AWS from that particular device. I was thinking to use an IAM Role per device however they are expecting to have 10k devices online by the end of the year. I’d use Terraform to manage the roles and AWS organizations to use multiple accounts since there’s a 5k IAM role quota per account. Does this sound manageable? or is there a better approach?

Hey everyone! Does anyone have an example of a CloudWatch Agent config. with a multi_line_start_pattern and/or filter.expression that includes RE2 escaped characters? I'm having a devil of a time getting this to work as expected. I've tried single escapes (e.g. \] for the literal ] character) and double-escapes (e.g. \\] for the same literal) and neither seems to be working right. For what it's worth, the configuration is deployed with Ansible, so the inputs are YAML and converted to JSON on the fly when the values are interpolated in the template file.

For example, any filters that I specify get translated as follows (snippet from a Jinja2 template file for the agent config):

  "filters": {{ __log_file.filters | to_json }},

The templating is working as-expected; I'm just not sure the final syntax is right. I've been checking against the RE2 syntax but I can't find a good example with escaped characters. Thanks for any help!

Hey everyone! Is it possible to attach multiple load balancers to ECS service like ALB for internal user and NLB for external use ?

hello all,
We are using aurora serverless v1 and few of them has a strange issue. Sometimes it dropts all connections. In error log around that time I see the db restarted. but did not see anything before that. any idea ?

Hi All, anyone who got experience with https://vantage.sh/ or other cost analyser?

We are testing v2 aurora... is that possible that v2 is slower than v1 on same size?

Need some help with kinesis agent not spamming to kinesis... i can see stuff happening in /var/log/aws-kinesis-agent/aws-kinesis-agent.logto the right kinesis, but i just dont see any blip from kinesis monitoring, anything else im missing?

Not sure where else to post ist so I'll try it here.

I have a bucket with CORS configuration allowing only certain domains as origins.
I am trying to use Cloudflare image resizing /cdn-cgi/image proxy but I get CORS errors when accessing something like:

<https://www.example.com/cdn-cgi/image/https://my-bucket.s3-eu-central-1.amazonaws.com/images/myimage.jpg>

It works when I allow all origins on my bucket but I don't want to do that.

Anybody had experience with something like this?

Looking for something to run custom devops scripts from. What do people tend to use ATM? Lamdba/Rundeck/Airflow/ArgoWorkflows? We use AWS with EKS so a k8s solution is an option.

I just downloaded the terraform codebuild project but make keeps erroring out

i want to make sure I understand everything correctly, this is a template which will allow me to automate my codebuild projects, correct?

please message me with a reply as I am going to head to bed and will check back when i wake up

i dont want the conversation to be buried

hey is anyone around?

i was able to migrate the github project into a place where there is no spaces, now i am getting curl could not resolve host cloudposse.tools when i try make Makefile

is there some other guides with more details in it?

i would like to know where exactly you are supposed to customize the variables as well as which ones you shouldnt customize

anyone around?

I guess not

hmm, it is friday

i really wanted to give the build a shot but its hard without detailed info

ive been trying to find an easy way to build out our codebuild projects for our ci/cd pipeline

it is a bit of a pain to create projects for each micro-service manually

any help would be much appreciated

You are using github, just use GH actions. Codebuild is almost never the answer

i rely heavily on the buildspec part

deploy out to my eks cluster

works really well

The cloudposse.tools domain resolves for me

i downloaded the file just as it was and tried compiling

There is no command that I'm familiar with that is make Makefile 🤔

What problem are you trying to solve?

Are you trying to consume the terraform module?

how do you run it?

i thought it needed to be compiled and that was what the Makefile was

hence why i ran that

I would follow the readme. There should be a usage section

thats the problem, i did

its not clear

You create a new directory, copy and paste the terraform code from the usage section, then run terraform init and terraform plan

That's it :)

ok, let me try that

Here's a good blog post on using modules in general

https://spacelift.io/blog/what-are-terraform-modules-and-how-do-they-work

i appreciate the guidance @RB (pat leave again)

havent really used terraform

read some about it

but this is the first time i am actually trying to use it

i usually stick to cloudformation

Let's move this over to the #terraform channel

sure

I need to know about lambda workflow we have code on S3
every time it will fetch the code while execute or it cache some where

Code Source

?

Any one please

ok @Warren Parad

Hi all!

Has anyone here have any experience with docker compose - ecs integration? I am trying to make service discovery work but the the cloudmap namespace can not be resolved inside the ecs container. Any help is appreciated!

I am currently having problems pulling aws docker images from their public registries:

$ docker pull <http://602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon/aws-load-balancer-controller:v2.4.3|602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon/aws-load-balancer-controller:v2.4.3>
Error response from daemon: Head "<https://602401143452.dkr.ecr.us-west-2.amazonaws.com/v2/amazon/aws-load-balancer-controller/manifests/v2.4.3>": no basic auth credentials

$ docker pull <http://602401143452.dkr.ecr.eu-west-1.amazonaws.com/amazon/aws-load-balancer-controller:v2.4.3|602401143452.dkr.ecr.eu-west-1.amazonaws.com/amazon/aws-load-balancer-controller:v2.4.3>
Error response from daemon: Head "<https://602401143452.dkr.ecr.eu-west-1.amazonaws.com/v2/amazon/aws-load-balancer-controller/manifests/v2.4.3>": no basic auth credentials

Can anyone relate?
From inside or outside the AWS network, with or without docker-login credentials, still same error

Hi all!
Found a bug? What can I do? Please help me.
Thanks.
Murphy.

The aws version is 4.25.0.

Hi everyone. I'm looking for a way to set log retention with aws_glue_job, but the result doesn't seem to work as it suggests that it should in the documentation. It just occurred to me, as I'm writing the question, that I've been trying to specify the logging parameters as NonOverridableArguments, so I'm going to look into not doing that, but I was curious if anyone knew of a resource that had a clear explanation of how the different options of logging behaved ( including the unrelated security_configuration object that also includes settings for logs, and appears to mandate the name of the log group. )

Ran into an issue with the most recent changes to the Cloudposse SFTP Transfer module. The latest PR adds functionality to set home directories via the sftp_user input variable (very helpful). I noticed if users are created with the restricted_home variable set to true, and then later that variable is set to false; the in-place updates to users fail with the following errors:

Error: error updating Transfer User (redacted): InvalidParameter: 1 validation error(s) found.
│ - minimum field size of 1, UpdateUserInput.HomeDirectoryMappings.
│ 
│ 
│   with module.transfer-sftp.aws_transfer_user.default["redacted"],
│   on .terraform/modules/transfer-sftp/main.tf line 53, in resource "aws_transfer_user" "default":
│   53: resource "aws_transfer_user" "default" {
│ 
╵
╷
│ Error: error updating Transfer User (redacted): InvalidParameter: 1 validation error(s) found.
│ - minimum field size of 1, UpdateUserInput.HomeDirectoryMappings.
│ 
│ 
│   with module.transfer-sftp.aws_transfer_user.default["redacted"],
│   on .terraform/modules/transfer-sftp/main.tf line 53, in resource "aws_transfer_user" "default":
│   53: resource "aws_transfer_user" "default" {
│ 
╵

Can we get s3 bucket access logs to cloudwatch logs ?

👋 Hello, team!

Hey all, possibly-stupid question: can AWS Support plans be set up with Terraform? Or is there some other way of sensibly managing support plans across a nontrivial org?

hello!

has anyone ever deployed an SSR nextjs app on AWS? I am experimenting with Amplify and want to ask what are some problems you all had with Amplify hosting. Thanks!

Hello team,
During effort to use newer version of terraform module cloudposse/terraform-aws-emr-cluster I come accross issue and before opening Issue on github I am suggested in template to ask for support here first.

While trying to update from version tag 0.23.0 and use latest version tag 1.0.0 I come on following error during terraform plan:

 Error: Unsupported block type

  on ../.tfmodules/terraform-aws-emr-cluster/main.tf line 523, in resource "aws_emr_cluster" "default":
 523:   dynamic "auto_termination_policy" {

Blocks of type "auto_termination_policy" are not expected here.

Any suggestion how to proceed or to open issue on github instead?

Tailscale - I have been using tailscale to connect to my private resources and it has been working great! However, I ran into a big problem recently. I got some contractors and shared my node with them. I just found out that it doesn't share the Subnet relayed traffic (reference: https://tailscale.com/kb/1084/sharing/#sharing--subnets-subnet-routers) Anyone have alternatives to tailscale that will help solve my problem?

I did think of creating a Gmail Account for them with the same domain as me but that would be an extra cost for each contractor just to have access to Tailscale.

Anyone use AWS code artifact? love it? hate it? We are considering it for pip. CI would push (already uses OIDC with circle CI), and devs would pull for docker container based environments

Hey guys, Is AWS subnets operate at network layer 2?