#aws - September 2021

48 messages

:aws: Discussion related to Amazon Web Services (AWS)
Archive: https://archive.sweetops.com/aws/

September 2021
Almondovar
Almondovarabout 5 years ago
Hi all, anyone has ever used the EC2 serial console connection? i am getting this message while trying to use it to all of our instances 😕
Daniel Huesca
Daniel Huescaabout 5 years ago(edited)
Hello everybody!

AWS DocumentDB related question - https://github.com/cloudposse/terraform-aws-documentdb-cluster

Can anyone please help me configure my terraform module to NOT create a new parameter group, but instead use the default one provided by AWS (or any previously created param group)?
There is no mention in the docs on how to do this, only a way to pass parameters for the module to create a new one.
Adnan
Adnanabout 5 years ago
Hi People,
anyone ever had this issue with the AWS ALB Ingress controller:
failed to build LoadBalancer configuration due to failed to resolve 2 qualified subnet with at least 8 free IP Addresses for ALB. Subnets must contains these tags: '<http://kubernetes.io/cluster/my-cluster-name|kubernetes.io/cluster/my-cluster-name>': ['shared' or 'owned'] and '<http://kubernetes.io/role/elb|kubernetes.io/role/elb>': ['' or '1']. See <https://kubernetes-sigs.github.io/aws-alb-ingress-controller/guide/controller/config/#subnet-auto-discovery> for more details.

So there three subnets with the appropriate tagging and many ips
I could not yet find the reason why it is complaining about the subnets
Almondovar
Almondovarabout 5 years ago
hi guys, is it any possible way to automate the enablement of ec2 console cable connection in every new ec2 i spin? the commands i am executing for ubuntu instances are the following:

sudo -i
vi /etc/ssh/sshd_config // and go down to edit line 
passwordAuthentication yes 
// saving with :wq!

systemctl restart sshd
passwd // input password 2 times

Grummfy
Grummfyabout 5 years ago
you can play with the cloud-init or user data section of your instance
Alex Jurkiewicz
Alex Jurkiewiczabout 5 years ago
Does the virtual console really use sshd??
Alex Jurkiewicz
Alex Jurkiewiczabout 5 years ago
I would assume a virtual console is using a tty, and bypassing ssh
Steve Wade (swade1987)
Steve Wade (swade1987)about 5 years ago
Does anyone know of anything similar to https://github.com/sportradar/aws-azure-login but written in Go?
Steve Wade (swade1987)
Steve Wade (swade1987)about 5 years ago
I currently have a script (see below) but it seems a little hacky ...

#! /usr/bin/env bash

AWS_PROFILE=${1}

AZURE_TENANT_ID="<redacted>"
AZURE_APP_ID_URI="<redacted>"
AZURE_DEFAULT_ROLE_ARN="arn:aws:iam::<redacted>:role/platform-engineer-via-sso"
AZURE_DEFAULT_DURATION_HOURS=1

# Make sure user has necessary tooling installed.
if ! which ag > /dev/null 2>&1; then
  echo 'Please install the_silver_searcher.'
  exit
fi

# Run the configuration step if not set.
 # shellcheck disable=SC2046
if [ $(ag azure ~/.aws/config | wc -l) -gt 0 ]; then
  printf "Already configured, continuing ...\n\n"
else
  printf "Use the following values when asked for input ... \n"
  printf "Azure Tenant ID: %s\n" ${AZURE_TENANT_ID}
  printf "Azure App ID URI: %s\n" ${AZURE_APP_ID_URI}
  printf "Default Role ARN: %s\n" ${AZURE_DEFAULT_ROLE_ARN}
  printf "Default Session Duration Hours: %s\n\n" ${AZURE_DEFAULT_DURATION_HOURS}
  docker run -it -it -v ~/.aws:/root/.aws sportradar/aws-azure-login --configure --profile "$AWS_PROFILE"
fi

# Perform the login.
docker run -it -it -v ~/.aws:/root/.aws sportradar/aws-azure-login --profile "$AWS_PROFILE"

printf "\nMake sure you now export your AWS_PROFILE as %s\n" "${AWS_PROFILE}"
Santiago Campuzano
Santiago Campuzanoabout 5 years ago(edited)
Does anyone know if it's possible to reserve/allocate a small pool of consecutive Public IP/ELastic IP Addresses on AWS ? I've been searching documentation with no luck
michael sew
michael sewabout 5 years ago
Has anybody used AWS Config Advanced Queries? basically, pulling aws describe data using SQL) i'm trying to pull config data using the aws cli, then throw it into a CSV or some other datastore for querying.
aws configservice  select-aggregate-resource-config  \
--configuration-aggregator-name AllAccountsAggregator  \
--expression "
SELECT
  resourceId,
  resourceName,
  resourceType,
  tags,
  relationships,
  configuration.storageEncrypted,
  availabilityZone
WHERE
  resourceType = 'AWS::RDS::DBInstance'
  AND configuration.engine = 'oracle-ee'
  AND resourceName = 'rds-uat-vertex-9'
" \
| jq -r '.'

.. i'm having problems parsing the outputs. this is mainly a JQ problem.
David
Davidabout 5 years ago
When using shield, is it best to put protections on a Route53 zone, or an ALB that that zone connects to, or both?

And then the same question with Route53 pointing to CloudFront, API Gateway, etc.
Steve Wade (swade1987)
Steve Wade (swade1987)about 5 years ago
before I start writing my own ... does anyone know of a lambda that takes an RDS snapshot and ships it to S3?
Jim Park
Jim Parkabout 5 years ago(edited)
I wrote one for Elasticache, to ship elasticache snapshot to another account and restore it. I’ll put together a gist for you. It’s not RDS, but there may be similar semantics.
michael sew
michael sewabout 5 years ago(edited)
RDS Q: I made a storage modification, but accidentally set to apply-in-maintenance window. How can I turn around and force it to apply-immediately? I'm in storage-full status.
jason einon
jason einonabout 5 years ago
hwy, not sure to post here or terraform... anyone been ale to create a rds read replica in a different vpc via terraform...i have been stuck on this for a fewdays... getting the error:

Error creating DB Instance: InvalidParameterCombination: The DB instance and EC2 security group are in different VPCs.
jason einon
jason einonabout 5 years ago
i am able to apply the desired config through the console but no through Terraform sadly 😞
Adnan
Adnanabout 5 years ago
Hi People,
Wanted to ask about experiences upgrading kubernetes eks versions.
I recently did an upgrade from 1.19 to 1.20.
After the upgrade some of my workloads are experiencing weird high cpu spikes.
But correlation does not equal causation so I wanted to ask if anyone here experienced something similar.
Alyson
Alysonabout 5 years ago
Hi all right with you?
Do you know if there is any web application to make it easier to navigate AWS S3?
AugustasV
AugustasVabout 5 years ago(edited)
Try to describe instances /usr/local/bin/aws ec2 describe-instances --instance-ids i-sssf --region --output text --debug and got that
nmkDIykR/VMOgP+bBmVRcm/QWkCbquedU53R9SAv9deDrjkWkLKuPEnHgu57eGq55K1nFTAVhJ2IG5u5C2IuNKCskgAqz6+JH5fMdlAhYtAzw6FTv+YTi9DFhJaBA9niDk+n2lNhtx/iIbDRNGGCrMXuQbU5hPeHy8ijY6g==', 'Authorization': b'AWS4-HMAC-SHA256 Credential=ASIAUXKPUFZ7UOBXM3GN/20210914/eu-west-1/ec2/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-security-token, Signature=a8d69a78cbf6ac49ba9cc7774d5e9625ec8a2843e7eedeaba2630da7a4a41e1f', 'Content-Length': '76'}>
2021-09-14 14:34:51,592 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): <http://ec2.eu-west-1.amazonaws.com:443|ec2.eu-west-1.amazonaws.com:443>

it’s private EC2 instance, why can’t get the output?
netstat -tnlp | grep :443
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      1013/nginx: maste
Steve Wade (swade1987)
Steve Wade (swade1987)about 5 years ago
does anyone have a clean way of authenticating (via kubectl) to EKS when using Azure AD as the OIDC identity provider?
not sure if people have hooked up Dex with Gangway to provide a UI for obtaining them?
Steve Wade (swade1987)
Steve Wade (swade1987)about 5 years ago
does anyone know if there is a recommended approach to alert on failed RDS snapshot to s3 exports?
Antarr Byrd
Antarr Byrdabout 5 years ago
I'm trying to try out Kinesis using CloudFormation. I'm getting failed invocations when my scheduler invokes the Lamba. But nothing is showing up in Cloudwatch logs. Any ideas how to handle/fix this?
AWSTemplateFormatVersion: "2010-09-09"
Description: "Template for AWS Kinesis resources"
Resources:
  DataStream:
    Type: AWS::Kinesis::Stream
    Properties:
      ShardCount: 1
      RetentionPeriodHours: 24
      Name: !Sub ${AWS::StackName}
  Lambda:
    Type: AWS::Lambda::Function
    Properties:
      Role: !Sub arn:aws:iam::${AWS::AccountId}:role/service-role/lambda_basic_execution
      Runtime: python3.6
      FunctionName: !Sub ${AWS::StackName}-lambda
      Handler: index.lambda_handler
      Code:
        ZipFile: |
          import requests
          import boto3
          import uuid
          import time
          import json
          import random

          def lambda_handler(event, context):
            client = boto3.client('kinesis', region_name='${AWS::Region}')
            partition_key = str(uuid.uuid4())
            response = requests.get('<https://randomuser.me/api/?exc=login>')
            if response.status_code == 200:
              data = json.dumps(response.json())
              client.put_record(
                StreamName='{AWS::StackName}',
                Data=data,
                PartitionKey=partition_key
              )
              print ("Data sent to Kinesis")
            else:
              print('Error: {}'.format(response.status_code))
  Schedule:
    Type: AWS::Events::Rule
    Properties:
      ScheduleExpression: "rate(1 minute)"
      State: ENABLED
      Targets:
        - Arn: !GetAtt Lambda.Arn
          Id: "TargetFunctionV1"
          Input: '{}'
  LogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !Sub /aws/lambda/${AWS::StackName}-lambda
      RetentionInDays: 7
  LogStream:
    Type: AWS::Logs::LogStream
    Properties:
      LogGroupName: !Ref LogGroup
      LogStreamName: !Sub /aws/lambda/${AWS::StackName}-lambda
  PermissionsForEventsToInvokeLambda:
    Type: AWS::Lambda::Permission
    Properties:
      FunctionName: !GetAtt Lambda.Arn
      Action: lambda:InvokeFunction
      Principal: <http://events.amazonaws.com|events.amazonaws.com>
      SourceArn: !GetAtt DataStream.Arn
Shreyank Sharma
Shreyank Sharmaabout 5 years ago(edited)
Hi,

Is it possible to add custom endpoint to AWS Kinesis Signalling Stream endpoint(kinesis.us-east-1.amazonaws.com),

Tried installing a nginx in an ec2 instance and tried to reverse proxy pointing (customendpoint -> kinesis.us-east-1.amazonaws.com) and used certbot to issue certificate to my custom endpoint
but the app is giving
https://<custom-domain>/describeSignalingChannel 404 (Not Found)

Thanks
Ozzy Aluyi
Ozzy Aluyiabout 5 years ago
Hi All, anyone know why my targets are stuck?
Ozzy Aluyi
Ozzy Aluyiabout 5 years ago
Target registration is in progress
Ozzy Aluyi
Ozzy Aluyiabout 5 years ago
it's been trying to register for over and hour now.
Ozzy Aluyi
Ozzy Aluyiabout 5 years ago
any fix/solution will be appreciated.
Omer Sen
Omer Senabout 5 years ago
Hi has anyone ran tfstate backend module with 1.0.7 version of terraform?
Omer Sen
Omer Senabout 5 years ago

│ Error: Unsupported argument
│ 
│   on main.tf line 8, in module "tfstate_backend":
│    8:   force_destroy = true
│ 
│ An argument named "force_destroy" is not expected here.
╵
╷
│ Error: Unsupported argument
│ 
│   on main.tf line 10, in module "tfstate_backend":
│   10:   bucket_enabled   = var.bucket_enabled
│ 
│ An argument named "bucket_enabled" is not expected here.
╵
╷
│ Error: Unsupported argument
│ 
│   on main.tf line 11, in module "tfstate_backend":
│   11:   dynamodb_enabled = var.dynamodb_enabled
│ 
│ An argument named "dynamodb_enabled" is not expected here.
╵
╷
│ Error: Unsupported argument
│ 
│   on main.tf line 13, in module "tfstate_backend":
│   13:   context = module.this.context
│ 
│ An argument named "context" is not expected here.
Omer Sen
Omer Senabout 5 years ago
was using master 😉
Omer Sen
Omer Senabout 5 years ago
that is why
michael sew
michael sewabout 5 years ago
JQ question: I want to get just the environment tag out of a set of RDS instance's tags (pulled from AWS Config advanced queries). Does anybody know how to pull out just the value of the "env" tag for each instance?
aws configservice  select-aggregate-resource-config  \
--expression "
SELECT tags
WHERE resourceType = 'AWS::RDS::DBInstance'
" | jq -r '.Results[]' | jq -r .tags 
[
  {
    "value": "MON/01:00",
    "key": "auto-schedule-start"
  },
	{
    "value": "prod",                  <==== I ONLY WANT THIS
    "key": "env"
  }
]
[
	{
    "value": "dev",
    "key": "env"
  },
  {
    "value": "daily",                 <==== I ONLY WANT THIS
    "key": "backup"
  }
]

my jq attempt:
| jq 'select(.key="env").value'

.. but it's returning all values, not just for the "env" tags. Any JQ folks here can assist? =]
Kian Sajjadi
Kian Sajjadiabout 5 years ago
Anyone here have any experience with setting up privatelink for fargate instances to pull images from ecr?
Alencar Junior
Alencar Juniorabout 5 years ago(edited)
Hi folks, is it possible on EKS to set by default desired capacity in node groups to zero and increase it "automatically" as soon a new service is deployed?
Currently I have a service(DWH) which runs daily for around 2 hours in a m5d.8xlarge instance and then becomes idle. I would like to avoid having that instance running for many hours without using it (currently trying to reduce costs).
Almondovar
Almondovarabout 5 years ago
hi guys i want to enable iam authentication in mariaDB but i have the feeling that its not supported. am i right? or is it because the db is not publicly accesible? as you can see in the screenshot the right one, is mysql and iam auth is enabled, but left one is mariaDB and i dont even see the option to enable it…
michael sew
michael sewabout 5 years ago
Cloudwatch Alarm SNS Question: I want to send Cloudwatch alarms to multiple destinations (2 MS teams channels & pager duty). All use a webhook.

Based on tutorials , it seems I make an SNS topic, make a lambda to translate/send a message to the webhook.

My question: do I need 3 separate lambdas to handle each destination? Or is there some other best practice / tool I should be doing ?
Fabian
Fabianabout 5 years ago
Hi anyone have an idea how long it'll take to restore Automatic Backups for Postgres RDS? I have 4 running for a while. I've also restored Snapshots which are already running.
Darren Cunningham
Darren Cunninghamabout 5 years ago
depends on the size of the backup (maybe storage class - probably not though as I don't think you can change this with automatics) and the size of the instance you've requested
Fabian
Fabianabout 5 years ago
Any rough idea?
Fabian
Fabianabout 5 years ago
I've been restoring for 1h now
Steve Wade (swade1987)
Steve Wade (swade1987)about 5 years ago
What is the recommended approach for alerting to slack on a failed lambda invocation? I have written a rds snapshot to S3 lambda that fires from an event rule but want to know when it fails.
O K
O Kabout 5 years ago
Hey, How can I specify EBS storage for brokers in AWS MSK module https://github.com/cloudposse/terraform-aws-msk-apache-kafka-cluster
Eric Steen
Eric Steenabout 5 years ago(edited)
Hi all, thanks for the amazing work. Does anyone have experience with vpn access to multiple regions using AWS transit gateway? cannot find an example of how to set this up. I am trying to wire up ec2_client_vpn with transit gateway in terraform.
Omer Sen
Omer Senabout 5 years ago(edited)
Hello what do you use to terminate/drain/remove nodes that is on Unready state on aws eks?
Mohamed Habib
Mohamed Habibabout 5 years ago
My codebuild jobs suddenly stopped working. I’m using docker inside codebuilds and it was working well but suddenly now seeing ERROR: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? is anyone experiencing a similar issue ?
michael sew
michael sewabout 5 years ago(edited)
Question on Event Subscriptions: I'm looking at RDS Event subscriptsion to try to connect to pagerduty.
Are there event subscriptions for services OTHER than RDS? I see documentDB, DMS, but I don't see things like EC2, ALB.. do they exist?