#terraform - October 2024

October 2024
Julio Chana
Julio Chanaabout 2 years ago
Hi!

First sorry if this is not the best place to ask, I'll move it to a different channel if it's better,

I'm running into this issue when deploying a helm chart with the module:
terraform {
  source = "git::<https://github.com/cloudposse/terraform-aws-helm-release.git//?ref=0.10.1>"
}

And I'm getting a constant drift for it in the metadata:

  # helm_release.this[0] will be updated in-place
  ~ resource "helm_release" "this" {
        id                         = "myapp"
      ~ metadata                   = [
          - {
              - app_version = "v2.8.6"
              - chart       = "myapp"
              - name        = "myapp"
              - namespace   = "myapp"
              - revision    = 16
              - values      = jsonencode(
                    {
...

Do you know what I can do so the information is properly understood and the drift only happening if there are real changes?

Thank you so much!
Stan V
Stan Vabout 2 years ago
Guys, would you be able to help me with issues I'm facing? I'm trying to deploy AWS EKS cluster with the LB, but I'm getting this error.
Stan V
Stan Vabout 2 years ago
Stan V
Stan Vabout 2 years ago
locals {
  cluster_name = var.cluster_name
}

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "20.15.0"

  cluster_name    = local.cluster_name
  cluster_version = "1.29"

  cluster_endpoint_public_access           = true
  enable_cluster_creator_admin_permissions = true

  cluster_addons = {
    aws-ebs-csi-driver = {
      service_account_role_arn = module.irsa-ebs-csi.iam_role_arn
    }
  }

  vpc_id     = module.vpc.vpc_id
  subnet_ids = module.vpc.private_subnets

  eks_managed_node_group_defaults = {
    ami_type = var.ami_type

  }

  eks_managed_node_groups = {
    one = {
      name = "node-group-1"

      instance_types = ["t3.medium"]

      min_size     = var.min_size
      max_size     = var.max_size
      desired_size = var.desired_size
    }

    two = {
      name = "node-group-2"

      instance_types = ["t3.medium"]

      min_size     = var.min_size
      max_size     = var.max_size
      desired_size = var.desired_size
    }
  }
}

module "lb_role" {
  source = "terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks"

  role_name                              = "shop_eks_lb"
  attach_load_balancer_controller_policy = true

  oidc_providers = {
    main = {
      provider_arn               = module.eks.oidc_provider_arn
      namespace_service_accounts = ["kube-system:aws-load-balancer-controller"]
    }
  }

  depends_on = [
    module.eks
  ]
}

resource "kubernetes_service_account" "service-account" {
  metadata {
    name      = "aws-load-balancer-controller"
    namespace = "kube-system"
    labels = {
      "<http://app.kubernetes.io/name|app.kubernetes.io/name>"      = "aws-load-balancer-controller"
      "<http://app.kubernetes.io/component|app.kubernetes.io/component>" = "controller"
    }
    annotations = {
      "<http://eks.amazonaws.com/role-arn|eks.amazonaws.com/role-arn>"               = module.lb_role.iam_role_arn
      "<http://eks.amazonaws.com/sts-regional-endpoints|eks.amazonaws.com/sts-regional-endpoints>" = "true"
    }
  }

  depends_on = [
    module.lb_role
  ]
}

resource "helm_release" "alb-controller" {
  name       = "aws-load-balancer-controller"
  repository = "<https://aws.github.io/eks-charts>"
  chart      = "aws-load-balancer-controller"
  namespace  = "kube-system"

  set {
    name  = "region"
    value = "eu-west-3"
  }

  set {
    name  = "vpcId"
    value = module.vpc.vpc_id
  }

  set {
    name  = "serviceAccount.create"
    value = "false"
  }

  set {
    name  = "serviceAccount.name"
    value = "aws-load-balancer-controller"
  }

  set {
    name  = "clusterName"
    value = local.cluster_name
  }

  depends_on = [
    kubernetes_service_account.service-account
  ]
}

rss
rssabout 2 years ago(edited)
v1.9.7
1.9.7 (October 2, 2024)
BUG FIXES:

config generation: escape map keys with whitespaces (#35754)
Mauricio Wyler
Mauricio Wylerabout 2 years ago
Hi. I’m using ATMOS for a multi-account multi-enviroment project on AWS… And I love it!
Now I need to deploy multiple times (mostly in the same environment) the same group of terraform components (about 20 components) by changing only the name (I’m using the tenant context to achieve this).
The idea is to have for example, demo-a, demo-b, demo-c, etc…

So, I used GO templates and it works…
ecs/service/application{{ if .tenant }}/{{ .tenant }}{{ end }}:
  ...
ecs/service/api{{ if .tenant }}/{{ .tenant }}{{ end }}:
  ...
...

And then
atmos terraform apply ecs/service/application/demo-a -s uw2-dev
atmos terraform apply ecs/service/api/demo-a -s uw2-dev
...

But I have the feeling there must be a better way to do this… (and probably easier)
Any idea? Thanks.
Leo Przybylski
Leo Przybylskiabout 2 years ago
Is anyone familiar with setting up flink workspaces on confluent?

I noticed there are some flink resources available through the confluent provider. I am not familiar enough to know if this is the best pattern to follow. For example, would it be better to run flink on kubernetes or on self hosted AWS resources? If anyone has some experience with this and can give some insight, I would appreciate it.
RB
RBabout 2 years ago
Did anyone see this ai atlantis song made by a community member? We need more ai songs about our tools lol

https://youtu.be/fThdaeqLDPs
Rishav
Rishavabout 2 years ago
Took a while to find the right combination of actions, but happy to share my guide on securing cloud-provisioning pipeline with GitHub Automation, which spans:
• "keyless" AWS authentication
• Terraform/Tofu IaC workflow
• deployment protections.
(this is my first blog/article in years and super-keen for any feedback, from content to formatting and anything in between -- thank you!)
setheryops
setheryopsabout 2 years ago
Anyone know if the Pluralith project is still alive? They havent had a new release since March of 2023 so im guessing not. It also looks like they are not responding to any issues either.
If it is dead does anyone know of a good alternative?
tretinha
tretinhaabout 2 years ago
Hey, I'm trying Atmos for the first time and I'm trying to set it up with opentofu. I have tofu available in my current path and I have a pretty straightforward (I guess) atmos.yaml file:
base_path: "./"

components:
  terraform:
    command: "tofu"
    base_path: "components/terraform"
    apply_auto_approve: false
    deploy_run_init: true
    init_run_reconfigure: true
    auto_generate_backend_file: false

stacks:
  base_path: "stacks"
  included_paths:
    - "deploy/**/*"
  # excluded_paths:
  #   - "**/_defaults.yaml"
  name_pattern: "{stage}/{region}"

logs:
  file: "/dev/stderr"
  level: Debug

However, when I try something like atmos terraform init -s dev/us-east-1 or atmos terraform init -s dev or atmos terraform init , I get:
exec: "terraform": executable file not found in $PATH

Any ideas? I'm not entirely sure about what I'm missing to make this work. Thank you!
tretinha
tretinhaabout 2 years ago
Hey, I'm trying to execute a plan and I'm getting the following output:
% atmos terraform plan keycloak_sg -s deploy/dev/us-east-1

Variables for the component 'keycloak_sg' in the stack 'deploy/dev/us-east-1':
aws_account_profile: [redacted]
cloud_provider: aws
environment: dev
region: us-east-1
team: [redacted]
tfstate_bucket: [redacted]
vpc_cidr_blocks:
    - 172.80.0.0/16
    - 172.81.0.0/16
vpc_id: [redacted]

Writing the variables to file:
components/terraform/sg/-keycloak_sg.terraform.tfvars.json

Using ENV vars:
TF_IN_AUTOMATION=true

Executing command:
/opt/homebrew/bin/tofu init -reconfigure

Initializing the backend...
Initializing modules...

Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Using previously-installed hashicorp/aws v5.70.0

OpenTofu has been successfully initialized!

Command info:
Terraform binary: tofu
Terraform command: plan
Arguments and flags: []
Component: keycloak_sg
Terraform component: sg
Stack: deploy/dev/us-east-1
Working dir: components/terraform/sg

Executing command:
/opt/homebrew/bin/tofu workspace select -keycloak_sg
Usage: tofu [global options] workspace select NAME

  Select a different OpenTofu workspace.

Options:

    -or-create=false    Create the OpenTofu workspace if it doesn't exist.

    -var 'foo=bar'      Set a value for one of the input variables in the root
                        module of the configuration. Use this option more than
                        once to set more than one variable.

    -var-file=filename  Load variable values from the given file, in addition
                        to the default files terraform.tfvars and *.auto.tfvars.
                        Use this option more than once to include more than one
                        variables file.
Error parsing command-line flags: flag provided but not defined: -keycloak_sg


Executing command:
/opt/homebrew/bin/tofu workspace new -keycloak_sg
Usage: tofu [global options] workspace new [OPTIONS] NAME

  Create a new OpenTofu workspace.

Options:

    -lock=false         Don't hold a state lock during the operation. This is
                        dangerous if others might concurrently run commands
                        against the same workspace.

    -lock-timeout=0s    Duration to retry a state lock.

    -state=path         Copy an existing state file into the new workspace.


    -var 'foo=bar'      Set a value for one of the input variables in the root
                        module of the configuration. Use this option more than
                        once to set more than one variable.

    -var-file=filename  Load variable values from the given file, in addition
                        to the default files terraform.tfvars and *.auto.tfvars.
                        Use this option more than once to include more than one
                        variables file.
Error parsing command-line flags: flag provided but not defined: -keycloak_sg

exit status 1

goroutine 1 [running]:
runtime/debug.Stack()
        runtime/debug/stack.go:26 +0x64
runtime/debug.PrintStack()
        runtime/debug/stack.go:18 +0x1c
<http://github.com/cloudposse/atmos/pkg/utils.LogError({0x105c70460|github.com/cloudposse/atmos/pkg/utils.LogError({0x105c70460>, 0x14000b306e0})
        <http://github.com/cloudposse/atmos/pkg/utils/log_utils.go:61|github.com/cloudposse/atmos/pkg/utils/log_utils.go:61> +0x18c
<http://github.com/cloudposse/atmos/pkg/utils.LogErrorAndExit(|github.com/cloudposse/atmos/pkg/utils.LogErrorAndExit(>{0x105c70460, 0x14000b306e0})
        <http://github.com/cloudposse/atmos/pkg/utils/log_utils.go:35|github.com/cloudposse/atmos/pkg/utils/log_utils.go:35> +0x30
<http://github.com/cloudposse/atmos/cmd.init.func17(0x10750ef60|github.com/cloudposse/atmos/cmd.init.func17(0x10750ef60>, {0x14000853480, 0x4, 0x4})
        <http://github.com/cloudposse/atmos/cmd/terraform.go:33|github.com/cloudposse/atmos/cmd/terraform.go:33> +0x150
<http://github.com/spf13/cobra.(*Command).execute(0x10750ef60|github.com/spf13/cobra.(*Command).execute(0x10750ef60>, {0x14000853480, 0x4, 0x4})
        <http://github.com/spf13/cobra@v1.8.1/command.go:989|github.com/spf13/cobra@v1.8.1/command.go:989> +0x81c
<http://github.com/spf13/cobra.(*Command).ExecuteC(0x10750ec80)|github.com/spf13/cobra.(*Command).ExecuteC(0x10750ec80)>
        <http://github.com/spf13/cobra@v1.8.1/command.go:1117|github.com/spf13/cobra@v1.8.1/command.go:1117> +0x344
<http://github.com/spf13/cobra.(*Command).Execute(...)|github.com/spf13/cobra.(*Command).Execute(...)>
        <http://github.com/spf13/cobra@v1.8.1/command.go:1041|github.com/spf13/cobra@v1.8.1/command.go:1041>
<http://github.com/cloudposse/atmos/cmd.Execute()|github.com/cloudposse/atmos/cmd.Execute()>
        <http://github.com/cloudposse/atmos/cmd/root.go:88|github.com/cloudposse/atmos/cmd/root.go:88> +0x214
main.main()
        <http://github.com/cloudposse/atmos/main.go:9|github.com/cloudposse/atmos/main.go:9> +0x1c

I'm not really sure what I can do since the error message suggests an underlying tofu/terraform error and not an atmos one. I bet my stack/component has something wrong but I'm not entirely sure why. The atmos.yaml is the same of the previous message I sent here yesterday. I'd appreciate any pointers
rss
rssabout 2 years ago
v1.10.0-alpha20241009
1.10.0-alpha20241009 (October 9, 2024)
NEW FEATURES:

Ephemeral resources: Ephemeral resources are read anew during each phase of Terraform evaluation, and cannot be persisted to state storage. Ephemeral resources always produce ephemeral values.
Ephemeral values: Input variables and outputs can now be defined as ephemeral. Ephemeral values may only be used in certain contexts in Terraform configuration, and are not persisted to the plan or state files.

terraform output -json now displays...
toka
tokaabout 2 years ago
I was trying to use https://registry.terraform.io/modules/cloudposse/stack-config/yaml/1.6.0/submodules/remote-state and found out that actually it's a submodule of yaml-stack-config module.

Since every submodule of yaml-stack-config is using context (null-label) that got me thinking:
if terraform-provider-context is meant to supersede null-label , should I even start using yaml-stack-config when starting my codebase pretty much from scratch?
RB
RBabout 2 years ago
muhaha
muhahaabout 2 years ago
Ola 👋
I am struggling with loop ( over subnets ) and this structure
variable "vpcs" {
  description = "List of VPCs"
  type        = list(map(any))

  default = [
    {
      name    = "vpc-1"
      cidr    = "10.0.0.0/16"
      subnets = [
        {
          name = "subnet-1"
          cidr = "10.0.1.0/24"
        },
        {
          name = "subnet-2"
          cidr = "10.0.2.0/24"
        }
      ]
    },
    {
      name    = "vpc-2"
      cidr    = "10.0.0.0/16"
      subnets = [
        {
          name = "subnet-3"
          cidr = "10.0.3.0/24"
        },
        {
          name = "subnet-4"
          cidr = "10.0.4.0/24"
        }
      ]
    }
  ]
}

any ideas? something like
for_each = { for v in var.vpcs, s in v.subnets : "${v.name}-${s.name}" => s }
Mark Johnson
Mark Johnsonabout 2 years ago(edited)
Would love to get some feedback on my latest project -- Terraform Module Releaser – a GitHub Action that automates versioning, tags, releases, and docs for Terraform modules in monorepos! Great for module collections in a single repo with tags representing logical folders/tf roots. Spent a lot of time to make this literally a drop-in for anyone using GHA with defaults sanely configured.

Check it out here: [https://github.com/techpivot/terraform-module-releaser]
Feel free to star if you find it useful! ⭐️ Happy to answer any Qs in 🧵 as well!
Prashant
Prashantabout 2 years ago(edited)
👋 Hello, team! This is about Rotate .pem for AWS EKS nodes, and the.pub and .private keys are generated using the source URL listed below.

https://github.com/cloudposse/terraform-aws-key-pair/blob/main/main.tf
Whenever we update the key pair name in the keypair.tf, the key pair name changes, but the .private and.pub keys remain the same as earlier.

Need change the .pub and .private keys for the EKS nodes.
rss
rssabout 2 years ago(edited)
v1.9.8
1.9.8 (October 16, 2024)
BUG FIXES:

init: Highlight missing subdirectories of registry modules in error message (#35848)
init: Prevent crash when loading provider_meta blocks with invalid names (<a href="https://github.com/hashicorp/terraform/pull/35842" data-hovercard-type="pull_request"...
Rishav
Rishavabout 2 years ago
Say you have Terraform running as part of your provisioning workflow, and the plan output is returned as a PR comment.

For a large enough project, you're likely to run into GitHub's character limit for comments. In that situation, would you prefer:

• 1️⃣ the first thousands of characters
• 2️⃣ the last thousands of characters
Before rushing to the second option because the plan summary is stored within the last few lines, consider that plan summary is pulled out at the top of the PR comment already.

So which "end" is more useful to you? And would the same "end" be useful from the apply command's output as well?
Leon Katz
Leon Katzabout 2 years ago
I'm running into an issue when adding routes for peering connections. I get the existing routes that exist for all subnets in a VPC and then try to use count to go through them and add a route. But it won't let me use a data element to get routes unless I comment out the aws_route that I'm trying to create then, after it gets the data I can uncomment to run it. This is the error
The "count" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. To work around this, use the
│ -target argument to first apply only the resources that the count depends on.
Leon Katz
Leon Katzabout 2 years ago
I tried assigning the data request to a local var, but still got the error
rss
rssabout 2 years ago(edited)
v1.10.0-alpha20241023
1.10.0-alpha20241023 (October 23, 2024)
NEW FEATURES:

Ephemeral resources: Ephemeral resources are read anew during each phase of Terraform evaluation, and cannot be persisted to state storage. Ephemeral resources always produce ephemeral values.
Ephemeral values: Input variables and outputs can now be defined as ephemeral. Ephemeral values may only be used in certain contexts in Terraform configuration, and are not persisted to the plan or state files.

terraform output -json now displays...
Bob Berg
Bob Bergabout 2 years ago
Hello, I previously submitted a pull request for Cloud Posse Terraform Datadog Platform, https://github.com/cloudposse/terraform-datadog-platform/pull/107. May I get some eyes on it or be directed to where I may find an approval? It's a simple one-line-change that fixes Advanced Scheduling for Synthetics Tests.
Slackbot
Slackbotabout 2 years ago
This message was deleted.
Roberts Jānis Sīklis
Roberts Jānis Sīklisabout 2 years ago(edited)
Hi folks, fairly new to terraform and have a few questions on how to approach a problem.

I wish to create a web portal where it is possible to provision an entire project with different environments (dev, prod, e.t.c).
Idea is to create a singular internal module which would include all of my needed public modules for example EC2, RDS, S3 and others.
For DEV and PROD environments I want default variables to already contain all the logic what these environments need and if needed the module caller can edit those values.
Result could be for example module caller could deploy and entire dev environment without providing any variables.

Question is it a fine approach to create this singular module this way?
Any suggestions would help! Thanks you!
Vitalii
Vitaliiabout 2 years ago
hello guys
I am playing Atmos and GitHub for my project
currently, I am having a problem with posting comments to GitHub pull requests with atmos terraform plan <stack> -s #####
I cant parse output in a readable this relative to terraform -no-color`
my question is:
can I run atmos terraform plan <stack> -s ##### -no-color or any other arguments that will be relative to -no color
In the documentation, I didn`t find anything about it
If you know any other way how I can post comments to pull requests in a readable way via atmos, or some parse way to parse
please help
appreciate any help
Brett Au
Brett Auabout 2 years ago
I am having a problem with the elasticache-redis module, when I turn on the variable
allow_ingress_from_this_vpc

I get the following error

│ Error: Inconsistent conditional result types
│
│   on .terraform/modules/redis_clusters.redis.aws_security_group/normalize.tf line 81, in locals:
│   81:   all_ingress_rules = local.inline ? [for r in local.all_inline_rules : r if r.type == "ingress"] : []
│     ├────────────────
│     │ local.all_inline_rules is tuple with 3 elements
│     │ local.inline is false
│
│ The true and false result expressions must have consistent types. The 'true' tuple has length 2, but the 'false' tuple has length 0.
╵
exit status 1
rss
rssabout 2 years ago(edited)
v1.10.0-beta1
1.10.0-beta1 (October 31, 2024)
NEW FEATURES:

Ephemeral resources: Ephemeral resources are read anew during each phase of Terraform evaluation, and cannot be persisted to state storage. Ephemeral resources always produce ephemeral values.
Ephemeral values: Input variables and outputs can now be defined as ephemeral. Ephemeral values may only be used in certain contexts in Terraform configuration, and are not persisted to the plan or state files.

terraform output -json now displays ephemeral...