#atmos - February 2024

Question on how to handle stack-unique configuration files in atmos. 🧵

Greetings.. running through a pared down version of
https://atmos.tools/design-patterns/organizational-structure-configuration

I was able to run atmos terraform deploy vpc-flow-logs-bucket -s org1-plat-ue2-prod without a problem

Then when I run atmos terraform deploy vpc -s org1-plat-ue2-prod I'm getting the following error:
╷
│ Error: stack name pattern '{namespace}-{tenant}-{environment}-{stage}' includes '{environment}', but environment is not provided
│
│ with module.vpc_flow_logs_bucket[0].data.utils_component_config.config[0],
│ on .terraform/modules/vpc_flow_logs_bucket/modules/remote-state/main.tf line 1, in data "utils_component_config" "config":
│ 1: data "utils_component_config" "config" {
│
╵
exit status 1

Here is the output of atmos describe component vpc --stack org1-plat-ue2-prod

We're still on 1.44 and have been in the weeds so didn't see all of the new stuff, but I read the latest releases since then and major kudos ❗️❗️❗️ on the latest work. This is looking phenomenal and will upgrade soon.

v1.57.0
what

Add default CLI configuration to Atmos code
Update/improve examples and docs
Update demo.tape

why

Add default CLI configuration to Atmos code - this is useful when executing Atmos CLI commands (e.g. on CI/CD) that does not require components and stacks

If atmos.yaml is not found in any of the searched locations, Atmos will use the default CLI configuration:
base_path: "."
components:
terraform:
base_path: components/terraform
apply_auto_approve: false
deploy_run_init:...

Hello 🙂 When using atmos github actions for terraform drift detection, I saw an example config like below. How can I specify all components? How can I specify components in specific folders?

   select-components:
      runs-on: ubuntu-latest
      name: Select Components
      outputs:
        matrix: ${{ steps.components.outputs.matrix }}
      steps:
        - name: Selected Components
          id: components
          uses: cloudposse/github-action-atmos-terraform-select-components@v0
          with:
            jq-query: 'to_entries[] | .key as $parent | .value.components.terraform | to_entries[] | select(.value.settings.github.actions_enabled // false) | [$parent, .key] | join(",")'
            debug: ${{ env.DEBUG_ENABLED }}

in the atmos terraform plan with github actions, the docs on website says Within the "plan" job, the "component" and "stack" are hardcoded (foobar and plat-ue2-sandbox). In practice, these are usually derived from another action..

Is there an example that practically uses components from “affected stacks”

I see it is using component as key in the yaml a lot, does that mean it support config one component? How we config multiple components in this case?

Hi, when using Atmos + cloudposse components to setup a multi-account AWS organization setup (accounts for identity, dns, audit, ...). Say we have a customer who is providing access to an AWS account within their own organization through a role we can assume from one of our own IAM roles. How would we be able to assume this role within our cloudposse setup so we can still use atmos & cloudposse components and store terraform state (S3) and locking (DynamoDB) on our own account while provisioning the actual infrastructure on the customer's account?

Hi, is there a document explaining why Atmos runs a reconfigure? As I look at example atmos.yamls, the default appears to alway reconfigure. Everytime I run a plan, even for the same component and stack in succession, it's constantly asking to migrate all workspaces

the one caveat is that I'm playing around right now and using local state

Hi, I was trying to deploy the https://github.com/cloudposse/terraform-aws-components/tree/main/modules/waf module, but I had a hard time figuring out how to use and_statement or the or_statement or the not_statement from the rules. I know I can do that with straight TF, but I can’t seem to be able to do that with Cloudposse module. Also, is Rule Groups not supported?
Can someone please shed some lights? Thank you in advance.


The following snippet is what I had, but I was only able to specify one statement.

byte_match_statement_rules:
  - name: "invalid-path"
    priority: 30
    action: block

    statement:
      field_to_match:
        uri_path:
          rule:
      positional_constraint: "STARTS_WITH"
      search_string: "/api/v3/test/"
      text_transformation:
        rule:
          priority: 0
          type: "NONE"

    visibility_config:
      # Defines and enables Amazon CloudWatch metrics and web request sample collection.
      cloudwatch_metrics_enabled: true
      metric_name: "uri_path"
      sampled_requests_enabled: true

v1.58.0
what

Improve Atmos UX and error handling


When a user just types atmos terraform or atmos helmfile, Atmos will show the corresponding Terraform and Helmfile help instead of checking for a component and stack and printing error messages if the component or stack is not found


If a user executes any Atmos command that requires Atmos components and stacks, including just atmos (and including from a random folder not related to Atmos configuration), and the CLI config points to an Atmos stacks...

v1.58.0
what

Improve Atmos UX and error handling


When a user just types atmos terraform or atmos helmfile, Atmos will show the corresponding Terraform and Helmfile help instead of checking for a component and stack and printing error messages if the component or stack is not found


If a user executes any Atmos command that requires Atmos components and stacks, including just atmos (and including from a random folder not related to Atmos configuration), and the CLI config points to an Atmos stacks...

v1.59.0
what

Update intro of Atmos (https://atmos.tools/)
Add page on Terraform limitations (https://atmos.tools/reference/terraform-limitations/)
Add backend.tf.json to .gitignore for QuickStart
Default to dark mode
Stylize atmos brand

why

Make it more compelling
Add missing context developers might lack without extensive terraform experience

v1.59.0
what

Update intro of Atmos (https://atmos.tools/)
Add page on Terraform limitations (https://atmos.tools/reference/terraform-limitations/)
Add backend.tf.json to .gitignore for QuickStart
Default to dark mode
Stylize atmos brand

why

Make it more compelling
Add missing context developers might lack without extensive terraform experience

v1.60.0
what

Fix an issue with the skip_if_missing attribute in Atmos imports with context
Update docs titles and fix typos
Update atmos version CLI command

why


The skip_if_missing attribute was introduced in Atmos release v1.58.0 and had some issues with checking Atmos imports if the imported manifests don't exist


Docs had some typos


When executing the atmos version command, Atmos automatically checks for the latest...

This page alone deserves a conf to reveal all its gems and secrets! 🤩

https://atmos.tools/reference/terraform-limitations/

Is there a way to visualize the atmos stacks when using github actions to plan and apply? Or is it on the roadmap?

I tried using the component.yaml's mixins key to copy over my local providers file and it failed.

Does that key only work with the source?

Im trying to run cloudposse/github-action-atmos-terraform-plan but Im getting this error

Run cloudposse/github-action-atmos-get-setting@v1
Error: SyntaxError: Unexpected token 'F', "
Found stac"... is not valid JSON

Error: SyntaxError: Unexpected token 'F', "
Found stac"... is not valid JSON

Hello, I am struggling to understand how atmos can provision resources into different AWS accounts in a multi-account AWS organization. For example, if I need to provision an IAM role in all my AWS accounts in my organization, how does atmos change provider configurations to gain access to my organization's child accounts?

v1.61.0
what

Update readme to be more consistent with atmos.tools
Fix links
Add features/benefits
Add use-cases
Add glossary

why

Better explain what atmos does and why

v1.62.0
Add atmos docs CLI command @aknysh (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2133369695" data-permission-text="Title is private" data-url="https://github.com/cloudposse/atmos/issues/537" data-hovercard-type="pull_request"...

v1.62.0
Add atmos docs CLI command @aknysh (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2133369695" data-permission-text="Title is private" data-url="https://github.com/cloudposse/atmos/issues/537" data-hovercard-type="pull_request"...

Hey all! some notable updates to the atmos docs. First, you can now open the from the command line. Just run atmos docs

💫 Here are some notable additions.

Best Practices for Stacks. https://atmos.tools/core-concepts/stacks/#best-practices
Best Practices for Components. https://atmos.tools/core-concepts/components/#best-practices
Added an FAQ. https://atmos.tools/faq
Challenges that led us to writing atmos. https://atmos.tools/reference/terraform-limitations

v1.63.0
Add integrations.github to atmos.yaml @aknysh (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2133610579" data-permission-text="Title is private" data-url="https://github.com/cloudposse/atmos/issues/538"...

v1.63.0
Add integrations.github to atmos.yaml @aknysh (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2133610579" data-permission-text="Title is private" data-url="https://github.com/cloudposse/atmos/issues/538"...

You guys are on fire with the Atmos changes

Looks great

Hello, I see you have this to support short form of aws region and zones, https://github.com/cloudposse/terraform-aws-utils#introduction do you have something similar for GCP

Hello, I need to use multiple modules from GCP module, can I config multiple source in component.yml.? it does not look like it support it. I should not use vendor.yaml in my case since I am not pulling it for the entire infra, just for that component. I did see that cloudposse solve this issue by adding another module to the main module so it only config one ex it needs to use both efs and kms module, instead of pulling two modules, it only need to pull efs since kms is also defined in efsmain.tf

Is there other way to solve this issue?

Hello, for the label order described here https://github.com/cloudposse/terraform-null-label
I see we can config label order as we would like. Does it work if the label order is {namespace}-{tenant}-{environment}-{stage} and the folder structure in stacks follows a different order namespace/stage/tenant/environment structure? I think it works, but would like to double check with the expert here. If it does work, is there any disadvantage of doing that

Interesting to follow https://github.com/opentofu/opentofu/issues/685#issuecomment-1945123152: use of the tf lockfiles ...

set the channel topic: 👽️

Hello, I am currently looking for tooling to optimize our Terraform environments. We previously used Terragrunt in our organization, but it has become too cumbersome over time and feels kind of "previous generation" tooling. We are in the process of setting up a PoC with Terramate, and it's super neat. Especially the orchestration is powerful. We also looked at Spacelift, but I don't see any reason to migrate to another CI/CD when we can get to a similar UX in GitHub Actions.

However, I just came across Atmos on Reddit and would love to understand how it compares to Terramate and Terragrunt.

How do you apply all stacks in a pipeline? If I want my pipeline to run atmos terraform apply, can I do an all flag instead of listing the stack and component? This is with GitHub Actions

How does one wrap a custom go binary around atoms cli ? I have a go binary to validate vpc connectivity when attaching vpc to transit gateway , I would like to run it as part of terraform execution . Has anyone tried this usecase?

v1.64.0
Create Landing page @osterman (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2137898667" data-permission-text="Title is private" data-url="https://github.com/cloudposse/atmos/issues/540" data-hovercard-type="pull_request"...

v1.64.1
🚀 Enhancements

Fix responsiveness of css @osterman (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2151917720" data-permission-text="Title is private" data-url="https://github.com/cloudposse/atmos/issues/543"...

v1.64.1
🚀 Enhancements

Fix responsiveness of css @osterman (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2151917720" data-permission-text="Title is private" data-url="https://github.com/cloudposse/atmos/issues/543"...

Hi all. Just wondering, should website or github action changes trigger a release? I always figured these changes would get a no-release label since there aren't any changes to atmos cli

FYI, the https://github.com/cloudposse/terraform-provider-utils PGP key has been added to the OpenTofu Registry to sign the provider

https://github.com/opentofu/registry/blob/main/keys/c/cloudposse/provider.asc

I’m having some trouble using the upstream providers.tf to assume the -admin suffixed role instead of the -terraform suffixed role when running atmos command locally.

Regarding the tfstate-bucket component. Is there an alternative suggested account to deploy this bucket in? I don't want to deploy it in root as i don't want anyone to access root.

Hi I have a quick question around templating and the likes can this be used anywhere or just in certain places. I tried using it like below
workspace_key_prefix: "infra-{{ .tenant }}-{{ .namespace }}-{{ .environment }}-{{ .stage }}-init"
but it does not render the values and keeps them as is when I do a atmos describe stacks

Hey folks,

We have the following atmos config.

stacks:
  base_path: "stacks"
  included_paths:
    - "org/**/*"
  name_pattern: "{stage}"
  excluded_paths:
    - "**/_*.yaml" # Do not consider any file beginning with "_" as a stack file

We made a classic mistake: Our application was simple and we didn't have multi-region on this project, so we went ahead and used name_pattern to be equal to just {stage} . We are of course now doing some disaster recovery work on this project and now adding a 2nd region, and this becomes a problem as we have Atmos thinking both are the same stage i.e. the VPC in ue1-dev can't be differentiated from the VPC in uw2-dev.

Simple solution would be to prefix our component instance names in the dev/us-west-2.yaml with uw2-***** , but that feels rough. Are there any other suggestions or way to migrate this name pattern issue that we should try out?

v1.64.2
Add Atmos CLI command aliases @aknysh (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2158013928" data-permission-text="Title is private" data-url="https://github.com/cloudposse/atmos/issues/547" data-hovercard-type="pull_request"...

@RB (pat leave again) ☝️ now you can do a tf with aliases 😃

v1.64.2
Add Atmos CLI command aliases @aknysh (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2158013928" data-permission-text="Title is private" data-url="https://github.com/cloudposse/atmos/issues/547" data-hovercard-type="pull_request"...

@Andriy Knysh (Cloud Posse) not really a patch version with the added functionality ...

v1.64.2
Add Atmos CLI command aliases @aknysh (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2158013928" data-permission-text="Title is private" data-url="https://github.com/cloudposse/atmos/issues/547" data-hovercard-type="pull_request"...

I have a pipeline where I am running a atmos terraform plan and then apply with one resource and it deploys successfully:

- name: Atmos - Terraform Plan 
  run: 
    atmos terraform plan resource1 -s orgs/dir/fake/sandbox/us-central1/resource1

Then when I add another component from the same stack and attempt a plan, it wants to destroy the previous resource I created because ot os (not in the configuration):

- name: Atmos - Terraform Plan 
  run: 
    atmos terraform plan resource1 -s orgs/dir/fake/sandbox/us-central1/resource1
    atmos terrafora plan resource2 -s orgs/dir/fake/sandbox/us-central1/resource2

Why is this happening and how can I resolve that? Resources are for GCP and pipeline is GHA

Hey just an FYI dropped a little PR to prevent atmos from crashing when using the azurerm backend and not providing a global key https://github.com/cloudposse/atmos/pull/548

v1.64.3
🚀 Enhancements

include some protection on the azurerm backend when global does not exist @aochsner (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2161523307" data-permission-text="Title is private"...

v1.64.3
🚀 Enhancements

include some protection on the azurerm backend when global does not exist @aochsner (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2161523307" data-permission-text="Title is private"...

Hi,
I am trying to integrate Atlantis with Atmos i have generated varfile pushed it to gitlab repo and generated atlantis.yaml file too but i am getting an error while running the atlantis plan command
Any help would be appreciated

Error: Failed to read variables file