#atmos - October 2023

anybody already got some renovatebot config to perform component updates?

and is there a way to suppress the output of the init (not skipping the init), eg when doing a atmos terraform show -json comp -s stack ?

Is there some built-in way to automatically wrap the atmos command with ATMOS_BASE_PATH ATMOS_CLI_CONFIG_PATH and setting it to PWD in order to use remote module states when working in the current dir, without using geodesic or moving the atmos file? I’m thinking makefile probably would be able to do this, but just wondering if there’s any other solutions

Are there any minimal examples of an atmos driven repo? As awesome as atmos/examples/complete is it's a lot when first approaching something new. What would be the absolute minimal repo which deploys a single component in a single stack?

Hey guys, I'm following atmos tutorial here and currently stuck on atmos vendor pull --component infra/vpc, keep getting:

No stack config files found in the provided paths:
- /<omitted>/atmos-poc/stacks/orgs/**/*


Check if `base_path`, 'stacks.base_path', 'stacks.included_paths' and 'stacks.excluded_paths' are correctly set in CLI config files or ENV vars.

There is a lot of information related to the usage and way of thinking when building with atmos. The components also provide a nice way to better understand the opinionated concepts related to separating and deploying/destroying parts that relate to eachother. However, one important aspect I have failed to discover yet, is how to handle EKS service roles for the application services you are building. These service roles will all have some kind of dependence on other components of the infrastructure. Whats the best practise setting up something like that? Do you just write something custom for it and reference the arns from the remote state?

I think atmos vendor pull should not default to atmos vendor pull [STACK] when it will always give the error command 'atmos vendor pull --stack <stack>' is not implemented yet

it should insted default to atmos vendor pull [COMPONENT]

since inherits is an array and doesn't merge when overridden, is there a solution where we could inherit without having to copy all other inherits from the main file?

ex:
• dev: retention_in_days: 3
• prod: retention_in_days: 120
we would rather not have 120 sprayed across ~68 files so would like to use inherits. doing so means updating the inheritance in dev and prod each time it changes (surely someone will forget and break prod).

just curious of any solutions.

what is this used for?

settings:
  config:
    is_prod: false

Seems Terraform is implementing stacks... US and foreign patents pending even! https://www.hashicorp.com/blog/new-terraform-testing-and-ux-features-reduce-toil-errors-and-costs

I'm about to do a bit of a spike on refactoring from a monolithic, landing zone terraform root module & state file to atmos... any tools or techniques to help w/ this migration... any tips? seems a bit daunting

v1.46.0
what

Update to Go 1.21
Update/improve Atmos help for atmos-specific commands
Improve atmos vendor pull command error handling
Add Atmos custom commands to atmos.yaml and docs
Add examples of Rego policies for Atmos components and stacks validation. Update docs. See https://atmos.tools/core-concepts/components/validation/
Add Sprig functions support to Go templates in Atmos vendoring
Improve context...

v1.46.0
what

Update to Go 1.21
Update/improve Atmos help for atmos-specific commands
Improve atmos vendor pull command error handling
Add Atmos custom commands to atmos.yaml and docs
Add examples of Rego policies for Atmos components and stacks validation. Update docs. See https://atmos.tools/core-concepts/components/validation/
Add Sprig functions support to Go templates in Atmos vendoring
Improve context...

It'd be nice to have developer tooling for atmos like a yaml spec in VScode for stack files

If I want to make a separate environment in a different continent/account, what would the best property of the label component be to change to differentiate between the 2? Having a hard time choosing as it impacts other parts of the stack too. Ideally I think I would be tenant but then I also would need to have tenant in the name_pattern config for atmos. But this would break all of the previous stacks. For example, if I want to use something like aws-team-roles, if both continents have production environments they would both be in stack acme-gbl-prod ({namespace}-{environment}-{stage}) ,which would not work. So then naming them differently would be the only option.

Are there downsides to directly modifying the workspace_key_prefix to force the S3 backend structure to match a non-standard component hierarchy?

Example:

# local directory structure
components/
  terraform/
     account/
     account-map/
     vpc/
     bespoke/
       component-a/
       component-b/
 
# auto-generated workspaces+prefixes in S3
account/core-ue1-root/terraform.tfstate
account-map/core-ue1-root/terraform.tfstate
...
bespoke-component-a/plat-ue2-devt/terraform.tfstate
bespoke-component-a/plat-ue2-prod/terraform.tfstate
...

# post hacking the workspace prefix to "bespoke/<component-name>"
account/core-ue1-root/terraform.tfstate
account-map/core-ue1-root/terraform.tfstate
...
bespoke/component-a/plat-ue2-devt/terraform.tfstate
bespoke/component-a/plat-ue2-prod/terraform.tfstate
...

I think the former approach could lead to a very flat S3 backend with potentially hundreds of top-level directories, with local sub-folders under components only distinguishable by prefix.

The second approach requires modifying the catalog of each new component to override the autogenerated workspace prefix that collapses all / characters to - , but would lead to a backend that:
a) mirrors the local component directory structure
b) groups all backend state files by their directory, whether this be a project, a team, service or “kind”

I’ve tested this approach in vanilla terraform, as there are some old unresolved github issues where people attempting this unsuccessfully, (resorting to hacking the key property), but it does appear to work in modern Terraform

Will this cause any problems with Atmos under the hood?

Is there a way to unset a deep merged property in atmos? I need certain tags on all my resources so somewhere in the hierarchy, its included in the defaults. But for kubernetes resources these tags are not needed and/or error. Setting these to null gives me another error. The only other solution I see would be to be more selective in importing these default tags, but this means more imports, so rather want to “exclude” rather than “include”.

For the atmos workflow files, it would be a nice addition to be able to tag the upper level steps, and be able to run all these steps specifying that tag (or skip), eg:

workflows:
  main-step-1:
    tags: [ slow ]
    steps: [ ... ]

  main-step-2:
    tags: [ fast ]
    steps: [ ... ]

  main-step-3:
    tags: [ slow ]
    steps: [ ... ]

atmos workflow --use-tags fast -f dummy or atmos workflow --skip-tags slow -f dummy
(bit like ansible playbooks)

I remember when running a terraform plan/apply, atmos would show the deep merge of the yaml stack component at the very top. I no longer see it and cannot find a way to turn it back on.

For atmos and subcommand, I see i can define a flag of type bool. How can I access this flag in the template, eg {{ .Flags.<flagname> }} throws a

template: step-0:5:14: executing "step-0" at <.Flags.xroot>: map has no entry for key "xroot"

For atmos and subcommand, when a flag is like require-root , how can I access this flag in the template?

Quick question, has atmos ever had a 3rd party security audit? asking for a client that has high security requirements

v1.47.0
what

Add vendor.yaml as a new way of vendoring components, stacks and other artifacts
Add/update docs

https://atmos.tools/core-concepts/vendoring/
https://atmos.tools/cli/commands/vendor/pull/



why
Atmos natively supports the concept of "vendoring", which is making copies of the 3rd party components, stacks, and other artifacts in your own...

v1.47.0
what

Add vendor.yaml as a new way of vendoring components, stacks and other artifacts
Add/update docs

https://atmos.tools/core-concepts/vendoring/
https://atmos.tools/cli/commands/vendor/pull/



why
Atmos natively supports the concept of "vendoring", which is making copies of the 3rd party components, stacks, and other artifacts in your own...

Hi there! New user of Atmos here. Just started experimenting yesterday and so far I like the promise of the framework/tool. I created a few simple stacks to experiment and am stuck with a terraform state related issue. I initially followed the guidance here (https://docs.cloudposse.com/components/library/aws/account/) to create a set of AWS accounts in a brand new AWS Master account (no other resource created). I am now trying to move the local state to S3 and am getting the following error:

Error: Error loading state error
│ 
│   with module.assume_role["hct-gbl-root-tfstate-purple-cow"].module.allowed_role_map.module.account_map.data.terraform_remote_state.data_source[0],
│   on .terraform/modules/assume_role.allowed_role_map.account_map/modules/remote-state/data-source.tf line 91, in data "terraform_remote_state" "data_source":
│   91:   backend   = local.ds_backend
│ 
│ error loading the remote state: S3 bucket does not exist.
│ 
│ The referenced S3 bucket must have been previously created. If the S3 bucket
│ was created within the last minute, please wait for a minute or two and try
│ again.
│ 
│ Error: NoSuchBucket: The specified bucket does not exist
│       status code: 404, request id: W7M341334K7N0DVR, host id: mWPCc2QiX7Aw3ar+uB1wravpTivYZdCnVqVPoNaQXe7NNyOdKGyHbNcR5hJgZ0R5MoYY5TvwN+M=

My understanding is that the S3 bucket gets created as part of the execution of this TF script. What am I missing?

Command executed:
atmos terraform plan tfstate-backend --stack core-ue1-root

Any help will be greatly appreciated!

I am getting the following error while provisioning a simple "vpc-flow-logs-bucket" component as outlined in the example.

Error: Invalid index
│ 
│   on ../account-map/modules/iam-roles/main.tf line 38, in locals:
│   38:     local.account_map.full_account_map[local.account_name]
│     ├────────────────
│     │ local.account_map.full_account_map is object with 12 attributes
│     │ local.account_name is "dev"
│ 
│ The given key does not identify an element in this collection value.

It looks like the iam-roles module (in account_map_ is looking for accounts using just the stage attribute where as the accounts created follow the format "tenant-stage". The list of accounts from account_map module has an account called "plat-dev", but the tenant name is not being included in the look up above. I did some more digging and it appears others had similar issue and they resolved it by passing an updated "description_format" variable to the stack.

  descriptor_formats:
    account_name:
      format: "%v-%v"
      labels:
      - tenant
      - stage
    stack:
      format: "%v-%v-%v"
      labels:
      - tenant
      - environment
      - stage

I did include the above block in my stack def - but nothing seems to be working. Did anyone else face a similar issue and if you did, what was the solution?