#helmfile - July 2020

helmfile
41 messages

Questions and discussion around helmfile https://github.com/roboll/helmfile and https://github.com/cloudposse/helmfiles
Archive: https://archive.sweetops.com/helmfile/

Jun 2020
July 2020
Aug 2020
opollie
opollieover 6 years ago
hi all,
i am pretty much new to helmfile and currently trying to migrate a huge helm2 umbrella chart to helmfile...
one of my problems is for some reason the needs: concept doesn't work in the way it's written in the doc (or at least i got it wrong):
when i do sync this:
- name: loggingMaster
  chart: elastic/elasticsearch
  version: 7.3.0
  condition: loggingMaster.enabled
  <<: *default
- name: loggingData
  chart: elastic/elasticsearch
  version: 7.3.0
  condition: loggingData.enabled
  <<: *default
- name: misc-es
  chart: ./charts/misc-es
  version: 0.1.0
  verify: false
  condition: loggingMaster.enabled
  needs:
  - loggingMaster

i get the following error:
in ./helmfile.yaml: "clusterinfra/misc-es" depends on nonexistent release "loggingMaster"

any idea why?
soumya
soumyaover 6 years ago
Is there a way I can prevent creation of new config maps every time I deploy through helm.
sean
seanover 6 years ago
Hi guys i'm having trouble getting readFile working with requiredEnv templating.

I currently have dev.yaml

environments:
  dev:
    values:
      - HelmRepo: custom-staging
{{ readFile "./helmfile-cs-base.yaml" }}

and helmfile-cs-base.yaml
<snip>

repositories:
  - name: custom
    url: <https://chartmuseum.internal.ourdomain.com/master>
    certFile: {{ requiredEnv CHARTMUSEUM_CERT_FILE }}
    keyFile: {{ requiredEnv CHARTMUSEUM_KEY_FILE }}
  - name: custom-staging
    url: <https://chartmuseum.internal.ourdomain.com/staging>
    certFile: {{ requiredEnv CHARTMUSEUM_CERT_FILE }}
    keyFile: {{ requiredEnv CHARTMUSEUM_KEY_FILE }}
    
<snip>

And when running it i get

err: failed to read helmfile-cs.yaml: reading document at index 1: yaml: unmarshal errors:
  line 32: cannot unmarshal !!map into string
  line 33: cannot unmarshal !!map into string
  line 36: cannot unmarshal !!map into string
  line 37: cannot unmarshal !!map into string
in ./helmfile-cs.yaml: failed to read helmfile-cs.yaml: reading document at index 1: yaml: unmarshal errors:
  line 32: cannot unmarshal !!map into string
  line 33: cannot unmarshal !!map into string
  line 36: cannot unmarshal !!map into string
  line 37: cannot unmarshal !!map into string

which corresponds to the repositories chunk.

im running helmfile version v0.119.1

Has anyone got any ideas?
jason witkowski
jason witkowskiover 6 years ago(edited)
Hey All, I'm having a really confusing problem that I think someone better than I can maybe easily pinpoint. I have a straight-forward helm file. I pull in a values file to the environment and try to use a top level variable and I immediately get an error.
helmfile:
environments:
  preprod:
    values:
      - vars/helmfile/realms/preprod.yaml

releases:
{{- $realm := .Values.realm -}}
...

values-file:
$ cat vars/helmfile/realms/preprod.yaml
realm: preprod

Error:
$ helmfile -e preprod -f helmfile-node-local-dns.yaml lint
in ./helmfile-node-local-dns.yaml: error during helmfile-node-local-dns.yaml.part.0 parsing: template: stringTemplate:18:21: executing "stringTemplate" at <.Values.realm>: map has no entry for key "realm"
jason witkowski
jason witkowskiover 6 years ago
Can anyone see what I'm doing wrong?
sean
seanover 6 years ago
Hi guys. I'm trying to figure out how to override the environment values in a helmfile from the CLI and i'm not sure how to go about it, since I think the cli flag provided values --state-values-set ImageTag=latest-master are being overriden by the environment values specified in my helmfile.

i'm running helmfile -f dev.yaml --debug -e flaky-smalt-cat-aujzyfqd0cc --interactive --state-values-set ImageTag=latest-master apply --context=3

Is there anyway to override what ImageTag is in dev.yaml from the CLI?
dev.yaml

environments:
  flaky-smalt-cat-aujzyfqd0cc:
    values:
    - HelmRepo: company-staging
    - Namespace: flaky-smalt-cat-aujzyfqd0cc
    - ImageTag: latest-staging
    - CustomerTag: flaky-smalt-cat-aujzyfqd0cc
    - Monolith: true
    - Domain: company.dev

bases:
  - ./namespace.yaml

namespace.yaml
sean
seanover 6 years ago
environments:
  default:
    values:
      - HelmRepo: company
      - Namespace: default
      - ImageTag: latest
      - CustomerTag: default-customer-tag

templates:

  chartnamespace: &chartnamespace
    namespace: "{{`{{ .Environment.Values.Namespace }}`}}"
    # missingFileHandler: Warn

  set: &set
    setTemplate:
      - name: image_tag
        value: "{{`{{ .Environment.Values.ImageTag }}`}}"
      - name: customerTag
        value: "{{`{{ .Environment.Values.CustomerTag }}`}}"
      - name: namespace
        value: "{{`{{ .Environment.Values.Namespace }}`}}"
      - name: baseHost
        value: "{{`.{{ .Environment.Values | get \"Domain\" \"<http://companycloud.com|companycloud.com>\" }}`}}"

  cs-ui-default: &cs-ui-default
    <<: *chartnamespace
    setTemplate:
      - name: image_tag
        value: "{{`{{ .Environment.Values.ImageTag }}`}}"
      - name: customerTag
        value: "{{`{{ .Environment.Values.CustomerTag }}`}}"
      - name: namespace
        value: "{{`{{ .Environment.Values.Namespace }}`}}"
      - name: config.server_name
        value: "{{`{{ .Environment.Values.CustomerTag }}.{{ .Environment.Values | get \"Domain\" \"<http://companycloud.com|companycloud.com>\" }}`}}"
      - name: baseHost
        value: "{{`.{{ .Environment.Values | get \"Domain\" \"<http://companycloud.com|companycloud.com>\" }}`}}"

  default: &default
    <<: *set
    <<: *chartnamespace

  cs-engine: &cs-engine
    name: cs-engine{{`{{ if ne .Environment.Values.Namespace "default" }}-{{ .Environment.Values.Namespace }}{{ end }}`}}
    chart: "{{`{{ .Environment.Values.HelmRepo }}`}}/cs-engine"
    <<: *chartnamespace
    setTemplate:
      - name: image_tag
        value: "{{`{{ .Environment.Values.ImageTag }}`}}"
      - name: customerTag
        value: "{{`{{ .Environment.Values.CustomerTag }}`}}"
      - name: namespace
        value: "{{`{{ .Environment.Values.Namespace }}`}}"
      - name: baseHost
        value: "{{`{{ .Environment.Values.CustomerTag }}.{{ .Environment.Values | get \"Domain\" \"<http://companycloud.com|companycloud.com>\" }}`}}"
      - name: config.server_name
        value: "{{`{{ .Environment.Values.CustomerTag }}.{{ .Environment.Values | get \"Domain\" \"<http://companycloud.com|companycloud.com>\" }}`}}"
      - name: monolith.enabled
        value: "{{`{{ .Environment.Values | get \"Monolith\" \"false\" }}`}}"
      - name: baseHost
        value: "{{`.{{ .Environment.Values | get \"Domain\" \"<http://companycloud.com|companycloud.com>\" }}`}}"

  cs-ui: &cs-ui
    name: cs-ui{{`{{ if ne .Environment.Values.Namespace "default" }}-{{  .Environment.Values.Namespace }}{{ end }}`}}
    chart: "{{`{{ .Environment.Values.HelmRepo }}`}}/cs-ui"
    <<: *cs-ui-default
  cs-api: &cs-api
    name: cs-api{{`{{ if ne .Environment.Values.Namespace "default" }}-{{  .Environment.Values.Namespace }}{{ end }}`}}
    chart: "{{`{{ .Environment.Values.HelmRepo }}`}}/cs-api"
    <<: *default
  cs-database: &cs-database
    name: cs-database{{`{{ if ne .Environment.Values.Namespace "default" }}-{{  .Environment.Values.Namespace }}{{ end }}`}}
    chart: "{{`{{ .Environment.Values.HelmRepo }}`}}/cs-database"
    <<: *default

repositories:
  - name: company
    url: <https://chartmuseum.companycloud.com/master>
    certFile: {{ requiredEnv "CHARTMUSEUM_CERT_FILE"  }}
    keyFile: {{ requiredEnv "CHARTMUSEUM_KEY_FILE" }}
  - name: company-staging
    url: <https://chartmuseum.companycloud.com/staging>
    certFile: {{ requiredEnv "CHARTMUSEUM_CERT_FILE" }}
    keyFile: {{ requiredEnv "CHARTMUSEUM_KEY_FILE" }}

helmDefaults:
  timeout: 600
  recreatePods: true
  atomic: true
  force: true

releases:
  - <<: *cs-engine
  - <<: *cs-ui
  - <<: *cs-api
  - <<: *cs-database
muhaha
muhahaover 6 years ago(edited)
guys? how are you installing cert-manager? this https://github.com/cloudposse/helmfiles/blob/master/releases/cert-manager.yaml will not remove CRDs in helmfile delete phase.
bradym
bradymover 6 years ago(edited)
I have a repo for which I need to deploy multiple releases: one is the main app, the other is a worker. I want them as separate releases as the worker should be able to scale independently of the main app. I have this part working great with helmfile. The worker listens to an SQS queue and processes the messages as they come in.

In stage I create an environment every time a new branch is deployed, and tear it down when that branch is merged. I'm trying to figure out the best way to specify when the worker should be deployed. Ideally we'd control that via an aws ssm parameter as that's how we manage our env vars. I was hoping I could do something like:

releases:
  - name: worker
    installed: <secretref+awsssm://V1/{{> .Values.repo }}/{{ .Environment.Name }}/DEPLOY_WORKER?region=us-west-1

But that doesn't work. Any ideas on how I could handle this? I'd like to avoid having a separate SQS queue per branch to keep down the number of things that have to be created/torn down.
Jonathan
Jonathanover 6 years ago
Been trying to access secrets from GCP Secret manager the following way in my values file:
password: <ref+gcpsecrets://project/secret-name?version=1>

And using a service account, which I can access the secret manually.
Despite this, I get an error regarding accessing the secret:
failed to render values files "config/values.yaml": expand <gcpsecrets://project/secret-name?version=1>: failed to get secret: rpc error: code = PermissionDenied desc = Request had insufficient authentication scopes.

It seems the service account permissions are bad, but they work when accessing manually:
bash-4.3# gcloud auth activate-service-account --key-file=keyfile.json
Activated service account credentials for: [<SERVICE_ACCOUNT>]
bash-4.3# gcloud secrets versions access 1 --secret="secret-name"

Does anyone have any pointers?
muhaha
muhahaover 6 years ago
How can I override environment values in helmfile? For example:
environments:
  default:
    missingFileHandler: Warn
    values:
       - external_dns:
           identity:
             tenant_id: {{ requiredEnv "EXTERNAL_DNS_IDENTITY_TENANT_ID" }}
             subscription_id: {{ requiredEnv "EXTERNAL_DNS_IDENTITY_SUBSCRIPTION_ID" }}
             resource_group: {{ requiredEnv "EXTERNAL_DNS_IDENTITY_RESOURCE_GROUP" }}
             name: {{ requiredEnv "EXTERNAL_DNS_IDENTITY_NAME" }}
             id: {{ requiredEnv "EXTERNAL_DNS_IDENTITY_ID" }}
       - values.external-dns.yaml

It should fail if values.external-dns.yaml is not present/populated && env vars are not supplied.
Craig Dunford
Craig Dunfordover 6 years ago
We are looking at storing our helm charts in an Azure ACR. Per https://docs.microsoft.com/en-us/azure/container-registry/container-registry-helm-repos the recommendation is to use helm's experimental OCI support to publish the charts to the registry. Does anyone know if helmfile supports pulling charts from an OCI repository via helm's experimental support?
vgdubrea
vgdubreaover 6 years ago
Hi Guys , I have a short question , any suggestions would be really helpful .
---
bases:
  - helmfile-first.yaml
  - helmfile-second.yaml

I would like to conditionally execute the helmfile-second.yaml without using if , else . is there an easy way I can do it ?
vgdubrea
vgdubreaover 6 years ago
my end goal is to avoid any releases mentioned in second Helmfile to be deployed.
muhaha
muhahaover 6 years ago
Guys? How should I version dynamic variables used in k8s deployments created with terraform ? Should I back commit these into git and then use helmfile with these values ? ( these are not secrets, just dynamic vars, not predictable )
Abhishek
Abhishekover 6 years ago
Hi everyone ! I have question, In case of nested helmfiles, do children helmfile inherit environment variables from parent helmfile. I'm not able to get the values in child helmfile. Below is my master helmfile and I'm trying to access values from imported-values.yaml but it's getting values only from folder/values.yaml
environments:
  default:
    values:
      - "imported-values.yaml"

helmfiles:
  - path: folder/helmfile.yaml
    values:
      - "folder/values.yaml"
Marcin Brański
Marcin Brańskiover 6 years ago
I’m kinda new to helmfile and haven’t yet figured everything out.
I’d like to use environment variable inside default function
tags: '{{ env "KUBE_PROMETHEUS_ALERT_MANAGER_OPSGENIE_TAGS" | default ` {{ range .CommonLabels.SortedPairs }}{{ .Name }}:{{ .Value }},{{ end }}` {{ env "STAGE" | default "N/A" }} }}'

For such syntax ☝️ I get:
template: stringTemplate:306: unexpected "{" in operand

The tricky part is that I want to concatenate string {{ range .CommonLabels.SortedPairs }}{{ .Name }}:{{ .Value }},{{ end }} to another tag which is in environment variable. How to do that?
Mr.OO
Mr.OOover 6 years ago
Hi,
I get the following error:
STDERR:                                                                                                              
  Error: no cached repo found. (try 'helm repo update'). error converting YAML to JSON: json: unsupported value: +Inf
  Error: plugin "diff" exited with error                                                                             
                                                                                                                     
COMBINED OUTPUT:                                                                                                     
  [debug] Created tunnel using local port: '44995'                                                                   
  [debug] SERVER: "127.0.0.1:44995"                                                                                  
  Error: no cached repo found. (try 'helm repo update'). error converting YAML to JSON: json: unsupported value: +Inf
  Error: plugin "diff" exited with error

if I do a "normal" helm diff upgrade it works with complaining.
helmfile template also work fine.
but I don't get helmfile diff to work

Has anyone stumbled into a similar problem?
I found this link, but it doesn't give that much information: https://github.com/helm/helm/issues/2909
joshmyers
joshmyersover 6 years ago
@Erik Osterman (Cloud Posse) (and others here) how are ya’ll doing Helmfile deployments via CI/CD, promoting per environment with per environment config? AFAIKR CP bundled a version of their helmfiles image with a per env container and slurped config values out of SSM with chamber, but doesn’t look to be the case anymore?
vgdubrea
vgdubreaover 6 years ago(edited)
Hi folks, One more question it seems that I am facing issues using {{ .Environment.Values.<> }} in my releases: inside helmfile.yaml.
my structure is like this
vgdubrea
vgdubreaover 6 years ago
vgdubrea
vgdubreaover 6 years ago
to give more context we do Helmfile apply -f helmfile.yaml inside Helmfile.yaml we are calling Helmfile-common.yaml , this common file has all the environments and their values file defined.
vgdubrea
vgdubreaover 6 years ago
Now I have some variable defined inside those environments that I need to use inside Helmfile but it seems {{ .Environment.Values.<> }} can't be used in Helmfile releases , any option anyone can suggest ?
muhaha
muhahaover 6 years ago(edited)
More love for delete hooks support https://github.com/roboll/helmfile/issues/802#issuecomment-659938624 ? ( crd post delete )
Paul Catinean
Paul Catineanover 6 years ago
Is anyone here using terraform with helmfile ?
Paul Catinean
Paul Catineanover 6 years ago
I can't figure out how to send environment variables to existing helmfile files
Chad Ostler
Chad Ostlerover 6 years ago
helmfile sync --concurrency <x> seems to only run one at a time no matter what concurrency i set. is this a known issue?
Chad Ostler
Chad Ostlerover 6 years ago
i’m wondering if it’s due to the fact i have an umbrella helmfile of many nested helmfiles that we use labels to deploy selectively.
Andreas Lindhé
Andreas Lindhéover 6 years ago
Ohoy! I've just started using helmfile, and I'm a bit lost about secrets management.

Is it mandatory to have the helm-secrets plugin installed, even if my secrets are not encrypted? The secret I'm trying to include is not very secret, so I have no reason to add the complexity that comes with SOPS...
Oleksii Chupryn
Oleksii Chuprynover 6 years ago(edited)
Hello! Is it possible to skip diff stage as part of apply command? Use case: override changes which were made manually (helm-diff doesn't show any diff in this case). So, i'm looking for the way of force execute helm upgrade --install . Thanks
Mathieu Frenette
Mathieu Frenetteover 6 years ago
Hey! 👋 I was wondering if there was a way to preview the template rendering of helmfile before actually applying? We often have templating issues that are hard to troubleshoot if we have no way to see what manifests were actually rendered by helmfile.
bradym
bradymover 6 years ago
helmfile template does that
Andreas Lindhé
Andreas Lindhéover 6 years ago
Can I include multiple values.yaml files in the values: section and they will be sequentially applied?
Andreas Lindhé
Andreas Lindhéover 6 years ago(edited)
Is the needs: directive broken? It complains to be that "default/y" depends on nonexistent release "x" while I definitely have a release x in my helmfile...
Dudi Cohen
Dudi Cohenover 6 years ago
Did anyone figure out a way to run helmfile template only on specific charts/releases?
Michal Matyjek
Michal Matyjekover 6 years ago
could not find a good answer: can I override name of the chart if syncing from helmfile.yaml?

something like helmfile -f charts/helmfile.yaml --name=notTheOneInHelmfile.yaml ?
Andreas Lindhé
Andreas Lindhéover 6 years ago(edited)
Is it possible to have helmfile label namespaces for me, or can it only put labels on Helm releases?
Graeme Gillies
Graeme Gilliesover 6 years ago
How would one go about creating hierarchical environments in helmfile? Lots of use of yaml anchors?
Sam Buckingham
Sam Buckinghamover 6 years ago
Has anyone else had issues with repositories not being updated? helmfile seems to be completely ignoring the repositories block in my yaml?
jason witkowski
jason witkowskiover 6 years ago
Hey All, I'm having some really weird issues about context deadline exceeded from helmfile reaching my clusters. The issue does not happen if i manually query the cluster or use helm directly. Has anyone else ever seen this ?
jason witkowski
jason witkowskiover 6 years ago(edited)
Another thing is we just went back and did a huge refactor across our repo to make massive use of templating since we deploy to many clusters across many regions and such. The performance of linting has degraded to point of not being able to run it in any meaningful way
June 2020Browse by dateAugust 2020