#helmfile - November 2019

November 2019
mumoshu
mumoshuabout 7 years ago
I'd like to deprecate "double-rendering" in favor of this feature: https://github.com/roboll/helmfile/issues/932

WDYT?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
How reddit uses Helmfile (2018). https://youtu.be/7Qxuo9W5SlY
Bart M.
Bart M.almost 7 years ago
hmm getOrNil can only return as string?
Paul Becotte
Paul Becottealmost 7 years ago
Another question- how does values templating work in helmfile? Like, I can't use .Values from a helmfile in .gotmpl files...but I can use values passed in from higher level helmfiles using the 'helmfiles' key. Butniflt feels like there is a way I'm missing to not need multiple layers. The kinds of things I want to do are declare a value in one place, and then use that value later in to build up other values.
Gourav
Gouravalmost 7 years ago
Hello All.. Is there any document/image which depicts the kiam, reloader and cert-manager? Which tries to show how they are interrelated. If anyone is having such link, can you please share with me?
Frank
Frankalmost 7 years ago(edited)
Hi all!
I have some helm charts that have to be installed in the right order, e.g.
1) install helm chart 1 which created some CRDs and so on
2) wait some time until helm chart 1 is settled
3) install helm chart 2
Frank
Frankalmost 7 years ago
I tried to do this with wait(on helmfile and release level) and atomic , but both seem not to work stable in my case.
(By the way I use helm3 , latest release, that might be important 😉 )
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
Wow Helmfile getting spammed in issues! Hope this is an isolated incident. GitHub has been amazingly spam free.
Andrew Nazarov
Andrew Nazarovalmost 7 years ago
Is it possible to prevent a release from updating when doing helmfile apply? I don’t see any flag allowing me to do so. The use case: some charts have some random values which are recalculated every time we run helm diff. So there are always some diffs. Hence this leads to unnecessary upgrades. This ignore is not the best solution for sure. Like it will also ignore all “positive changes” that should be syncronized. But at the moment I don’t see anything else that could have been done to solve this. Any thoughts on this?
Andrew Nazarov
Andrew Nazarovalmost 7 years ago
No, it was a bad idea. It’s better to upgrade a release every time.
Paul Becotte
Paul Becottealmost 7 years ago
What are the random numbers for? I generate secrets- but i use hooks so that I only generate them if they don't already exist. If you're generating random values in manifests that you apply, then you are certainly changing stuff.
Andrew Nazarov
Andrew Nazarovalmost 7 years ago(edited)
Usually this comes from certs, like tls.crt, tls.key, caBundle, etc fields. Artifactory and Stash charts are know for doing that. Also I’ve seen something similar from Elasticsearch, but not anymore. Cannot find any examples with generated strings and random numbers right now, but they definitely existed before).
Paul Becotte
Paul Becottealmost 7 years ago
Any way to run helmfile hooks without helmfile sync?
Paul Becotte
Paul Becottealmost 7 years ago
(am investigating using our helmfile setup with our argocd deployment...but I made extensive use of helmfile hooks)
mumoshu
mumoshualmost 7 years ago
@Paul Becotte Unfortunately it isn't possible today. But a feature request is welcomed.
If you could also include your specific use-cases for hooks(fetching charts and modifying it, creating namespaces by calling kubectl, etc) that would be great
Paul Becotte
Paul Becottealmost 7 years ago
Sure- helmfile is golang?
mumoshu
mumoshualmost 7 years ago
Yep
Paul Becotte
Paul Becottealmost 7 years ago
Probably not gonna be a PR then :)
Paul Becotte
Paul Becottealmost 7 years ago
I'll put in an issue though
mumoshu
mumoshualmost 7 years ago
Thanks!
Thomas Burton
Thomas Burtonalmost 7 years ago
Afternoon. Question:
Thomas Burton
Thomas Burtonalmost 7 years ago
I am running into an error with installing any charts through helmfile.
I get the following error:
Thomas Burton
Thomas Burtonalmost 7 years ago
Thomas Burton
Thomas Burtonalmost 7 years ago
This is my helmfile
Thomas Burton
Thomas Burtonalmost 7 years ago
Thomas Burton
Thomas Burtonalmost 7 years ago
any help would be greatly appreciated
Thomas Burton
Thomas Burtonalmost 7 years ago
managed to fix by running helmfile destroy then helmfile apply
mumoshu
mumoshualmost 7 years ago
I periodically see folks being trapped by the fact that atomic isnt enabled by default.
Hopefully we are able to cut v1 and start turning it on by default since then 🙂
https://github.com/roboll/helmfile/issues/776
mumoshu
mumoshualmost 7 years ago
@Erik Osterman (Cloud Posse) I'd appreciate your comment here! https://github.com/roboll/helmfile/issues/932#issuecomment-553188582
Thomas Burton
Thomas Burtonalmost 7 years ago
Anyone able to help here

After running helmfile --environment staging apply successfully I go to check my releases and they're not listed even though part of the logs suggests otherwise.
Thomas Burton
Thomas Burtonalmost 7 years ago
This is the output from the logs
Thomas Burton
Thomas Burtonalmost 7 years ago
I am running a tiller deployment in the staging namespace
Thomas Burton
Thomas Burtonalmost 7 years ago
However, if I run helm list --tiller-namespace staging it returns absolutely nothing
Thomas Burton
Thomas Burtonalmost 7 years ago
Another weird thing is that the pods start correctly
Thomas Burton
Thomas Burtonalmost 7 years ago
Andrew Nazarov
Andrew Nazarovalmost 7 years ago(edited)
What’s in helmDefaults.tillerNamespace in your helmfile.yaml?
Thomas Burton
Thomas Burtonalmost 7 years ago
@Andrew Nazarov it was originally default. Changed it to {{ .Environment.Name }} but still no luck.
Andrew Nazarov
Andrew Nazarovalmost 7 years ago
oh, you are using tillerless plugin)
Andrew Nazarov
Andrew Nazarovalmost 7 years ago
Then try helm tiller run staging -- helm ls
Thomas Burton
Thomas Burtonalmost 7 years ago
A-HA!
Thomas Burton
Thomas Burtonalmost 7 years ago
Thanks v much
Thomas Burton
Thomas Burtonalmost 7 years ago
Worked
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
3.0 official!!l?
yuri
yurialmost 7 years ago
seems official, whats the support status of helmfile with helm v3? sorry was not following lately
Paul Becotte
Paul Becottealmost 7 years ago
I've been using v3 for the last month
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
With Helmfile?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
I know there have been a few PRs to add support
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
Haven’t tested it yet
Paul Becotte
Paul Becottealmost 7 years ago
Yeah with helmfile
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
@Daren @chris ☝️ FYI
chris
chrisalmost 7 years ago
@chris has joined the channel
Paul Becotte
Paul Becottealmost 7 years ago
Something I just thought of...the first time I had to set an env variables...butbinhavent been doing that
Shikhar Goel
Shikhar Goelalmost 7 years ago
Hi...i am not able to run helmfile using helm 3...i am getting the following error.....` Error: context deadline exceeded
Error: plugin "diff" exited with error`
Shikhar Goel
Shikhar Goelalmost 7 years ago(edited)
Hi.......I am seeing this issue. I have also referred this 'https://github.com/roboll/helmfile/issues/863' but it didn't work. I took pull from master today only.`Upgrading ../../helm-installer/rook-operator
in ./helmfile.yaml: in .helmfiles[0]: in environments/singlenode/00-helmfile.yaml: failed processing release rook-operator: helm exited with status 1:
Error: invalid argument "300" for "--timeout" flag: time: missing unit in duration 300`
Shikhar Goel
Shikhar Goelalmost 7 years ago
Hi...--concurrency=1 is not working in helm3...
Ryan Smith
Ryan Smithalmost 7 years ago
https://github.com/helm/helm/issues/3701 Does anyone hit this still? For some reason updating (adding) environment variables doesn't get properly applied to the PodTemplateSpec of the deployment. 🤷‍♂️
TBeijen
TBeijenalmost 7 years ago
Seem to run into a strange situation where failing to decrypt sops secrets (because not having KMS permissions) fails silently, resulting in an error due to missing variable, hiding the actual problem. In other projects error clearly states sops failing. So... probably overlooking some silly thing. Does this ring any bells?
jedineeper
jedineeperalmost 7 years ago
are there any special changes to helmfile for the helmv3 upgrade? my helm installation seems to be fine but the latest version of helmfile isn't happy? Is there a change in syntax?
mumoshu
mumoshualmost 7 years ago
it's still rough around the edges but...

https://github.com/mumoshu/terraform-provider-helmfile
mumoshu
mumoshualmost 7 years ago
just realized that there was another way to call helmfile from tf:

https://kubedex.com/using-helmfile-with-terraform/
https://github.com/Kubedex/terraform-helmfile

it's interesting as well 🙂
mumoshu
mumoshualmost 7 years ago
one advantage of my terraform-helmfile-provider would be that it exploits terraform plan output to show diff from helmfile diff
mumoshu
mumoshualmost 7 years ago
and you can even use terraform output to view helmfile apply results
Rhooker
Rhookeralmost 7 years ago
@Rhooker has joined the channel
dustinvb
dustinvbalmost 7 years ago
This is still rough but I am setting up a demo for it as we speak...
https://github.com/codefresh-contrib/cfstep-helmfile
If there is interest in putting CI in Codefresh for the project that might be something that would help keep the Dockerfile contents updated.
Also, is this ready for Helm 3.0 can I simply supply the newest helm binary and expect this to work?
Roderik van der Veer
Roderik van der Veeralmost 7 years ago
I’ve got a quick question. I have a nice big Helmfile deploying an application stack. It is completely deployed within a namespace to separate it from other things running on the cluster. I now want to install a complete second instance of the stack, in a different namespace but on the same cluster. I imagined that because it is namespaced, it would just work, except it modified my original deploy. What would be the best way to enable such a usecase? Prefixing the release name with the namespace? (helm 2, latest helmfile, GKE)
Roderik van der Veer
Roderik van der Veeralmost 7 years ago
I’m struggling a bit with --namespace and namespace definition in releases. I run helmfile with --namespace namespaceA. All my releases have namespace: {{ .Namespace }} and one (ingress controller) has namespace: something in the release. But it gets deloyed to namespaceA . Anyone have an idea on how to do this (apart from running helmfile twice)?
Josh McDade
Josh McDadealmost 7 years ago
Hi, I'm having issues running helmfile -n my-ns apply from v0.93.1 (I have set HELMFILE_HELM3=1 for my helm3 releases). Every release errors:

helm exited with status 1:
  Error: context deadline exceeded
  Error: plugin "diff" exited with error

The version of diff plugin is the same as the one in the Dockerfile.helm3 diff 3.0.0-rc.7
mumoshu
mumoshualmost 7 years ago
@Erik Osterman (Cloud Posse) btw, are you still using atlantis? just wanted to ask if it can be a good idea to use atlantis for deploying k8s apps, as we now have the tf provider for helmfile

https://github.com/roboll/helmfile/issues/386
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
Yes, we are still using atlantis (and starting to poc terraform cloud)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
which is why all this is extra exciting.
mumoshu
mumoshualmost 7 years ago
(wow a year old issue
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
we're using helmfile with atlantis, using a custom step
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
...but would rather just keep it vanilla terraform
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
also, excited about the possibility of relying on fewer ENVs
mumoshu
mumoshualmost 7 years ago
that's cool.
i tried it but ended up building my own chatops solution w/ a github app that triggers circleci job according to the body of comments
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
and mod + variant?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
did you do the github app b/c for #chatops using #github-actions is too slow?
mumoshu
mumoshualmost 7 years ago
the chatops solution was github app + variant + helmfile/eksctl, but we have an another pipeline that continually updates things like base docker image tags with mod + variant
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
ok
mumoshu
mumoshualmost 7 years ago(edited)

did you do the github app b/c for #chatops using #github-actions is too slow?

no. but i think it works. we're planning to move everything to github actions
mumoshu
mumoshualmost 7 years ago
btw i was wondering if we can create an "oneshot" version of atlantis, that is executed on each github action run
mumoshu
mumoshualmost 7 years ago
so that we don't need to host atlantis..
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
hah! that would be cool
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
I think it might be worth opening a feature request for that one
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
see what the receptiveness will be
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
The key is to get buy-in from luke 😃
mumoshu
mumoshualmost 7 years ago
okay i will! thanks for the input