#helmfile - December 2019

December 2019
Andrew Nazarov
Andrew Nazarovalmost 7 years ago
I’ve double checked with the latest v0.94.1, the behaviour is the same as I described in https://github.com/roboll/helmfile/issues/1010.
I’ve decided to move it out of the thread about a tf provider as it looks bloated.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
@mumoshu binary releases for terraform-provider-helmfile courtesy of @Andriy Knysh (Cloud Posse) 😃
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
Andrew Nazarov
Andrew Nazarovalmost 7 years ago
Am I right that the construction {{ .Environment.Values | get "my.var" "value" }} allows my.var not to be presented in environments files at all?
Andrew Nazarov
Andrew Nazarovalmost 7 years ago
I was wondering if helmfiles: could be used in helmfile referenced via helmfiles: from the main one. Will check this out.
Shurka
Shurkaalmost 7 years ago
Hi. Is there any guide for migrating from helm2 to helm3?
Shurka
Shurkaalmost 7 years ago
i mean if i have existing releases in datacenters, managed by helmfile, how do i migrate to manage them with helm3.
Andrew Nazarov
Andrew Nazarovalmost 7 years ago
What event makes needs: succeed? Actually, I’m interested in waiting for the postsync hook to finish.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
@here helmfile users at #aws-reinvent should connect tonight!
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
davidvasandani
davidvasandanialmost 7 years ago
@mumoshu I'll come say hello!
Andrew Nazarov
Andrew Nazarovalmost 7 years ago
Regarding of sub-helmfiles. It seems Cloud Posse uses them extensively since they have a collection of dedicated helmfiles. @Erik Osterman (Cloud Posse) how do you, guys, deal with environments? I’ve noticed you don’t have environments: blocks in your dedicated helmfiles. You don’t use them at all?
Sebastian Bauer
Sebastian Baueralmost 7 years ago
Hello, I would like to know what do you think about exposing absolute basePath to current yaml? https://github.com/roboll/helmfile/pull/981
Pierre Humberdroz
Pierre Humberdrozalmost 7 years ago
Does someone know how to disable validation ?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
can have @Andriy Knysh (Cloud Posse) help contribute it
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago(edited)
(we're struggling to use it e2e on terraform cloud without this)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
I can finish implementing and testing it tomorrow, if @mumoshu will not do it tonight :)
mumoshu
mumoshualmost 7 years ago
Replied, i'd appreciate your contribution and will try to review it asap once submitted!
mumoshu
mumoshualmost 7 years ago

Add flag to download kubectl and helmfile
mumoshu
mumoshualmost 7 years ago
would those be literally flags?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
any implementation preferences?
mumoshu
mumoshualmost 7 years ago
not sure. im not familiar with this kind of feature
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
prior art: atlantis downloads terraform
mumoshu
mumoshualmost 7 years ago(edited)
would adding helmfile_version and helmfile_download_url_template to the resource makes sense?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
ya, something like that
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
Before executing helmfile command, we need to download kubectl and helmfile, and possibly assume a role if provided
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
All optional
mumoshu
mumoshualmost 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
We need it for terraform cloud
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
Do we need to also download helm? Helmfile doesn’t do it automatically?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
yes, good point - so I think we need helm, helmfile, and kubectl
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
Yes
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
And assume role code if it’s provided
mumoshu
mumoshualmost 7 years ago
got it
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
(lol, and hrm... maybe this is cascading out of control)
mumoshu
mumoshualmost 7 years ago
iam assume role?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
need helm-diff plugin too
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
and then helm-diff might have deps
mumoshu
mumoshualmost 7 years ago
ah
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
ugh
mumoshu
mumoshualmost 7 years ago
helm-diff has no deps as it's a single binary tool written in golang
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
maybe we generalize it?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago(edited)
package_deps which is a list of binaries by URL
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
Yes
mumoshu
mumoshualmost 7 years ago
wouldn't assume role automatically done today by installing aws cli and using the correct kubeconfig?
mumoshu
mumoshualmost 7 years ago
package_deps would worth a dedicated terraform provider, right?
mumoshu
mumoshualmost 7 years ago
and i think it doesn't match with helmfille providers use-case
mumoshu
mumoshualmost 7 years ago
as it would typically require multiple versions of helmfile/helm/kubectl binaries to co-exist
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
Order of operations matters
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
possibly - just for background, we tried using null_resource with local-exec provisioner to download, but cannot get it to work due to limited ability to control when it's executed (we need it executed prior to helmfile-provider getting invoked which depends on the binaries)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
Terraform cloud can’t even plan without those binaries
mumoshu
mumoshualmost 7 years ago
yep. i wondered there was some way to explicitly draw a dependency between local-exec and helmfile resourec
mumoshu
mumoshualmost 7 years ago
apparently it isn't possible?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
so for example, we do this successfully for downloading aws cli in one step and calling it in another, both using local-exec
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
however, since local-exec is out of phase with terraform-helmfile-provider, we cannot control for the order of execution
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
so the first time we run it, it works
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
Local exec is not executed on plan
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
precisely
mumoshu
mumoshualmost 7 years ago
ah good to know!
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
astro also donwloads terraform
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
on demand based on version
mumoshu
mumoshualmost 7 years ago
maybe we should add helmfile_version, helm_version, kubectl_version, helm_diff_version for starter
mumoshu
mumoshualmost 7 years ago
and implement some other provider to install arbitrary pkg from e.g. apk if necessary, for use with depends_on = [that_provider.some_bin] to draw dependency
mumoshu
mumoshualmost 7 years ago
ah okay depends_on = [that_provider.some_bin] won't work anyway
mumoshu
mumoshualmost 7 years ago
as the provider won't install it on plan. that's the same thing as local-exec/null-resource
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
can one tie into the init phase?
mumoshu
mumoshualmost 7 years ago
but it is unlikely to provide helmfile_releaseset resources on init
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
I think a better way would be not to pollute terraform-provider-helmfile with loading external packages, but using data sources b/c they are executed on terraform plan
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
we can use terraform-provider-shell b/c it already implements data source together with resource https://github.com/scottwinkler/terraform-provider-shell/blob/master/examples/test.tf#L5
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
or we can add a similar data source to terraform-provider-helmfile
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
If the query constraint arguments for a data resource refer only to constant values or values that are already known, the data resource will be read and its state updated during Terraform's "refresh" phase, which runs prior to creating a plan

Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
Each data source in turn belongs to a provider, which is a plugin for Terraform that offers a collection of resource types and data sources that most often belong to a single cloud or on-premises infrastructure platform.

Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
Each provider may offer data sources alongside its set of resource types

Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
@mumoshu @Erik Osterman (Cloud Posse) ^
mumoshu
mumoshualmost 7 years ago
@Andriy Knysh (Cloud Posse) ah, so your idea is based on the fact that datasource is evaluated even at plan time, right?
mumoshu
mumoshualmost 7 years ago(edited)
would it look like this?

data "helmfile_binary" "default" {
  version = "0.94.1"
}

data "helm_binary" "default" {
  version = "3.0.0"
}

resource "helmfile_releaseset" "myapp" {
  helmfile_binary = "${helmfile_binary.default.path}`
  helm_binary = "${helm_binary.default.path}`
}
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
yes!
mumoshu
mumoshualmost 7 years ago
excellent! that would be much much better than adding version related keys to resources.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago(edited)
in the data source, we can hide the fact that it downloads the data (make it more specific, as you showed above)), or we can allow to execute any commands (as terraform-provider-shell does) and make it more generic
mumoshu
mumoshualmost 7 years ago
let's hide it for now, as it would be dedicated to helmfile provider's use-case.
mumoshu
mumoshualmost 7 years ago
adding optional key like download_url and/or apk_package would be alright
Andrew Nazarov
Andrew Nazarovalmost 7 years ago
At the moment we are just building a docker container with all the stuff pre-installed. Looking forward to trying a new implementation.
Vadim Bauer
Vadim Baueralmost 7 years ago(edited)
What do you think?

Do the helmfile concepts make sense in the context of Pulumi with Helm?
Here is a Helm example with Pulumi -> https://github.com/pulumi/pulumi-kubernetes#deploying-a-helm-chart

I am thinking about moving to Pulumi becuase of its first class k8s support. As a bonus there is also Helm support.
It would be possible to bridge the terraform-provider-helmfile to Pulumi but I wonder if it will provided value to add another layer.

As much as I like the awesomeness of the Helmfile, aren't it too much of layers? 1 -> Pulumi 2-> Helm 3-> Helmfile especially in the imperative nature of Pulumi.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
@Vadim Bauer see related discussion here: https://sweetops.slack.com/archives/CQCDCLA1M/p1573246618316100
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
@mumoshu adds some interesting insights as well
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
also, see the comment from @tamsky
Juan Soto
Juan Sotoalmost 7 years ago
Hi, I have a k8s cluster running, also the kubeconfig is correctly configured. I wanna start playing with helmfile. Could you suggest me a good tutorial in order to start learning ?
mumoshu
mumoshualmost 7 years ago
maybe try cloudposse's guide? https://docs.cloudposse.com/tools/helmfile/
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
(It’s a bit dated at this point)
TBeijen
TBeijenalmost 7 years ago
Q: Adding aws-iam-authenticator. Good idea? Bad idea?
mumoshu
mumoshualmost 7 years ago
could u elaborate a bit more? how does it relate to helmfile?
TBeijen
TBeijenalmost 7 years ago
@mumoshu See thread
TBeijen
TBeijenalmost 7 years ago
Other Q: I was hoping to toggle certain applications using values defined in the cluster-specific environment. Like this (but that throws a 'map has no entry' error):
release:
  - name: efs-provisioner
    namespace: kube-system
    chart: stable/efs-provisioner
    version: "0.8.0"
    installed: {{ .Environment.Values.efsProvisioner.installed }}
    values:
      - releases/efs-provisioner.yaml.gotmpl

Is this possible in some way?
Dudi Cohen
Dudi Cohenalmost 7 years ago
Hey guys, i'm trying to get the name of the chart as a variable rather than the name of the release:
releases:
- name: external-dns
  chart: my-repo/external-dns
  namespace: kube-system
  version: 1.14.0
  installed: true
  values:
  - "{{`{{ .Release.Chart }}`}}/values-dev.yaml"

Any idea on how I change .Release.Chart to only include the chart name?
Jonathan
Jonathanalmost 7 years ago
replace Chart with Name like so: - "{{{{ .Release.Name }}}}/values-dev.yaml" That should then resolve to "external-dns/values-dev.yaml"
Dudi Cohen
Dudi Cohenalmost 7 years ago
Thanks @Jonathan, but I need the name of the chart, not the name of the release as these two could be different.
TBeijen
TBeijenalmost 7 years ago
Is helm 3 not able to create namespaces? I run into an error whenever helmfile.yaml contains a release targeted at a non-existing namespace. (Using roboll/helmfile:helm3-v0.94.1)
Ryan Smith
Ryan Smithalmost 7 years ago
Lazy question 😅, does helmfile support replace("string_here", "to_replace", "replace_with") functionality? I recall it inherits functionality from another project, but I'm spacing out on finding that information.
Ryan Smith
Ryan Smithalmost 7 years ago
(Actually, I thought of a work around, but I'll leave the question here in case it's helpful for others.)
Alucas
Alucasalmost 7 years ago
hmm I’m having issues adding inline values to my environment files using something like:
environments:
  dev:
    values:
    - region: "westus"

- name: chart
  chart: stable/chart
    set:
      - name: config.client.external_labels.region
        value: {{ .Environment.Values.region }}

What am I missing here?
Alucas
Alucasalmost 7 years ago
in ./helmfile.yaml: error during helmfile.yaml.part.0 parsing: template: stringTemplate:87:30: executing "stringTemplate" at <.Environment.Values.region>: map has no entry for key "region"
Alucas
Alucasalmost 7 years ago
Okay, this is weird.. If I use my base: environments.yaml at the top of the helmfile it doesn’t work, if I move everything in the environments.yaml file into the helmfile it works.
mumoshu
mumoshualmost 7 years ago
@Alucas hm... sounds like helmfile needs to parse your helmfile.yaml template to finally notice that it needs to load your base environments.yaml. this is kind of a chicken-and-egg problem
Alucas
Alucasalmost 7 years ago
If I load the base: and the env at the top it doesn’t work, so I cannot have the base: environment.yaml that also includes the environments.
mumoshu
mumoshualmost 7 years ago
We need to resolve #932. This is a fundamental issue in helmfile
mumoshu
mumoshualmost 7 years ago
ya
mumoshu
mumoshualmost 7 years ago
so try this
mumoshu
mumoshualmost 7 years ago
bases:
- environments.yaml
---
- name: chart
  chart: stable/chart
    set:
      - name: config.client.external_labels.region
        value: {{ .Environment.Values.region }}

mumoshu
mumoshualmost 7 years ago
this would work as helmfile separates your template at ---, loading the base environments.yaml before rendering the second part
mumoshu
mumoshualmost 7 years ago
just show me your full example including bases and environments and everything else if this doesnt make sense(i might be misreading something
mumoshu
mumoshualmost 7 years ago
#helmfile users - do you already use needs? I'm considering to rename it to after. wdyt?

https://github.com/roboll/helmfile/issues/1018#issuecomment-565238688
Alucas
Alucasalmost 7 years ago
@mumoshu I think that fixed it for now, thanks!
Bart M.
Bart M.almost 7 years ago
is there any way to mark a release as 'to be deleted'? we have the issue that we deploy multiple versions of the same microservices with helmfile, but now we need to clean up old deprecated deploys somehow...
Dudi Cohen
Dudi Cohenalmost 7 years ago(edited)
Is there a way to iterate over helmfiles contents?
For example:
helmfiles:
- path: ../*/*/release-*.yaml

releases:
{{ range $key, $values := helmfiles }}
- name: {{ .name }}
  chart: {{ .chart }}
  version: {{ .version }}
  installed: {{ .installed }}
{{ end }}

@mumoshu please 🙂
Jonathan
Jonathanalmost 7 years ago
Hey! Does anyone have any ideas on how one would go about allowing an additional release to be installed when setting a value/environment variable? not sure how the selector flag would handle installing the correct release, and not install it if the variable is not set. This is how I structured my helmfile, any pointers/questions regarding this would be great!
bases:
- ../commons/environments.yaml
---
releases:
- name: "release-name"
  namespace: {{ .Namespace }}
  labels:
    chart: "release-name"
  chart: stable/example-chart
  values:
  - ../releases/values.yaml.gotmpl
{{ if eq .Values.feature_branch "Y" }} # prefferably an env variable, not sure how that is done
  - extraval:
      enabled: True
- name: "feature_branch_release"
  namespace: {{ .Namespace }}
  chart: stable/other-chart
  values:
  ../releases/feature_branch_values.yaml.gotmpl
{{ end }}
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
We generate N releases dynamically using an environment with a custom schema that we loop through
Juan Soto
Juan Sotoalmost 7 years ago
I want to deploy a custom helm chart, but I can’t see how to reference to a local chart
Juan Soto
Juan Sotoalmost 7 years ago
all the examples I see they suggest:
Juan Soto
Juan Sotoalmost 7 years ago
chart: stable/prometheus references to a remote repository. I want to run the chart I have in my local directory.
Juan Soto
Juan Sotoalmost 7 years ago
I want to deploy my helm chart which is in a local directory chart1/
Juan Soto
Juan Sotoalmost 7 years ago
Any ideas?
mumoshu
mumoshualmost 7 years ago
@Juan Soto Try chart: ./chart1 :simple_smile:
Marjan Jordanovski
Marjan Jordanovskialmost 7 years ago(edited)
Hello guys, I'm trying to create one deployment.yaml file inside helm chart, that will create 2 pods when I run helmfile. So my problem is that I do not know how to properly reference values inside deployment.yaml that will point to my .yaml.gotmpl files.
Marjan Jordanovski
Marjan Jordanovskialmost 7 years ago(edited)
So what I want is that on every place in deployment.yaml where it's {{ .values }} , I want helm to detect that there are 2 gotmpl inside values/ (I do not want to specify exactly the name of that gotmpl because that would mean that I need 2 deployment files) , and to somehow iterate through them in order to create 2 deployments from that one deployment.yaml (for example first deployment would get all values from first gotmpl, and second created deployment would get all values from second gotmpl)
Jonathan
Jonathanalmost 7 years ago
Is there any way to have different values for the same property in the values yaml.gotmpl file depending on what an environment variable is set to? e.g. if $ENV is set to "enable", service.annotations and image.tag are set to value "x" and "y" respectively, but if $ENV is "disable" they are set to value "a" and "b"?
Juan Soto
Juan Sotoalmost 7 years ago
Hello I am having this error when I try to reference the value of an environment variable inside the deployment
Juan Soto
Juan Sotoalmost 7 years ago
this is the deployment file, and I want to set the value of the environment variable helm.environment into ENVIRONMENT var. The value was set from command line using helmfile -e dev lint
Juan Soto
Juan Sotoalmost 7 years ago
Any ideas?
Juan Soto
Juan Sotoalmost 7 years ago
Juan Soto
Juan Sotoalmost 7 years ago
Juan Soto
Juan Sotoalmost 7 years ago
Pierre Humberdroz
Pierre Humberdrozalmost 7 years ago(edited)
I have never used the stages features but should it not just be {{ .Values.helm.environment }} ?
Juan Soto
Juan Sotoalmost 7 years ago
I changed it to .Values as you suggested but no luck.
Juan Soto
Juan Sotoalmost 7 years ago
at <.Values.helm.environment>: nil pointer evaluating interface {}.environment
Juan Soto
Juan Sotoalmost 7 years ago
any ideas @mumoshu ?
mumoshu
mumoshualmost 7 years ago
@Juan Soto You can't use environment values in helm chart templates
mumoshu
mumoshualmost 7 years ago
Helmfile environment values are dedicated to templates processed by Helmfile, like helmfile.yaml itself, and values.yaml.gotmpl read and rendered by Helmfile.
mumoshu
mumoshualmost 7 years ago
So you write it like value: {{ .Values.helm.environment }} in your "chart" template, without .Environment
Juan Soto
Juan Sotoalmost 7 years ago
I see.
mumoshu
mumoshualmost 7 years ago
and pass that specific chart values by writing like this in helmfile.yaml

releases:
- name: yourapp
   chart: ./chart1
  values:
  - helm:
       environment: {{ .Values.helm.environment }}

mumoshu
mumoshualmost 7 years ago(edited)
or
releases:
- name: yourapp
  chart: ./chart1
  values:
  - somevalues.yaml containing helm.environment

Juan Soto
Juan Sotoalmost 7 years ago
Juan Soto
Juan Sotoalmost 7 years ago
That is what I did. And then, I referenced {{ environment_chart }} inside the helm chart.
Juan Soto
Juan Sotoalmost 7 years ago
Is it a good practice @mumoshu?
mumoshu
mumoshualmost 7 years ago
yes that looks good
Juan Soto
Juan Sotoalmost 7 years ago
it worked
Juan Soto
Juan Sotoalmost 7 years ago
🙂
Yingjun Hu
Yingjun Hualmost 7 years ago
It seems like it's the similar question I posted yesterday https://github.com/roboll/helmfile/issues/1045. I was trying to pass the environment values from my top level helmfile to sub helmfiles but wasn't able to. Any advice?
Yingjun Hu
Yingjun Hualmost 7 years ago
From what I understand after debugging yesterday, it seemed to me that whenever there's a change of directory, the env values will be cleaned and not be passed to nested helmfiles.
Juan Soto
Juan Sotoalmost 7 years ago
Another question. As I have two environments ready to be deployed. I would to implemente an strategy that allows me to deploy both environments at the same time. With the current configuration, each time I deploy to dev, it overwrites the stage environment. Are there any solution to have both envs up and running? What do you think?
mumoshu
mumoshualmost 7 years ago
@Yingjun Hu fyi i've replied to you in https://github.com/roboll/helmfile/issues/1045 😃
mumoshu
mumoshualmost 7 years ago
⭐️ Requesting feedbacks regarding two biggest fundamental issues in Helmfiel and corresponding enhancement proposals:

- https://github.com/roboll/helmfile/issues/1048
- https://github.com/roboll/helmfile/issues/932
Juan Soto
Juan Sotoalmost 7 years ago
Hi, I ’ve been trying different approaches but I am still having problems referencing Environment Values in the helmfile.yaml
Juan Soto
Juan Sotoalmost 7 years ago
mumoshu
mumoshualmost 7 years ago(edited)
Your helmfile.yaml?
Juan Soto
Juan Sotoalmost 7 years ago
mumoshu
mumoshualmost 7 years ago
Separate releases and bases with ---
Juan Soto
Juan Sotoalmost 7 years ago
Great! It worked
Juan Soto
Juan Sotoalmost 7 years ago
Now I am having different issue
Juan Soto
Juan Sotoalmost 7 years ago
mumoshu
mumoshualmost 7 years ago
you need to create it with kubectl create ns dev beforehand in helm 3
mumoshu
mumoshualmost 7 years ago(edited)
or use raw chart

releases:
- name: ns
  chart: incubator/raw
  values:
  - resources:
    - apiVersion: v1
       kind: Namespace
      metadata:
        name: dev
- name: yourapp
  chart: yourchart
  namespace: dev
  needs:
  - ns
Juan Soto
Juan Sotoalmost 7 years ago
Can’t I include the creationg of the ns using the chart?
mumoshu
mumoshualmost 7 years ago
you can. see above
Juan Soto
Juan Sotoalmost 7 years ago
ok
Juan Soto
Juan Sotoalmost 7 years ago
Juan Soto
Juan Sotoalmost 7 years ago
in ./helmfile.yaml: failed to read helmfile.yaml: reading document at index 1: yaml: line 7: mapping values are not allowed in this context
Juan Soto
Juan Sotoalmost 7 years ago
humn…identation problem
mumoshu
mumoshualmost 7 years ago
yup. fix it and it would work
Juan Soto
Juan Sotoalmost 7 years ago(edited)
great. Fixed!. What’s more, both environments are being deployed to their relevant namespaces
Juan Soto
Juan Sotoalmost 7 years ago
Both environment are working! Great
Juan Soto
Juan Sotoalmost 7 years ago
I am having the following error
Juan Soto
Juan Sotoalmost 7 years ago(edited)
$ helmfile -e stage apply
in ./helmfile.yaml: failed to read helmfile.yaml: reading document at index 1: yaml: unmarshal errors:
  line 31: cannot unmarshal !!int `5` into []string
Juan Soto
Juan Sotoalmost 7 years ago
Juan Soto
Juan Sotoalmost 7 years ago
any ideas?
mumoshu
mumoshualmost 7 years ago
try seeing line 31 of your subhelmfile
mumoshu
mumoshualmost 7 years ago(edited)
the error indicates it's due to a type mismatch between your replicas defined in stage.yaml and your call-side(line 31 of sub-helmfile)
Juan Soto
Juan Sotoalmost 7 years ago
mumoshu
mumoshualmost 7 years ago
values: {{ .Environment.Values.replicas }} seems wrong
Juan Soto
Juan Sotoalmost 7 years ago
namespace and environment values are string and they went through
Juan Soto
Juan Sotoalmost 7 years ago
but replicas variable for some reason is not working.
Juan Soto
Juan Sotoalmost 7 years ago
It has an integer as it value.
mumoshu
mumoshualmost 7 years ago
yeah so use value: {{ .Environment.Values.replicas }} not values: {{...}
Juan Soto
Juan Sotoalmost 7 years ago
thanks
Juan Soto
Juan Sotoalmost 7 years ago
Now I need to put that value inside the helm chart
Juan Soto
Juan Sotoalmost 7 years ago
in chart1/values.yml
Juan Soto
Juan Sotoalmost 7 years ago
Juan Soto
Juan Sotoalmost 7 years ago
I am going to change it to
Juan Soto
Juan Sotoalmost 7 years ago
Juan Soto
Juan Sotoalmost 7 years ago
$ helmfile -e stage apply
Building dependency release=caylent-helmfile, chart=caylent
in ./helmfile.yaml: helm exited with status 1:
  Error: cannot load values.yaml: error converting YAML to JSON: yaml: invalid map key: map[interface {}]interface {}{".Values.nreplicas":interface {}(nil)}
mumoshu
mumoshualmost 7 years ago
you cant use go template expressions in yaml file
mumoshu
mumoshualmost 7 years ago
change it to .gotmpl
Andrew Nazarov
Andrew Nazarovalmost 7 years ago
Do sub-helmfiles inherit helmDefaults?
Roderik van der Veer
Roderik van der Veeralmost 7 years ago
I’m trying to run helmfile with helm 3, (3.0.2) and i installed helm diff 3.0.0-rc.7 but when running the diff plugin fails without an error message
Comparing release=namespace, chart=incubator/raw
Comparing release=shared-secets, chart=settlemint/shared-secrets
Comparing release=rabbitmq, chart=stable/rabbitmq-ha
Comparing release=launchpad, chart=settlemint/launchpad
Comparing release=ingress, chart=stable/nginx-ingress
in ./helmfile.yaml: 5 errors:
err 0: failed processing release rabbitmq: helm exited with status 1:
  
err 1: failed processing release namespace: helm exited with status 1:
  
err 2: failed processing release launchpad: helm exited with status 1:
  
err 3: failed processing release shared-secets: helm exited with status 1:
  
err 4: failed processing release ingress: helm exited with status 1:
  

Any idea what’s wrong or on how to debug this?
Roderik van der Veer
Roderik van der Veeralmost 7 years ago(edited)
Sorry for the flurry of questions, but what would be the reason that i get a ’failed to download incubator/raw` error with this helmfile
 repositories:
  - name: stable
    url: <https://kubernetes-charts.storage.googleapis.com>
  - name: incubator
    url: <https://kubernetes-charts-incubator.storage.googleapis.com>
  - name: harbor
    url: <https://helm.goharbor.io>
  - name: elastic
    url: <https://helm.elastic.co>
  - name: kiwigrid
    url: <https://kiwigrid.github.io>


helmDefaults:
  cleanupOnFail: true
  verify: true
  wait: true
  timeout: 600
  force: true
  atomic: true

releases: 
  - name: mynamespace
    chart: incubator/raw
    version: 0.2.3
    values:
      - resources:
        - apiVersion: v1
          kind: Namespace
          metadata:
            name: {{ .Namespace }}
Roderik van der Veer
Roderik van der Veeralmost 7 years ago
processing releases in group 1/4: alpha/mynamespace
worker 1/1 started
worker 1/1 finished
worker 1/1 started
Upgrading release=mynamespace, chart=incubator/raw
exec: helm upgrade --install --reset-values mynamespace incubator/raw --version 0.2.3 --verify --wait --timeout 600s --force --atomic --cleanup-on-fail --namespace alpha --values /var/folders/nr/3q7q00qs4hv46dwwffc9jp6h0000gn/T/values359668070 --history-max 10
exec: helm upgrade --install --reset-values mynamespace incubator/raw --version 0.2.3 --verify --wait --timeout 600s --force --atomic --cleanup-on-fail --namespace alpha --values /var/folders/nr/3q7q00qs4hv46dwwffc9jp6h0000gn/T/values359668070 --history-max 10: 
worker 1/1 finished

FAILED RELEASES:
NAME
mynamespace
err: release "mynamespace" in "helmfile.yaml" failed: failed processing release mynamespace: helm exited with status 1:
  Error: failed to download "incubator/raw" (hint: running `helm repo update` may help)
in ./helmfile.yaml: failed processing release mynamespace: helm exited with status 1:
  Error: failed to download "incubator/raw" (hint: running `helm repo update` may help)
Roderik van der Veer
Roderik van der Veeralmost 7 years ago
incubator is there when doing helm repo list, searching returns the raw chart and version
Roderik van der Veer
Roderik van der Veeralmost 7 years ago
found it:
Error: failed to fetch provenance "<https://kubernetes-charts-incubator.storage.googleapis.com/raw-0.2.3.tgz.prov>"
helm.go:76: [debug] failed to fetch provenance "<https://kubernetes-charts-incubator.storage.googleapis.com/raw-0.2.3.tgz.prov>"
Andrew Nazarov
Andrew Nazarovalmost 7 years ago(edited)
Yep, verify: false should fix this.
Roderik van der Veer
Roderik van der Veeralmost 7 years ago
is it normal that if i put it to true in the defaults, that setting it to false on the release does not work?
Andrew Nazarov
Andrew Nazarovalmost 7 years ago
Could be a bug:)
Juan Soto
Juan Sotoalmost 7 years ago
Hi, thank you all for helping me to develop my solution with Helmfile
Juan Soto
Juan Sotoalmost 7 years ago
now I need to integrate this deployment in CircleCI
Juan Soto
Juan Sotoalmost 7 years ago
They way I am deploying my app is:
Juan Soto
Juan Sotoalmost 7 years ago
Juan Soto
Juan Sotoalmost 7 years ago
Anybody has experience on running helmfile pipelines on CircleCI ? I already have the source code in github.
Juan Soto
Juan Sotoalmost 7 years ago
and the K8s cluster is running on GKE
Juan Soto
Juan Sotoalmost 7 years ago
Anybody has tested that ?
Balaji J
Balaji Jalmost 7 years ago
Hi all, hope you doing good. Im completely new to helm and helmfile. Can someone please clarify the difference between using helmfile vs creating a umbrella chart with dependency using requirements.yaml?
Are they not solving the same usecase of deploying a group of charts for a complete deployment of applicaiton with dependant charts?
Can you please clarify to understand the advantage the helmfile gives apart from environment specific deployment support like dev,production etc?
DanB
DanBalmost 7 years ago
We are looking into adopting helmfile for our existing helm use cases. We currently have 15-20 active clusters at any given time, but only about 10 are permanent (we spin up clusters several times throughout the day). These clusters are spread accross aws accounts, on prem and across regions. We don't have the concept of a linear dev -> stage -> prod. To get to the point, can anyone point me to an example helmfile that might fit this use case?
Yingjun Hu
Yingjun Hualmost 7 years ago
It seems like the releases defined in helmfile are executed async?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
I think there's a --concurrency flag you can use to set parallelism to 1 and then it would be synchronous
Yingjun Hu
Yingjun Hualmost 7 years ago
I see, thanks!