#codefresh - April 2019

Mark Grube
Mark Grubeover 7 years ago
how many variables are you setting at build time? I usually just use environment variables that are set at run time instead of build time. I add the environment variables through the pod definition. This allows my Dockerfile to be pretty generic
Mark Grube
Mark Grubeover 7 years ago
I see your screenshot that your are setting aws secrets and the port for your container. I don’t think the port matters on your Docker container, so I am assuming its more about your aws secrets. I never was a fan of keeping keys lying around, and I liked having everything controlled through IAM. May I suggest taking a look at https://github.com/uswitch/kiam ? That’s what we use to make pods have specific IAM permissions, and it works great.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
We should move that “use_codefresh” direnv function into geodesic
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
@oscarsullivan_old that’s the default behavior of geodesic
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
that’s why you can run
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
docker run myco/myinfra | bash
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
to install geodesic
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
to avoid that, do this:
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago

cmd:
      - "-l"
      - "-c"
      - "./tests/run.sh"
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
i am not sure about command vs cmd
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
looks like cmd is canonical
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
oh, so yea, cmd is the arg passe to the entrypoint
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
vs commands is run after the entrypoint
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
you can try this:
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago

cmd: ["-l", "-c", "true"]
commands:
      - "my command"
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
no
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
hrm… maybe
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
😃
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
nono
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago(edited)
in that example, you are passing --version to the entrypoint which is bash
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
so you should get the version of bash back
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
you could alternatively change the entrypoint
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago(edited)
study up on ENTRYPOINT vs CMD in docker
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
then see what we’re doing in the Dockerfile for geodesic
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
yea, the subtle nuances are often misunderstood by Dockerfile authors
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
and you see them misused and abused
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
I liken it to the system call

int execve(const char *filename, char *const argv[],
                  char *const envp[]);
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
ENTRYPOINT ~ filename
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
CMD ~ argv
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
(this is my mental model, not the docker explanation)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
You could also clone as a step in the codefresh pipeline
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
that would be accessible from inside of geodesic
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago(edited)
the benefit of doing it that way is that you can leverage the git integrations already available in codefresh without putting a secret (e.g. /conf/ssh/deployment-repo-key into git)
rss
rssover 7 years ago
Webhooks issue
Apr 3, 09:27 UTC
Resolved - The incident been resolved
Apr 3, 09:24 UTC
Investigating - We are currently investigating an issue where git webhooks don't trigger pipelines.
Running builds manually works as expected
rss
rssover 7 years ago
Codefresh availability issue
Apr 3, 12:29 UTC
Investigating - We are currently investigating a problem affecting Codefresh availability
rss
rssover 7 years ago
Codefresh availability issue
Apr 3, 12:46 UTC
Monitoring - Codefresh site is up, we're still monitoring the system
Apr 3, 12:29 UTC
Investigating - We are currently investigating a problem affecting Codefresh availability
rss
rssover 7 years ago
Codefresh availability issue
Apr 3, 13:34 UTC
Update - Google just updated us on an ongoing incident, we're monitoring the issue with google team
Apr 3, 12:46 UTC
Monitoring - Codefresh site is up, we're still monitoring the system
Apr 3, 12:29 UTC
Investigating - We are currently investigating a problem affecting Codefresh availability
rss
rssover 7 years ago
Codefresh availability issue
Apr 3, 14:23 UTC
Resolved - This incident has been resolved.
Apr 3, 14:22 UTC
Update - We are continuing to monitor for any further issues.
Apr 3, 13:34 UTC
Update - Google just updated us on an ongoing incident, we're monitoring the issue with google team
Apr 3, 12:46 UTC
Monitoring - Codefresh site is up, we're still monitoring the system
Apr 3, 12:29 UTC
Investigating - We are currently investigating a problem affecting Codefresh availability
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
@Igor Rodionov ☝️
Igor Rodionov
Igor Rodionovover 7 years ago
?
Igor Rodionov
Igor Rodionovover 7 years ago
@oscarsullivan_old ^
Igor Rodionov
Igor Rodionovover 7 years ago
check this env vars
Igor Rodionov
Igor Rodionovover 7 years ago
To debug it would be useful to see env vars and codefresh.yaml
casey
caseyover 7 years ago
@oscarsullivan_old I just setup a pipeline using the codefresh/notify/slack/deploy/webapp target using BUILD_HARNESS_VERSION=0.18.0
casey
caseyover 7 years ago
I would check your environment variables and make sure they are set correctly, I was getting similar errors, and it was because of env variables not being set
casey
caseyover 7 years ago
does anyone have suggestions for the strategy I should use for deploying to production from codefresh? I feel unsafe giving the ClusterRole codefresh-role permission to deploy to the whole cluster
casey
caseyover 7 years ago
Or is that the only way possible?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
We’ve added support for DocumentDB (MongoDB) to our Codefresh Enterprise terraform module: https://github.com/cloudposse/terraform-aws-codefresh-backing-services
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago(edited)
This makes everything that is absolutely essential to running Codefresh Enterprise (onprem) a fully managed service by AWS.
dustinvb
dustinvbover 7 years ago
Awesome! I made this little example app a little more deployable... Still working out the kinks. https://github.com/codefresh-contrib/example-voting-app
dustinvb
dustinvbover 7 years ago
Requires a few IPs right now and only works with Cloud LB not sure how to make this more portable using Istio or NGINX as both would require a DNS integration using the LB as is while requiring 2 IPs per running Helm Release is a bit much but this is also meant to demo a few things then toss after playing.
rss
rssover 7 years ago
Bitbucket access issue
Apr 11, 11:15 UTC
Investigating - We are currently investigating a problem affecting bitbucket integrations
rss
rssover 7 years ago
Build services disruption
Apr 11, 13:11 UTC
Investigating - We are currently investigating this issue
rss
rssover 7 years ago
Bitbucket access issue
Apr 11, 14:43 UTC
Resolved - The incident has been resolved
Apr 11, 11:15 UTC
Investigating - We are currently investigating a problem affecting bitbucket integrations
rss
rssover 7 years ago
Build services disruption
Apr 11, 14:43 UTC
Resolved - The incident has been resolved
Apr 11, 13:11 UTC
Investigating - We are currently investigating this issue
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago(edited)
Here are the slides from the webinar today.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Video will be posted as soon as it's available.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
rss
rssover 7 years ago
Helm repositories issue
Apr 11, 19:04 UTC
Resolved - A fix has been deployed to production and confirmed to have fixed the issue. Helm repositories are fully operational.
Apr 11, 18:49 UTC
Monitoring - The issue has been identified, and the corresponding fix was applied.
Helm repositories are accessible again,
We're closely monitoring the platform to ensure everything is working as expected.
Apr 11, 18:47 UTC
Identified - Currently Helm repositories provided by Codefresh are having issues.
This issue is under...
dustinvb
dustinvbover 7 years ago(edited)
@Erik Osterman (Cloud Posse) Do you happen to have an example of running https://github.com/cloudposse/github-commenter as a step in Codefresh pipeline using a Docker image? I've seen in other project snippets of codefresh.yml formatted examples but nothing in this project.
dustinvb
dustinvbover 7 years ago
Came up with this really quickly.

  GitHubCommenter:
    title: Add GitHub Comment
    image: cloudposse/github-commenter:latest
    environment:
      - GITHUB_TOKEN=${{GITHUB_TOKEN}} #Must be created see link for info below.
      - GITHUB_OWNER=${{CF_REPO_OWNER}}
      - GITHUB_REPO=${{CF_REPO_NAME}}
      - GITHUB_COMMENT_TYPE=pr
      - GITHUB_PR_ISSUE_NUMBER=${{CF_PULL_REQUEST_NUMBER}}
      - GITHUB_COMMENT="" #Your custom comment goes here
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
dustinvb
dustinvbover 7 years ago
Thanks!
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
dustinvb
dustinvbover 7 years ago
Is there a how-to on helmfile in Codefresh like an example codefresh.yml with an explanation on the steps?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Not that well documented
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
The “ctl” command can be ignored. It just shows how to call Helmfile
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
In this example, we support both blue/green and rolling
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Also, this example shows how we use the monochart
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
I’d be happy to jump on a call and walk you through it
Mark Grube
Mark Grubeover 7 years ago
How are you guys running stuff like integration tests and system tests? CF can’t directly access internal services that are deployed, so are you guys using some kind of proxy container in the cluster to run these kinds of tests?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
This allows you to run it all in your k8s cluster
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
we've used this for things like connecting to artifactory or using consul
dustinvb
dustinvbover 7 years ago
Yup this runner allows you to talk to the internal pods of the Kubernetes cluster using the container ips if needed.
Mark Grube
Mark Grubeover 7 years ago
this will just show up as a different run time environment on your pipeline?
dustinvb
dustinvbover 7 years ago
Add the runner to the cluster and then select the runner for the pipeline and everything happens on your cluster.
dustinvb
dustinvbover 7 years ago
Yes
Mark Grube
Mark Grubeover 7 years ago(edited)
Is there a way to split up what runs on CF runtime env and what runs on our infra? I assume that venona still reports everything up to CF UI, right?
dustinvb
dustinvbover 7 years ago
Yes all comes back to UI
dustinvb
dustinvbover 7 years ago
Each pipeline has a runtime setting. You can use our CLI to call another pipeline from your original as a step in the original and have that child pipeline run on your infra.
Michael Kolb
Michael Kolbover 7 years ago
Has anyone been using cf_export stating the name of an existing environment variable? I have tried to use it, but the variable output is getting altered.
FreeStye Bash Code
 - FEATURE=$(echo ${{CF_BRANCH}} | cut -c 9- )
      - echo $FEATURE
      - cf_export $FEATURE


Output
TI-595                                                                                                                                                  
Exporting TI-595=-595


I am seeing the cf_export command parse the static environment variable value vs. storing the defined value with the assignment cf_export $FEATURE.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
@Michael Kolb here’s a pipeline step example to setup ENV vars
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
  env:
    title: Setup Environment
    stage: Init
    fail_fast: true
    image: ${{build_image}}
    working_directory: &cwd ${{main_clone}}/${{PROJECT}}
    commands:
      - cf_export BUILD_HARNESS_VERSION=0.18.0

      # Github Commenter
      - cf_export GITHUB_OWNER=${{CF_REPO_OWNER}}
      - cf_export GITHUB_REPO=${{CF_REPO_NAME}}
      - cf_export GITHUB_COMMENT_TYPE=pr
      - cf_export GITHUB_PR_ISSUE_NUMBER=${{CF_PULL_REQUEST_NUMBER}}
      - cf_export GITHUB_COMMENT_FORMAT_FILE=${{CF_VOLUME_PATH}}/${{CF_REPO_NAME}}/codefresh/terraform/comment.txt.gotmpl
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
it exports Codefresh env vars and also those defined in Codefresh UI (Global config section and ENV vars section in each pipeline)
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)over 7 years ago
you mean the example does not use the ${{...}} syntax?
Michael Kolb
Michael Kolbover 7 years ago
So, in the documentation it says that you can state an existing variable
state the name of an existing environment variable (like EXISTING_VAR)
https://codefresh.io/docs/docs/codefresh-yaml/variables/#using-cf_export-command
Michael Kolb
Michael Kolbover 7 years ago
However, this hasn’t stored that variable assignment
Michael Kolb
Michael Kolbover 7 years ago
and the displayed use case does not show assignment in the code
Michael Kolb
Michael Kolbover 7 years ago
ersion: '1.0'
steps:
  freestyle-step-1:
    description: Freestyle step..
    title: Free styling
    image: alpine:latest
    commands:
      - cf_export VAR1=VALUE1 VAR2=VALUE2 EXISTING_VAR

  freestyle-step-2:
    description: Freestyle step..
    title: Free styling 2
    image: ${{VAR1}}
    commands:
      - echo $VAR2
      - curl http://$EXISTING_VAR/index.php
Michael Kolb
Michael Kolbover 7 years ago(edited)
is the documentation referencing a pipeline variable defined in the pipeline vs. in the pipeline yaml?
dustinvb
dustinvbover 7 years ago
Yes pipeline variables are in the pipeline SPEC file. If you need variables for the pipeline to be set during pipeline execution then you'd use cf_export to have them pop out to pipeline.
dustinvb
dustinvbover 7 years ago
The existing variable section is mentioning step-2 in the YAML above.
dustinvb
dustinvbover 7 years ago
You do not need to specify a variable name before the cf_export
Michael Kolb
Michael Kolbover 7 years ago(edited)
thanks, the documentation page could use more examples of cf_export variable assignment like the example that you provided - cf_export GITHUB_OWNER=${{CF_REPO_OWNER}}
Michael Kolb
Michael Kolbover 7 years ago
they display assignment as static references
Michael Kolb
Michael Kolbover 7 years ago
and the reference EXISTING_VAR is misleading if this is pulling from the pipeline variable definition vs. an existing variable
Michael Kolb
Michael Kolbover 7 years ago
documentation clarification would help, since I have had to run several builds trying to figure out how the cf_export process handles environment variables
Michael Kolb
Michael Kolbover 7 years ago
is there any documentation on the codefresh/kube-helm image that provides connection to Kubernetes & Tiller?
dustinvb
dustinvbover 7 years ago
Michael Kolb
Michael Kolbover 7 years ago
the docker hub release
Michael Kolb
Michael Kolbover 7 years ago(edited)
i found the command that i need kubectl config use-context <<cluster name>>
Michael Kolb
Michael Kolbover 7 years ago
the container had all the tools installed. Unfortunately, there was not documentation on the docker hub site.
dustinvb
dustinvbover 7 years ago
I would recommend trying to set the working_directory: ${{main_clone}} for the step to ensure you're in the clone repository directory.
rss
rssover 7 years ago
Docker incident: Docker Hub elevated errors
Apr 25, 17:33 UTC
Investigating - Docker has reported an incident affecting Docker Hub Web and Docker Hub Automated Builds components.
It shouldn't affect normal pulling and pushing operations.
We will keep monitoring this.

More information here: https://status.docker.com/
rss
rssover 7 years ago
Docker incident: Docker Hub elevated errors
Apr 25, 18:10 UTC
Resolved - Docker has reported this incident as Resolved.
"[Resolved] The issue has been resolved. The DB is backup again."

More information here: https://status.docker.com/pages/incident/533c6539221ae15e3f000031/5cc1ed78790d0e1ca1c8fcd4
Apr 25, 17:33 UTC
Investigating - Docker has reported an incident affecting Docker Hub Web and Docker Hub Automated Builds components.
It shouldn't affect normal pulling and pushing operations.
We will keep monitoring this.

More...
Michael Kolb
Michael Kolbover 7 years ago(edited)
Does Codefresh have API that accepts commands to start an existing pipeline? I have an integration test that will take hours and would like the option to start another pipeline once the integration tests complete.