#codefresh - August 2019

August 2019
dustinvb
dustinvbabout 7 years ago
@Erik Osterman (Cloud Posse) Do you have an example step where you utilize Helmfile in a Codefresh pipeline?
dustinvb
dustinvbabout 7 years ago
Have someone who would like to see how this is used in a Codefresh pipelin
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
yes, but not maintained/current.
dustinvb
dustinvbabout 7 years ago
There is nothing your team uses currently as part of Codefresh deploy with a helmfile in a pipeline? I thought this was something you were doing regularly for a majority of your Helm deploys.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
oh that
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago(edited)
but the ctl command there is optional. we have an inline approach we use with our latest pipelines.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
instead of that deploy_helmfile, we use a step more like
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago

deploy_helmfile:
    title: Deploy with helmfile
    stage: Deploy
    image: "${{TOOL_DOCKER_REPO_PREFIX}}/${{AWS_ENV_NAME}}:latest"
    working_directory: /conf/
    environment:
      - 'KUBECONFIG=${{CF_KUBECONFIG_PATH}}'
      - 'REPO_ROOT=${{CF_VOLUME_PATH}}/${{CF_REPO_NAME}}'
      - 'APP_HOST=${{APP_HOST_DOMAIN}}'
    shell: bash
    commands:
      # Announce the release version
      - echo "* Preparing to deploy ${RELEASE_NAME} with image ${IMAGE_NAME}:${IMAGE_TAG}"
      - echo "*     with flavor \"$FLAVOR\" into namespace ${RELEASE_NAMESPACE}"
      - kubectl config use-context "${KUBE_CONTEXT}"
      - cd $(conf-directory helmfiles) && echo Working directory is now $PWD
      - direnv exec . helmfile --namespace="${RELEASE_NAMESPACE}" --environment="${FLAVOR}"
        -f "${REPO_ROOT}/deploy/helmfile.yaml" sync
    when:
      steps:
      - name: ask_for_permission
        on:
        - approved
Alex Siegman
Alex Siegmanabout 7 years ago(edited)
Seems that pending-approval steps don't parse variables in the title field. I'd love to be able to customize that text for better UX on reused pipelines. Anyone done something similar? Trying to do this:

  ask_for_permission:
    type: pending-approval
    title: Deploy release of ${{APP_NAME}}:${{IMAGE_TAG}} to ${{FLAVOR}}?
    stage: Prepare

But I get the text seen in title with no substitution for the variables. Is this a feature request for the codefresh folks?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
@Alex Siegman totally agree
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
We've submitted a similar feature request
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
I think the more that do it, the better
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
Go to support.codefresh.io to add your vote for this
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
I know @Jeremy G requested it too
Jeremy G
Jeremy Gabout 7 years ago
@Jeremy G has joined the channel
Alex Siegman
Alex Siegmanabout 7 years ago
Feature request sent
btai
btaiabout 7 years ago
for codefresh, sometimes my pipeline will pull a previously built image to use as the cache and other times it will say the following:
Previous built image: <http://r.cfcr.io/org/repo_name@sha256:hash|r.cfcr.io/org/repo_name@sha256:hash> exists on the daemon and will be us
ed as cache for the build 
btai
btaiabout 7 years ago
so im wondering if you have many devs working concurrently on the same repo (in different branches) then does that mean that it’s more likely your pipeline will need to pull the image (adds ~1.5-2min for a 2.5GB image) as opposed to already having the image existing on the daemon
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago(edited)
yes, I think it depends on if the build hits a worker (and thus different daemon) with the image cached.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
@Igor Rodionov spent a lot of time looking into this recently
btai
btaiabout 7 years ago
😕
btai
btaiabout 7 years ago
when you say worker
btai
btaiabout 7 years ago
you mean k8s worker i assume that codefresh is running on?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
so behind the scenes they run Docker-in-Docker (for security reasons).
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
it spawns a dind on demand
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
i thin their SaaS product works slightly different than their enterprise product though
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
best thing is to go to <http://support.codefresh.io|support.codefresh.io> and open issue
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
they'll explain better than me 😃
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
@dustinvb is on vacay i think
btai
btaiabout 7 years ago
is dustin the only cf guy on sweetops?
btai
btaiabout 7 years ago
i was under the impression each pipeline had a persistent volume that would store many docker image layers
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
this is just an informal community driven channel. @Dan Garfield and @Francisco Cocozza are here too, but may not be SME.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
And @Oriel Belzer for enterprises sales questions 😃
btai
btaiabout 7 years ago(edited)
no worries, i wasnt planning on reaching out to anyone directly but if they read this channel and have comments, im not opposed 😛
btai
btaiabout 7 years ago
i guess im wondering how I can maximize the image existing on the daemon so the build doesnt spend an extra 1.5~2min pulling the image first
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
yea, we've been bit by this as well. for on prem, we reduced the number of build nodes and increased the size of the nodes.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago(edited)
not sure what to do for the hosted version though to improve cache hits (or if its even the problem we had)
btai
btaiabout 7 years ago
that makes sense
btai
btaiabout 7 years ago
for on prem
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
btai
btaiabout 7 years ago
is it possible to have a pipeline that listens on a github org? allowing it to create new projects/pipelines based on new repo creation?
btai
btaiabout 7 years ago
woops forgot @Erik Osterman (Cloud Posse) already asked this above ^^ 😅
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
haha
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
😃
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
yea, not yet.
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
however, using repo templates might not be a bad start
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
also, in the repo template, include a pipeline-spec
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
and a makefile
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago

version: '1.0'
kind: pipeline
metadata:
  name: ourcompany/example-app/destroy-pull-request
  isPublic: false
  project: etan-tolling/example-app
  deprecate:
    repoPipeline: true
    implicitGitCloneService:
      context: github
      scm:
        provider: github
        owner:
          name: ourcompany
        name: example-app
spec:
  triggers:
    - name: ourcompany/destroy-pr
      type: git
      repo: ourcompany/example-app
      events:
        - pullrequest.closed
        - pullrequest.unlabeled
      pullRequestAllowForkEvents: false
      commentRegex: /.*/gi
      branchRegex: /.*/gi
      branchRegexInput: regex
      provider: github
      disabled: false
      options:
        noCache: false
        noCfCache: false
        resetVolume: false
      context: CF-default
      contexts: []
      variables: []
  steps: {}
  stages: []
  variables: []
  contexts:
    - Slack
    - GitHub
  specTemplate:
    location: git
    repo: ourcompany/example-app
    path: ./codefresh/destroy-pull-request.yml
    context: github
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
that's an example spec
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago

codefresh create pipeline -f destroy-pull-request.yml
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)about 7 years ago
A Codefresh pipeline for creating pipelines
btai
btaiabout 7 years ago
^ that requires a monorepo 😕
rss
rssabout 7 years ago
Github services are partially degraded
Aug 22, 10:14 UTC
Investigating - GitHub informs:
Issues, PRs, Dashboard, Projects, Notifications are partially degraded
rss
rssabout 7 years ago
Github services are partially degraded
Aug 22, 11:11 UTC
Update - GitHub reports a major outage on
Issues, PRs, Dashboard, Projects, Notifications servicesAug 22, 10:14 UTC
Investigating - GitHub informs:
Issues, PRs, Dashboard, Projects, Notifications are partially degraded
rss
rssabout 7 years ago
Github services are partially degraded
Aug 22, 12:23 UTC
Resolved - This incident has been resolved.Aug 22, 11:11 UTC
Update - GitHub reports a major outage on
Issues, PRs, Dashboard, Projects, Notifications servicesAug 22, 10:14 UTC
Investigating - GitHub informs:
Issues, PRs, Dashboard, Projects, Notifications are partially degraded
rss
rssabout 7 years ago
Bitbucket has reported an incident on their service
Aug 27, 17:23 UTC
Investigating - Incident details:
Bitbucket commits API high response time and timeouts
This incident affects: API, SSH, Git via HTTPS, Mercurial via HTTPS, and Pipelines.

More info: https://bitbucket.status.atlassian.com/incidents/d1nyzp56331y
rss
rssabout 7 years ago
Bitbucket has reported an incident on their service
Aug 28, 08:47 UTC
Resolved - The incident has been resolved and all systems are operational.Aug 27, 17:23 UTC
Investigating - Incident details:
Bitbucket commits API high response time and timeouts
This incident affects: API, SSH, Git via HTTPS, Mercurial via HTTPS, and Pipelines.

More info: https://bitbucket.status.atlassian.com/incidents/d1nyzp56331y
btai
btaiabout 7 years ago
for those using direnv in codefresh, do you have a seperate file that you rename to .envrc when the pipeline runs? How do you handle committing that to the repo for the pipeline and having a local copy
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
I don’t follow your question
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
But this is how we did it
rss
rssabout 7 years ago
Codefresh build clusters networking issues
Aug 31, 14:39 UTC
Investigating - Amazon has reported connectivity issues affecting some instances.
They are currently working on investigating the issues
More information can be found on
https://status.aws.amazon.com/
rss
rssabout 7 years ago
Codefresh build clusters networking issues
Aug 31, 20:47 UTC
Resolved - AWS Incident is no longer affecting CF.Aug 31, 14:39 UTC
Investigating - Amazon has reported connectivity issues affecting some instances.
They are currently working on investigating the issues
More information can be found on
https://status.aws.amazon.com/