archived the channel

Want to see if we can get a more abstract version of

## Run tests in docker container
docker/test:
	docker run --name terratest --rm -it -e GITHUB_TOKEN \
		-e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY -e AWS_SESSION_TOKEN -e AWS_ENDPOINT_URL \
		-e PATH="/usr/local/terraform/bin:/go/bin:/usr/local/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" \
		-v $(CURDIR)/../../:/module/ cloudposse/test-harness:latest -C /module/test/src test

in the build (or test?) harness (and some other magic here as well).
What would be the best place to add it? as it involves docker, aws and terraform

Most Cloud Posse projects install build-harness under the project's repository root to manage things like building Docker images or generating README.md from README.yaml. In turn, build-harness installs some tools it needs when it needs them. We had been using INSTALL_PATH to determine where build-harness installs its tools, intending that they be installed under the build-harness tree, but this is in conflict with Geodesic, which uses INSTALL_PATH to determine where to install the Geodesic wrapper/command. This was causing build-harness to install gomplate in /usr/local/bin , which was not the intention.

So we changed build-harness to install its tools into PACKAGES_INSTALL_PATH instead. Unfortunately, this broke some workflows, particularly pre-commit workflows, which use INSTALL_PATH to direct where build-harness installs tools needed for the workflow. The quick fix is to replace INSTALL_PATH with PACKAGES_INSTALL_PATH in the workflow where the intent is to have build-harness install tools in a particular directory.

@Erik Osterman (Cloud Posse) @Andriy Knysh (Cloud Posse) FYI, for about 22 hours yesterday, make packages/install was broken and either errored out (if run with /bin/sh) or claimed the package was no longer supported. ☝️ Anyone reporting such errors during that time period should simply be advised to try it again.

did something change with make packages/install/terraform-docs?

@Jeremy G has joined the channel

Hi!
I was about to build some Spacelift images using docker targets from build-harness.
It all works locally. However, trying to setup GitLab CI/CD Pipeline and first thing that I learn is that we don't have GitLab Runner with shell executor. It is also not supported path (internally in company) to have EC2 node hosting GitLab Runner and using bash as shell executor to build Docker images using docker.
Our GitLab Runner is Kubernetes runner, which means GitLab CI/CD jobs run in K8S pods. Since this necessitates to use Docker-in-Docker, the recommended approach for building images from Dockerfiles is to use kaniko. This happens also to be one of the recommended approaches by GitLab as described in their docs here.

I am curious if there was ever interest in adding kaniko targets to build-harness?

@here critical update - <http://git.io|git.io> deprecation 2022-04-29 means that build-harness links may stop working.

https://github.com/cloudposse/build-harness/issues/314

Hello, good Build Harness folks - I was wondering if there was any thought given to renaming the default branch from master to main in GitHub? We’re planning to move our build-harness-extensions project that way, so the thought occurred to check with the mothership. Thanks!

This message was deleted.

Haven’t seen that error but I think you are right! Maybe a newer version of awk warns

Anyone else get the following when running make commands (help is a good example)

Available targets:

awk: cmd. line:1: warning: regexp escape sequence `\_' is not a known regexp operator

GNU Awk 5.1.0, API: 3.0 (GNU MPFR 4.1.0, GNU MP 6.2.0)

Looks like the awk regex in has an escaped _ which I'm not sure is needed ( https://github.com/cloudposse/build-harness/blob/master/Makefile.helpers#L57 )

@awk '/^[a-zA-Z\_0-9%:\\\/-]+:/

I'm ecstatic about our new assert-set and assert-unset macros. They are now in entirely in native make so and report line numbers where variable now unset or set.

I'm stumped.  I have a little make file:

GO ?= $(shell which go 2>/dev/null)
GOEXT_TMP_DIR := $(shell pwd -P)/tmp

.PHONY: test
test:
	export GOPATH=$(GOEXT_TMP_DIR) && \
	$(GO) get -u <http://github.com/securego/gosec/cmd/gosec|github.com/securego/gosec/cmd/gosec>

This go get command is supposed to get the source files for this gosec tool, build it and install the binary to $GOPATH/bin.  It works fine:

romulus:cicd-test cbarlock$ make -f test.mak
export GOPATH=/Users/cbarlock/vscode/cicd-test/tmp && \
	/usr/local/go/bin/go get -u <http://github.com/securego/gosec/cmd/gosec|github.com/securego/gosec/cmd/gosec>
romulus:cicd-test cbarlock$ ls tmp
bin	src
romulus:cicd-test cbarlock$ ls tmp/bin
gosec

But when I run the same target as a build harness extension:

romulus:cicd-test cbarlock$ make test
export GOPATH=/Users/cbarlock/vscode/cicd-test/tmp && \
	/usr/local/go/bin/go get -u <http://github.com/securego/gosec/cmd/gosec|github.com/securego/gosec/cmd/gosec>
romulus:cicd-test cbarlock$ ls tmp
src

Only the source files are downloaded and the binary is not created.  Any ideas why this is happening?

Adding @U010XGY9B46 bot

This is probably why we don’t see it

Consider using a Docker container instead for build-harness?

@David Schmidt makes sense, but first we have heard of it.

We have a number of teams (10-ish) running a number of PR builds daily through Travis - all of which are hitting github for the build-harness as well as a repo of extensions we built. I’m finding that we hit rate limiting pretty easily if we do unauthenticated curls to get the build-harness bootstrapped unless we use a github user and token. Since this differs from the nominal bootstrapping process, I wonder if others have hit a similar situation?

@UUB28NLDS help keep tabs! 😉

your workaround is good for now! 😃

@Andrew Mackett seems like we have some bug in build-harness - others too have had problems with it not installing gomplate or not being installed in the right place

The problem was gomplate not being found. I thought that was being installed in the make readme/deps step but perhaps I have an issue with my $GOPATH or something like that.

Anyway, using brew install gomplate fixed the situation and make readme worked fine after that. Thanks for the help!

@Andriy Knysh (Cloud Posse) is...

(also, we're working on a GitHub action to eliminate this!)

can you try make readme/init

Hi all. I’m having an issue when running make readme for a PR on cloudposse/terraform-aws-vpc-flow-logs-s3-bucket

make readme
curl --retry 3 --retry-delay 5 --fail -sSL -o /Users/andrewmackett/git/terraform-aws-vpc-flow-logs-s3-bucket/build-harness/vendor/terraform-docs <https://github.com/segmentio/terraform-docs/releases/download/v0.4.5/terraform-docs-v0.4.5-darwin-amd64> && chmod +x /Users/andrewmackett/git/terraform-aws-vpc-flow-logs-s3-bucket/build-harness/vendor/terraform-docs
make: gomplate: No such file or directory
make: *** [readme/build] Error 1

Any suggestions?

More detail here:
https://github.com/cloudposse/terraform-aws-vpc-flow-logs-s3-bucket/pull/4#issuecomment-575714601

I need to update our package for that

Awesome!

I have successfully tried v0.8-rc.2 . Works pretty well. Seems to handle every case out-of-the-box that came up as GitHub issues with the awk version.

v0.8.0-rc.1 does (also haven't tested yet)

(i haven't tested it yet)

@cytopia doesn't the official terraform-docs now support HCl2?

@Erik Osterman (Cloud Posse) there's another commit added to the terraform-docs PR which handles multiline description heredoc text.

No, or at least not maintained. We use #helmfile for deploying charts

Hi all, do you have in build-harness any switch to deploy helm charts?

@deftunix has joined the channel

@Can Kutlu Kinay has joined the channel

@curious deviant has joined the channel

@Tertius Stander has joined the channel

@Eugene has joined the channel

@Kostis (Codefresh) has joined the channel

@Taras has joined the channel

@Roderik van der Veer has joined the channel

@Taylor has joined the channel

and yes, I tried it many times, works (with TF 0.12 binary and 0.12 version of the module, master branch)

yes try in geodesic

I am guessing if I run geodesic, it would be more likely to work as well, right?