#ops

ops

Archived
0149

Archive: https://archive.sweetops.com/ops/

erik
erikalmost 5 years ago
archived the channel
Gabe
Gabeover 6 years ago
Does anyone have experience with consul connect and envoy as the side proxy? I'm using ECS and have the following questions:

1. How do you register your services? We currently use registrator but it doesn't seem to be very well maintained and doesn't support consul connect. We have looked at docker-consul-envoy but it seems like we would need to generate a service configuration for each service which seems tedious.
2. Do you have to explicitly define the upstreams for each service? This seems like a maintenance nightmare.
3. How do you manage/define intentions which allow service A to talk to service B?
Here is an example configuration from a Hashicorp walk through as an example for the upstreams definition:

service {
  name = "dashboard"
  port = 9002

  connect {
    sidecar_service {
      proxy {
        upstreams = [
          {
            destination_name = "counting"
            local_bind_port  = 5000
          }
        ]
      }
    }
  }

  check {
    id       = "dashboard-check"
    http     = "<http://localhost:9002/health>"
    method   = "GET"
    interval = "1s"
    timeout  = "1s"
  }
}
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 6 years ago
Adding @U010XGY9B46 bot
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 6 years ago
@UUB28NLDS help keep tabs! 😉
Janis Peisenieks
Janis Peisenieksalmost 7 years ago
I guess this means that one of the core problems in my case is that I didn’t know where to look to see if the domain was being blacklisted.
Janis Peisenieks
Janis Peisenieksalmost 7 years ago
Janis Peisenieks
Janis Peisenieksalmost 7 years ago
@Andriy Knysh (Cloud Posse) Looks like blacklisting is the issue here. This is from Cisco/Talos’ site:
Rich Allen
Rich Allenalmost 7 years ago
Spinning up an aws fargate cluster + vpc and running this on a schedule, seems like way overkill. Is there a better on-demand approach here? Lambda would timeout.
Rich Allen
Rich Allenalmost 7 years ago
Question here. I'm trying to run the following on a cron like schedule.

docker run --rm -it -u $(id -u):$(id -g) linkchecker/linkchecker <https://somesite.com> -t 40 -o html > ourreport.html
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
Your system date and time are not correct could be the issue
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
Why net::err_cert_authority_invalid error occur?
Basically, there are two main reasons for receiving this kind of error:

The Certificate authority that had issued this certificate is not trusted or the certificate has expired
and *Your system date and time are not correct*.

Janis Peisenieks
Janis Peisenieksalmost 7 years ago
Anyways, thank you for your time and help!
Janis Peisenieks
Janis Peisenieksalmost 7 years ago
I just got a report from a user, and what they see is this.
This to me would indicate that something is trying to show something else than what is supposed to be there, since the certificate is issued by AWS, and is working for 99% of our other users.
Janis Peisenieks
Janis Peisenieksalmost 7 years ago
Ok, I’ll take a look at that, thanks!
Janis Peisenieks
Janis Peisenieksalmost 7 years ago
The thing is that I’ve never been able to replicate on my own, and for users it usually starts working if they switch networks from their work to another network (like a hotspot). It seems to be an all-or-nothing for some users. So either all of the things from the CDN load, or none do.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago(edited)
these are just some of the headers that could affect it
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
<https://medium.com/@tom.cook/edge-lambda-cloudfront-custom-headers-3d134a2c18a2>
<https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/default-src>
<https://content-security-policy.com>

Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
something should be diff there
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
take a look at headers and compare them on working and not working sites
Janis Peisenieks
Janis Peisenieksalmost 7 years ago
And the weird thing is that we always serve the user-content from CDN from our main domain. That part doesn’t change, which (if I understand CSP correctly) means that CSP shouldn’t be changing as well. At least we don’t change it in any way
Janis Peisenieks
Janis Peisenieksalmost 7 years ago
Yes
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
also, do you use SSL to load assets from your domain?
Janis Peisenieks
Janis Peisenieksalmost 7 years ago
hmm
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
so, since Content-Security-Policy could be diff on each site, the behavior could be different
Janis Peisenieks
Janis Peisenieksalmost 7 years ago
Which makes me think that it shouldn’t be Content-Security-Policy realted
Janis Peisenieks
Janis Peisenieksalmost 7 years ago
which makes me think that there is something wrong (or at least not right) with our domain
Janis Peisenieks
Janis Peisenieksalmost 7 years ago
The main problem here is that it’s nowhere near consistent, and seems to happen sporadically for some users, but usually quite a few
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
  'Content-Security-Policy': 'frame-src <https://aaaaaaa.com>; default-src \'none\'; img-src \'self\' data: blob: *; script-src \'self\' blob: <https://aaaaaaaa.org>; style-src \'unsafe-inline\'; object-src \'none\'; frame-ancestors \'self\'; connect-src \'self\' <https://aaaaaa.com/>; base-uri \'self\'; form-action \'self\'; manifest-src \'self\'');

Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
for example, take a look at Content-Security-Policy header, it can block many resources from loading on a site
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
it must be something else, e.g. headers, security settings, etc.
Andriy Knysh (Cloud Posse)
Andriy Knysh (Cloud Posse)almost 7 years ago
@Janis Peisenieks the domain name itself should not be suspicious to any enterprise firewalls unless it’s explicitly blocked (blacklisted)
Janis Peisenieks
Janis Peisenieksalmost 7 years ago
No, no, what I mean is that we have our own CDN (i.e. janis-user-content.com) that serves the content our users upload. Basically a user-upload sandbox domain to make sure it doesn’t share any cookies with our main domain. We then serve all user-content from that CDN/Domain inside of our app.
Well some Enterprise firewalls seem to not trust our user-content domain and sporadically block it.
I’m wondering if someone else has any tips/tricks on how to make the domain look… less suspicious?
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
...it's not intended to be used as a CDN
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
(for this you need a proxy)
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)almost 7 years ago
Do you mean like hotlinking to <http://raw.githubusercontent.com|raw.githubusercontent.com>?
Janis Peisenieks
Janis Peisenieksalmost 7 years ago
We have set ours up, but it looks like it’s being somehow magically throttled/blocked by some ISP’s/Enterprise firewalls and the like. And I’m curious whether someone knows how to go about making sure that a domain like that doesn’t get blocked as suspicious
Janis Peisenieks
Janis Peisenieksalmost 7 years ago
Hi everyone! This seems like the most appropriate channel to ask for help.
Does anyone have any experience setting up user-content domains? I.e. githubusercontent.com ?
ols
olsalmost 7 years ago
Interested in thoughts on the TSB outage report if anyone has read it yet: https://www.tsb.co.uk/news-releases/slaughter-and-may/
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)about 7 years ago
I don't have any context...
Sharanya
Sharanyaabout 7 years ago
Create Jenkinsfile to deploy UI code to S3 bucket.
Sharanya
Sharanyaabout 7 years ago
Did anyone Come across NPM memory Issues ?
Sharanya
Sharanyaover 7 years ago
Hey Folks, Trying to find some Terraform Modules related to AWS - app stream service ( for creating fleets and stacks) any help appreciated
Jonathan Le
Jonathan Leover 7 years ago(edited)
I didn't know where to post this, but if you're dealing with TLS restricted redis: https://www.compose.com/articles/redli-your-new-redis-command-line-client-from-compose/
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
no, but it reminds me of https://marbot.io/
Igor
Igorover 7 years ago
erik
erikover 7 years ago
set the channel description: Archive: https://archive.sweetops.com/ops/
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Changing secrets is a new release
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse)over 7 years ago
Yes, same.
Gonzalo Peci
Gonzalo Peciover 7 years ago
I believe I asked this before, but how do you guys run etcd/consul/etc config chabges or even secrets?

EG you have one or multiple secrets using config from etcd /someparam
You want to update it.

Most examples I see with confd and etc (even in code) means you potentially kill all services using that parameter if you the wrong one. Same applied to secrets.

We handle it by not hotreloading and just triggering a force deploy (roll/blue green even if no changes detected)