Matt Horan's Blog


It's Always DNS

dns monitoring gke coredns kubernetes grafana

There’s a joke among sysadmins: it’s always DNS. DNS is an integral part of the modern internet. It often works and therefore nobody ever thinks about it. But when it doesn’t work, weird things happen — and nobody ever thinks about DNS. Even when they know it’s always DNS. Recently I had my own “it’s always DNS” moment, and I figured I’d share my experience here so that maybe you’ll remember: it’s always DNS.


Blackbox Monitoring with Prometheus

blackbox-monitoring monitoring prometheus google-cloud grafana gke

Prior to migrating from Cacti to Prometheus for infrastructure monitoring, I’d already been using Prometheus for blackbox monitoring. A couple of years ago I was looking for a way to monitor the health of various services I had deployed across virtual machines and containers running on my home network. I had used Pingdom for this in the past, but they killed their free plan in 2019. I had quite a few services to monitor, including multiple Web servers, a mail server, IRC server, and more. I surveyed the hosted service landscape but the available free options didn’t support the variety of services I needed to monitor; and the paid services cost as much as a single VPS at ARP Networks.


Migrating from Cacti to (Google Managed) Prometheus

cacti prometheus google-cloud monitoring grafana gke

I’ve been using Cacti to monitor infrastructure for nearly 20 years. Cacti is a great tool for polling devices via SNMP and renders pretty graphs via RRDtool. However, I’ve been trying to consolidate infrastructure and have settled on Kubernetes (specifically, Google Kubernetes Engine) for running as much of my infrastructure as possible. While it’d be possible to run Cacti on Kubernetes, I wanted to find another option that was more “Kubernetes native”.


1 of 1