$ cat post/the-firewall-dropped-it-/-the-queue-backed-up-in-silence-/-we-kept-the-old-flag.md

the firewall dropped it / the queue backed up in silence / we kept the old flag


October 29, 2018: Kubernetes Wars and Platform Engineering

Today I find myself reflecting on the week that was in tech as we approached mid-October of 2018. Kubernetes had already claimed its throne in the container wars, but the battle for dominance wasn’t over yet—tools like Helm, Istio, and Envoy were emerging as key players in the ecosystem. The serverless landscape was buzzing with excitement, and Terraform’s 0.x version was still a work in progress. Meanwhile, GitOps was starting to take root, and Prometheus + Grafana had become the new standard for monitoring.

I spent much of my week trying to untangle some kubernetes issues at work. We were moving our microservices architecture over to Kubernetes, and I found myself on the front lines of a battle between the cluster’s stability and our application’s resilience. One particular evening, as I sat in front of my computer, I wrestled with a stubborn issue that was causing our services to crash right after they started up. It turned out to be a tricky DNS resolution problem that we hadn’t encountered before.

The struggle felt personal; it wasn’t just about fixing the service but ensuring our team could trust Kubernetes as their deployment platform. We argued internally about whether we should stick with Helm or switch to something else, weighing pros and cons in a heated discussion late into the night. In the end, I ended up spending hours diving deep into the Helm documentation, trying to understand why it wasn’t playing nicely with our network setup.

As the week wore on, other tech topics popped up like Paul Allen’s passing and IBM’s acquisition of Red Hat. These events, while significant, felt distant compared to the day-to-day work we were doing. I couldn’t help but chuckle at HN’s “Paper Airplane Designs” and “iPhones are allergic to helium,” which were just minor distractions from my work.

But then, the news about Facebook users deleting their accounts en masse hit me harder than any of those other stories. It was a stark reminder that while we obsess over technical challenges, there’s an entire user base relying on us to deliver a good experience. That night, as I sat debugging Kubernetes, I couldn’t shake off the thought that our decisions had real-world consequences.

As October 29th rolled around, I found myself reflecting not just on the day’s events but on the broader landscape of platform engineering. The term “GitOps” was starting to gain traction, and I began to see its potential in streamlining our deployment processes. At the same time, I couldn’t ignore the importance of traditional DevOps practices that kept our systems up and running.

The week left me feeling both accomplished and slightly overwhelmed. We had made progress on Kubernetes but still had a long way to go. As I typed my final lines for the day, I thought about how much technology can change in just a few years and the responsibility we have to keep pace with those changes while ensuring our users are happy.

So here’s to another chapter in platform engineering: let’s build better systems together.