$ cat post/ping-with-no-reply-/-we-kept-it-running-on-hope-/-the-stack-still-traces.md

ping with no reply / we kept it running on hope / the stack still traces


Title: August 30, 2021 - A Mosaic of Complexity


August 30, 2021 marks another day on the long march of technology and ops. It’s been a year where the lines between platform engineering, SRE, and DevOps have blurred even further, all while dealing with the lingering effects of a global pandemic that has forced us to adapt our infrastructure for remote-first workforces.

I woke up this morning to a cacophony of buzzwords—eBPF, ArgoCD, Flux, and Backstage. They’re like new flavors of ice cream in an already crowded grocery store; everyone’s talking about them but it can be hard to tell which ones you need. Meanwhile, Kubernetes is still the base flavor that everything else builds upon.

This morning I was debugging a cluster issue with one of our staging environments where eBPF wasn’t behaving as expected. It’s those little moments when all the layers of complexity come crashing down around your ears and you have to remember why you started down this path in the first place. It’s not just about writing code, but also making sure that the systems we build are robust enough to handle real-world scenarios.

Speaking of real-world scenarios, I was arguing with a colleague about how best to implement a GitOps solution for our application deployments. We’re trying to move away from the manual chaos of SSHing into nodes and tweaking configurations, but every step forward feels like it requires more tools and processes. Flux is maturing nicely, which is good, but as always, we end up in a loop of “should I use this or that” and “how do we tie everything together without making a mess?”

The other day, the team had an interesting discussion about whether or not to move our internal developer portal from Backstage. Backstage is fantastic for giving developers quick access to documentation and tools, but it also comes with its own set of complexities. The debate was heated—some argued that Backstage’s complexity would slow down development, while others saw it as a necessary evil for maintaining a cohesive platform.

It’s moments like these that remind me why I love this job. You never know what you’re going to find when you dive into the codebase or start a new project. Today, our internal metrics dashboard was showing an unusual spike in traffic to one of our services. After some digging, it turned out to be a caching issue—something we’d been meaning to address but had put off for too long.

Speaking of which, I’ve been thinking about climate change and the tech industry’s carbon footprint as I read through reports on how critical this issue is. It’s easy to get caught up in the daily minutiae, but every now and then, a reminder like the IPCC report hits home. We’re part of an industry that has immense power and responsibility to lead by example.

Apple’s privacy policies have been making headlines again—this time, their plan to “think different” about encryption is causing some ripples. It’s interesting how these decisions impact not only our technology but also the broader societal implications. As developers and ops engineers, we’re often at the mercy of these large companies’ choices, but they also give us opportunities to rethink things.

In this era of complexity fatigue with Kubernetes, it’s refreshing to see tools like ArgoCD and Flux gaining traction. They bring a level of automation that can make our lives easier, even if the initial setup is daunting. And eBPF—well, it’s still too early to tell what its long-term impact will be, but I’m intrigued by the potential.

As I close out this day, I find myself reflecting on how much has changed in just a few years. The tools we use and the way we think about infrastructure are constantly evolving. It’s both exciting and overwhelming at times, but that’s why I keep coming back to this field. There’s always something new to learn, and every problem is an opportunity for growth.

Until next time—here’s to more complexity, more learning, and hopefully fewer cache misses!


That’s my take on a typical day in August 2021. Let me know what you think!