September 25, 2023. I woke up to another morning of chatbot updates, finops reports, and DevOps meetings. The AI arms race is in full swing, with every tech company trying to outdo the next with bigger models and faster training times. LLMs have become a mainstream concern for platform engineering teams like mine.

One of our biggest challenges this month was dealing with the infrastructure demands of hosting large language models (LLMs) in-house. We’ve seen a significant increase in compute power required, and our team has been wrestling with optimizing resource allocation to handle these requests without breaking the budget. The shift from cloud-hosted services to self-managed LLMs is both exciting and nerve-wracking.

Today, we spent most of the meeting discussing how to use WebAssembly for server-side tasks. It’s still early days, but the potential to bring more of our client-side technologies into a server environment is tantalizing. I find myself toggling between excitement about the possibilities and trepidation over the complexity it introduces. Will we end up with another layer that just complicates things? Only time will tell.

One argument today was about whether we should invest in Kubernetes-native tools or stick with more traditional DevOps practices. The jury is still out, but one thing is clear: the landscape of platform engineering is shifting rapidly. We need to be agile and willing to experiment, even if that means overengineering some solutions temporarily.

But amidst all this tech wizardry, something rather mundane happened today—a Raspberry Pi 5 arrived in the office. It’s always a treat to play with new hardware, but this time it sparked some thoughts on developer experience and the balance between innovation and practicality. While the new model is undoubtedly faster and more capable, we need to ensure that our teams can actually use these tools effectively without creating unnecessary complexity.

On another front, I’ve been grappling with a particularly stubborn issue related to WiFi connectivity. A few weeks ago, during a team meeting on an airplane (yes, one of those Southwest flights mentioned in the HN), we were hit by a lack of internet access. This got me thinking about how infrastructure and developer experience are intertwined. Sometimes the most basic services can have the biggest impact on productivity.

Speaking of WiFi, our internal network experienced some flaky behavior this week, causing occasional connectivity issues for team members working remotely. We had to work through multiple layers of diagnostics to understand why certain devices were intermittently losing connection. It turns out it was a simple issue—a bad cable in one of the office’s corners. A reminder that even with all our fancy tools and automation, sometimes the solution is as basic as tightening a nut.

Another fun challenge came up when we had a rare outage on our internal tracking system, which led to some minor chaos until we quickly debugged it. These small-scale issues remind us why we need robust monitoring and logging in place—no matter how polished your platform might be, real-world use cases will always throw curveballs.

As I wrap up my day, reflecting on the past month, I’m struck by how much has changed and yet how little. The tech world is bustling with new ideas and tools, but at its core, it’s still about solving problems, often in messy, unpredictable ways. Whether it’s managing a major LLM infrastructure or just tightening a loose cable, we keep pushing forward.

Until next time,

Brandon