November 4th, 2024. It’s been a whirlwind month in tech, and I’ve found myself knee-deep in the realities of what this era has to offer. As someone who’s seen the rise and fall of countless frameworks and platforms over the years, this period feels like an explosion of innovation—mixed with a dose of reality checks.

AI/LLM Infrastructure: A Tale of Two Worlds

It’s been fascinating watching how AI has taken on new dimensions post-ChatGPT. The landscape is now filled with various large language models (LLMs), each vying for supremacy in different niches. From chatbots to code generation, the possibilities are endless. However, managing the infrastructure for these LLMs isn’t as glamorous as it seems. I spent a chunk of my week dealing with latency and resource allocation issues on our newly deployed Anthropic model. We had some early wins but also faced significant performance bottlenecks due to network congestion and overloaded GPUs.

Platform Engineering: It’s Mainstream, But Is It Worth It?

Platform engineering has truly taken off. Gone are the days where you could get away with just being a developer; now, you need to understand DevOps, CI/CD pipelines, and platform design. At my company, we’ve been exploring whether it’s worth creating yet another internal platform for various microservices. The argument for standardization is strong, but so is the counter-argument that custom solutions can lead to more efficient and tailored experiences. I found myself in a heated debate with a colleague who was pushing for a third-party SaaS solution. It’s a tough call when the cost of switching versus maintaining an internal platform is so high.

CNCF Landscape: Overwhelming but Necessary

The Cloud Native Computing Foundation (CNCF) landscape has grown to the point where it’s almost overwhelming. With Kubernetes at its core, there are now dozens of projects that layer on top, each promising something unique. We’re evaluating a new project called Starburst for managing our data pipelines. It’s an interesting tool, but as always, the learning curve is steep. I spent some time arguing against integrating it with our existing infrastructure; after all, we already have a working solution with a solid support team.

WebAssembly on Server Side: Another Nail in the JVM’s Coffin?

WebAssembly (Wasm) has been making waves as a server-side technology. I’ve been toying around with using Wasm for some edge computing tasks. It’s impressive how well it can integrate with Node.js and other JavaScript runtimes, but there are still issues with memory management and debugging that need to be addressed. For now, I’m keeping an eye on it, but I don’t think it will completely replace JVM anytime soon.

Developer Experience: A Discipline of Its Own

The focus on developer experience (DX) has shifted from just improving tooling to understanding the entire workflow. I’ve been working with our product team to streamline how they interact with our APIs. It’s not just about making the tools prettier; it’s about ensuring that every interaction is as seamless as possible. We’re using a new tool called RadixDlq for better error handling and logging, which has drastically improved our debugging process.

FinOps and Cloud Cost Pressure

FinOps continues to be a critical part of the infrastructure equation. With cloud providers raising costs and offering less generous discounts, we’ve had to get more creative with cost management. I recently set up an automated system that monitors our AWS bill daily and sends alerts when spending exceeds certain thresholds. It’s a small but effective way to keep costs in check.

DORA Metrics: Just the Beginning

Digital orchestration readiness assessment (DORA) metrics are now widely adopted, and it’s clear why. They provide a structured way to measure DevOps performance. We’re using them to track our release cadence, lead time for changes, and deployment frequency. It’s not just about hitting these numbers; it’s about understanding where we can improve.

The Finale: Staff+ Engineering Tracks

The normalization of staff+ engineering tracks has brought a new level of clarity in my company. I’m now working with both development and ops teams to define clear paths for career growth. We’re using DORA metrics to identify pain points and areas for improvement, ensuring that everyone is on the same page.

---

In this era, it’s all about finding balance between the latest technologies and real-world constraints. The tech landscape might be overwhelming, but the opportunities are endless. I’m looking forward to what November 5th brings, ready to dive into whatever challenges come my way.