Debugging the Real World with WebAssembly

---

January 1st, 2024. Another year on the books. I’m sitting at my desk, staring out a window that’s been covered in snowflakes and frost for weeks now. It feels like the world is just starting to catch up to where it was last January, when AI infrastructure exploded post-ChatGPT and platform engineering became mainstream.

I’ve spent much of the past year working on our web applications, specifically trying to integrate WebAssembly (Wasm) into our server-side architecture. The idea of running compiled code directly in browsers has been around for a while, but with the rise of platforms like Cloudflare Workers and Vercel’s Cloud Functions, it’s finally becoming more practical.

The Journey So Far

We started small—adding some Wasm modules to handle image processing tasks, like resizing or applying filters. It was straightforward enough that we didn’t have any major issues at first. But as we began to integrate Wasm into more critical parts of our platform, things got tricky.

One particular issue I still remember vividly: memory management in Wasm. The garbage collector is different from what we’re used to with Node.js or Go. It’s not as simple to allocate and free up memory, especially when dealing with large datasets. We had a lot of crashes because our Rust-based Wasm modules were running out of heap space. I spent days tracking down the leaks, profiling, and adjusting our code.

It was frustrating, but also educational. Debugging memory issues in Wasm required a different mindset compared to traditional server-side programming. The experience taught me that WebAssembly is not just another layer to bolt onto your application; it’s fundamentally different and comes with its own set of challenges.

Platform Engineering and FinOps

As platform engineering became mainstream, I found myself spending more time on developer experience (DevEx) than ever before. Our goal was to make sure developers could quickly spin up new services without getting stuck in the weeds. But that meant we had to balance ease-of-use with performance and cost.

FinOps, or financial operations, became a significant focus for us. We started using DORA metrics to track our deployment frequency, lead time, mean time to recovery (MTTR), and change failure rate. It was clear that if we didn’t optimize for these metrics, our costs would skyrocket in the cloud.

One of the biggest lessons I learned is that you can’t just throw money at infrastructure issues. We had to be smart about it. We started using cost optimization tools like AWS Budgets and Azure Cost Management to monitor spending closely. This required us to rethink how we were deploying and scaling services, ensuring they only consumed what was necessary.

The LLM Infrastructure Boom

Of course, the LLM (large language model) infrastructure explosion didn’t go unnoticed. ChatGPT had been all the rage last year, and now everyone was talking about it. Our team started exploring ways to integrate smaller LLMs into our applications—things like using them for auto-completion in our editor Zed.

But as we dove deeper, we realized that these models were resource-intensive. Running a few of them on our servers quickly blew through our budget. We had to find a way to optimize or even offload some of the processing. This led us down the path of experimenting with edge computing and serverless functions.

Lessons Learned

Looking back, I’m proud of what we accomplished, but also humbled by how much there is to learn. The tech landscape is changing so quickly; it’s hard to keep up sometimes. But that’s why I love it so much—there are always new challenges and opportunities to grow.

As 2024 begins, I’m excited about where our platform will go next. WebAssembly is just one piece of the puzzle, but it’s a fascinating one. And with DevEx becoming more established, we’re better equipped than ever to deliver quality software quickly and cost-effectively.

So here’s to another year of learning, debugging, and pushing boundaries. Here’s to the challenges that come with building platforms in 2024 and beyond.

---

That’s my journal entry for the new year. Wishing you all a fantastic 2024 filled with new technologies and exciting adventures!