$ cat post/a-race-condition-/-the-segfault-taught-me-the-most-/-it-ran-in-the-dark.md

a race condition / the segfault taught me the most / it ran in the dark


Title: Kubernetes on My Mind - A New Hope?


August 31, 2015

Today, I find myself reflecting on a time when containers were becoming mainstream and microservices were just starting to be discussed in earnest. It’s been exactly two years since Docker released its first version, and the tech world was abuzz with containerization talk. CoreOS had just come onto the scene, Kubernetes (then called Omega) was announced by Google, and everyone seemed to be building their own orchestration tools or adapting Mesos.

I remember sitting in a small conference room at work, discussing potential strategies for moving our services into containers. It felt like the future of DevOps was being written on the whiteboards around us—services breaking down, running in Docker containers, orchestrated with some form of Kubernetes or its clones.

The buzz about Google’s Kubernetes announcement had just started to pick up. There were rumors it would revolutionize how we deploy and manage our services, but no one knew exactly what that meant yet. We toyed with the idea of integrating Kubernetes into our existing infrastructure, but there was so much unknown—how stable was it? What are the learning curves for us and our ops team?

One day, a colleague came to me with an interesting project: building out our own container orchestration system. The idea excited me; I thought we could leverage Mesos and Marathon, which we were already using successfully for some of our microservices. But as I delved deeper into the project, I found myself questioning whether we should go all-in on something that was still in its early days.

The “Two Days Ago” hacker news story really hit home when someone from law enforcement approached a developer to halt work on their project. The implications were stark: while we worked on our own system, Kubernetes was moving fast and gathering momentum. It felt like the risk of sticking with our internal solution was too high—what if Google decided to take a different approach? Would we be left behind?

I found myself torn between the comfort of what we knew (Mesos/Marathon) and the uncertainty of adopting something new that could potentially disrupt everything we had been doing. The “Show HN: My SSH server knows who you are” story also resonated; I couldn’t help but wonder how our system would handle these kinds of security challenges as it scaled.

On a more personal note, there were nights when I lay awake worrying about whether we were making the right choice. Should we wait and see if Kubernetes matured or should we jump in now? The “It’s done, there is no way back” story kept me up thinking—what if our decision to stick with Mesos/Marathon turned out to be the wrong one?

But as the months passed, it became clear that Kubernetes was gaining traction. CoreOS and others were contributing significant features, and Google was actively pushing its development forward. The community around it was growing rapidly, and more and more companies started sharing their experiences.

In the end, I decided we needed to take a chance on Kubernetes. It wasn’t just about following the latest trend; it was about making sure our infrastructure could handle the future demands of our services. We began integrating Kubernetes into our system, and while there were growing pains—bugs, misconfigurations, and occasional outages—the long-term benefits outweighed the immediate challenges.

Looking back, adopting Kubernetes at that time was a defining moment for our team. It pushed us to adapt quickly and embrace change in our infrastructure. While we didn’t fully understand all the implications at first, the decision to move forward with Kubernetes ultimately paved the way for more efficient deployment and management of our services.

That’s my journey through those early days of Kubernetes. A mix of excitement, hesitation, and eventually, a commitment to move forward together. It’s a reminder that in tech, sometimes you just have to take a leap and trust that the community will help guide you along the way.


This blog post reflects on the real-world challenges and decisions made during the early adoption of Kubernetes, capturing the excitement and uncertainty of that time.