November 2017 was an exciting time to be in tech, with Kubernetes seemingly everywhere. The container wars were being won, and we saw the rise of new tools like Helm and Istio. Meanwhile, serverless was gaining momentum, but I was more focused on the container orchestration battle.

I had just started working at a company where Kubernetes had been deployed across multiple teams. One of my primary tasks as an engineer was to help set up a CI/CD pipeline for our services, ensuring they could be seamlessly moved in and out of the cluster. It was a mix of excitement and anxiety, as I dove into the world of YAML files and kubectl commands.

The first major issue we faced wasn’t with Kubernetes itself, but with Helm. We decided to use Helm to manage our application deployments, which seemed like a smart choice at the time. But soon after setting up the basic templates, we started running into permission issues and misconfigurations. It was frustrating to see pods failing repeatedly because of things as simple as missing paths in a chart.

I spent many late nights wrestling with Helm’s templating engine, trying to get it to play nice with our existing infrastructure. It felt like I was back in the days of dealing with complex bash scripts, but this time, everything had to be perfect for 100 pods running across multiple nodes.

At one point, I nearly gave up on Helm and considered rolling my own templating solution. But then a colleague mentioned Kubernetes’ own native templating support via annotations. This opened up a new avenue, allowing us to manage our deployments in a more native way, reducing the complexity of our Helm charts.

The next challenge came when we started running into networking issues. Our services needed to talk to each other, and with Istio on the rise, it seemed like a natural fit for service mesh. However, getting Istio set up was a nightmare. The initial setup required manually configuring a plethora of resources in Kubernetes, and any misstep would lead to cascading failures.

One particularly memorable day, I spent hours trying to debug why one of our services wasn’t able to communicate with another. It turned out that the service mesh’s sidecar containers had been misconfigured due to a typo in the YAML file. After fixing it, everything worked as expected—a good reminder that even when using modern tools, basic human error can still be the culprit.

As we continued to work through these issues, I began to realize how much value Kubernetes was bringing to our development process. It wasn’t just about running containers; it was about enabling a new way of thinking and working together as a team. We started to see an improvement in deployment cycles and reliability, which made all the late nights worthwhile.

Looking back at that time, I’m grateful for the challenges we faced. They pushed us to learn more deeply about Kubernetes and related technologies. And while Helm and Istio have evolved significantly since then, they remain essential tools in our tech stack today.

Kubernetes truly was—and still is—the new king of container orchestration. As we continue to navigate the ever-evolving landscape of cloud-native technologies, I’m excited to see what the future holds.