Title: A Day in the Life of a Platform Engineer: Kubernetes Complexity Fatigue

---

October 4, 2021 was just another day in the life of a platform engineer. I woke up to a cold New England morning and poured myself a cup of coffee, trying to shake off the remnants of the night before. The reality is that these days can blend into each other quite easily, but this particular morning felt different.

Last week, I had spent some time helping out a team with a Kubernetes deployment issue. They were running into problems with their stateful set, and after a couple of late nights, we finally managed to stabilize it. But the solution wasn’t pretty—long YAML files, convoluted CRDs, and a mix of Helm charts that didn’t really mesh well together.

As I sat down at my desk, I opened up a new Terminal window and fired up kubectl. The dashboard looked familiar yet slightly overwhelming. Kubernetes is always evolving, and keeping up with all the latest changes can be daunting. Today, though, we were dealing with a more mundane issue: persistent volume claims not being created properly.

I started by running a couple of commands to check the state of our deployments:

kubectl get pods -n mynamespace

It was clear that something wasn’t right—too many pods in Failed state. I switched over to the logs to see what was going on:

kubectl describe pod <failed-pod-name> -n mynamespace

The log output showed a few cryptic error messages related to persistent volume claims. This is where things got interesting. Persistent Volume Claims (PVCs) and Persistent Volumes (PVs) can be tricky, especially in a multi-namespace environment like we have here.

I decided to break down the problem into smaller pieces:

1. Check if PVs are being created properly: kubectl get pv -n mynamespace
2. Ensure PVCs are correctly referencing the PVs: kubectl get pvc -n mynamespace
3. Inspect storage classes and volume bindings: kubectl describe sc

After a few iterations, I realized that one of our storage classes was misconfigured. We had set up multiple storage classes with slightly different parameters, and it seems we didn’t have the right combination for this particular deployment.

Fixing the issue involved making some changes to the storage class definition:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: my-storage-class
parameters:
  type: gp2
provisioner: kubernetes.io/aws-ebs

Once I made these adjustments and redeployed, everything started falling into place. It was a bit of a relief to see the pods start coming back up without any issues.

This incident brought home the complexity fatigue that many in our industry are experiencing right now. While Kubernetes is undoubtedly powerful, it can be overwhelming when you have multiple namespaces, different storage classes, and complex stateful workloads all in one place.

I’m not alone in feeling this way either. The recent flurry of articles about “Kubernetes Complexity Fatigue” resonated with me. It’s a reminder that while these tools are essential for our modern infrastructure, they require constant attention to detail and careful management.

On the flip side, it was also reassuring to see technologies like Backstage starting to gain traction. Internal developer portals help us manage all of this complexity more effectively by providing visibility into our tech stack and workflows. As I continue to work through today’s tasks, I’ll be looking for opportunities to integrate some Backstage features to streamline our processes.

As the day goes on, I expect there will be other issues to tackle, but at least now I know I can rely on my experience and tools like Kubernetes to get us back on track. Here’s hoping the rest of the week isn’t quite as eventful!

---

That was a typical morning for me—full of debugging, learning, and occasionally feeling like we’re swimming against a tide of complexity. But that’s why we do this, right? To make sense of it all and keep pushing forward.