$ cat post/the-prod-deploy-froze-/-old-servers-never-forget-/-it-was-in-the-logs.md

the prod deploy froze / old servers never forget / it was in the logs


Title: Config Management Wars: A Year in Review


2012 was a year of intense activity in the realm of configuration management tools. I remember the debates and arguments well, as we were deep into the heart of the “Puppet vs Chef” wars. It wasn’t just about which tool to use; it was about whether or not you even needed one.

The backdrop of this fight was set by the emerging DevOps movement, where the lines between development and operations were blurring at a rapid pace. We were looking for better ways to manage our infrastructure, and configuration management tools seemed like the perfect solution.

At the start of 2012, Puppet and Chef had become the frontrunners in the battle. Both claimed to offer superior solutions, but the truth was that they each had their strengths and weaknesses. Puppet’s declarative approach felt more elegant, while Chef offered a procedural model that made it easier for developers who were used to writing code.

The problem we faced at work was simple: our infrastructure was growing rapidly, and we needed a tool that could scale with us. We had a few projects running on both Puppet and Chef, and the constant switching between tools was causing more headaches than it was solving.

As an engineer, I found myself spending a lot of time in these debates. Should we stick with Puppet or move to Chef? Would one ever become dominant, or would they continue coexisting? The discussions often devolved into technical arguments that felt more like religious wars than rational decision-making processes.

During this period, the Netflix Chaos Engineering initiative was making waves. Their approach to building resilient systems by intentionally injecting faults and failures into their infrastructure inspired us to think about our own practices. We started experimenting with these ideas within our own environment, which helped us identify weak points that we hadn’t considered before.

Meanwhile, OpenStack’s launch added another layer of complexity to the mix. As a cloud provider, it was clear that we needed tools that could handle both our on-prem and cloud infrastructure. This led to some interesting discussions about whether we should standardize on one tool or continue using a hybrid approach.

By December 2012, the continuous delivery book had just been published, and it became evident that this philosophy of delivering code frequently and reliably was becoming more widely accepted. The tools used for configuration management were becoming integral to our ability to deliver value quickly and consistently.

In the end, after months of debate and experimentation, we decided to stick with Puppet but began integrating some Chef features. This hybrid approach allowed us to leverage the best aspects of both tools without completely switching horses mid-stream. It wasn’t perfect, but it was a pragmatic solution that helped us move forward.

Looking back, 2012 felt like the year when configuration management tools really came into their own. The debates and arguments may have been loud and sometimes heated, but they also pushed us to think deeply about our processes and how we could make our infrastructure more robust and scalable.

As we moved into 2013, I was excited for what the next year would bring. The journey from that point onward has only continued to evolve as new tools and techniques emerged, making it a truly dynamic time in our industry.