$ cat post/netstat-minus-tulpn-/-i-typed-it-and-watched-it-burn-/-i-still-have-the-diff.md

netstat minus tulpn / I typed it and watched it burn / I still have the diff


Title: The IoT Commissioner’s Dilemma: Security Updates and Platform Engineering


September 4, 2023. Another day in the life of a platform engineer turned engineering manager. I woke up with the news that FCC Commissioner Jessica Rosenworcel had proposed regulating security updates for Internet of Things (IoT) devices. It’s an interesting move, given the recent explosion of AI/ML infrastructure and the ongoing discussions around platform engineering and FinOps.

The past few months have been a whirlwind. ChatGPT sparked a whole new era in LLMs, and suddenly everyone was talking about AI models as if they were the next big thing. CNCF’s landscape has become overwhelming—every month brings more Kubernetes distributions, service meshes, and observability tools. WebAssembly is finally starting to gain traction on the server side, and developer experience (DX) has become a real discipline with its own conferences and best practices.

But amidst all this, IoT security remains a thorny issue. Last week, I spent a fair amount of time digging into an IoT-related security incident that affected one of our customers. It was a classic case of missing security patches due to outdated firmware. Our infrastructure team had already been pushing for better patch management and automated update systems, but integrating this into the existing platform was proving more challenging than we anticipated.

The Patch Management Dilemma

One thing I learned from this experience is just how complex IoT security really is. Unlike traditional server environments, which are often well-managed by system administrators, IoT devices can span multiple manufacturers and software versions. Ensuring that all these components stay up to date requires a robust update strategy.

We started by implementing a script to monitor for known vulnerabilities in our internal devices and services. We also set up alerts for when new patches were released so we could apply them promptly. However, this approach has its limitations. For one thing, not every IoT device can handle frequent updates without causing performance issues. Additionally, there’s the challenge of rolling out updates to thousands of disparate devices that might be spread across different networks or even countries.

Developer Experience and FinOps

As we worked on these infrastructure improvements, I couldn’t help but think about how they fit into the broader landscape of platform engineering. The goal is always to create a reliable, maintainable system while keeping an eye on costs. That’s where FinOps comes in—balancing development needs with financial constraints.

One of the projects I’ve been working on recently involves optimizing our microservices architecture to reduce serverless function execution times and minimize cold start latency. This has required deep dives into various cloud providers’ billing models, understanding how different deployment strategies affect costs, and finding the right balance between performance and budget.

Reflections on DORA Metrics

Speaking of cost optimization, I’ve been reading up on DevOps Research & Assessment (DORA) metrics, which have become widely adopted in recent years. These metrics are great for measuring the efficiency and effectiveness of development teams, but they can also serve as a guidepost for making informed decisions about how to allocate resources.

For example, if one of our services is consistently rated low on lead time, it might indicate that there’s an issue with our CI/CD pipeline or our automated testing setup. We could then focus efforts on improving those areas rather than constantly chasing the latest tech trends.

The Tyranny of the Marginal User

And speaking of efficiency, I came across an interesting article titled “The Tyranny of the Marginal User.” It’s a classic piece that reminds us to consider the broader implications of our technical decisions. In my experience, this often means thinking beyond just the most common use cases and ensuring that our systems can handle edge scenarios.

For instance, in one project, we had to design an API that could handle requests from both high-traffic websites and occasional mobile apps. The key was finding a balance between performance under heavy load and robustness when dealing with infrequent bursts of traffic.

Conclusion

As I write this, I’m reflecting on the past few months. IoT security is just one piece of the puzzle in our ever-evolving tech landscape. Platform engineering, DX, FinOps—these are all important disciplines that intersect to create a more resilient and efficient infrastructure.

For now, I’ll continue pushing for better patch management practices and automated updates while keeping an eye on how these changes impact overall costs. And who knows? Maybe one day IoT devices will have the same level of security monitoring as our servers do today.

Until then, it’s just another day in the life of a platform engineer.