$ cat post/ping-with-no-reply-/-we-shipped-it-on-a-friday-night-/-i-strace-the-memory.md

ping with no reply / we shipped it on a Friday night / I strace the memory


Title: The Linux Desktop Experiment That Almost Went Wrong


February 11, 2002 was just another day in the life of a sysadmin, except this time it seemed like everything I touched decided to go sideways. Back in those days, we were all feeling the aftershocks of Y2K and dealing with the fallout from the dot-com bubble bursting. Linux on the desktop was starting to gain traction—Red Hat 7.1 had just hit the shelves, and everyone was buzzing about it.

At my job at a mid-sized tech company, our IT department decided to take a shot at rolling out Linux as the default OS for new PCs. The reasoning behind it was clear: with so many free tools available, why not save money? We were all excited; after all, this was 2002 and who would have thought that open source could be mainstream?

We started small, just a few pilot machines in our development department. Everything seemed to work perfectly for the first few weeks—code editors like Vim and Emacs worked flawlessly, and tools like Apache and Sendmail were rock-solid. The sysadmin team was thrilled with how easy it was to set up and manage these systems compared to Windows 2000.

However, as we started rolling out more machines, issues began to surface. One of our senior developers came over to my desk one morning, looking visibly frustrated. “I can’t get my SSH keys working,” he said, waving a cryptic error message at me. I took a look and realized that the default .ssh/ directory permissions were set to 755 instead of 700, which meant his private key was accessible by others on the network.

“Nice catch,” I thought sarcastically as I worked through the issue, changing the permissions back. But this was just the beginning. Soon after, we started getting reports about users experiencing weird kernel panics and system hangs during bootup. It turned out that our custom-built init scripts were conflicting with the init process managed by System V.

I spent hours poring over logs and configuration files, trying to figure out what could be causing these crashes. The sheer number of processes involved in a modern Linux desktop was overwhelming—X11, GNOME, KDE, HAL, D-BUS… it felt like every single piece needed its own set of tweaks and configurations.

Worse still, our backup scripts were failing because they didn’t understand the new filesystem structures used by Ext3. As I sat there, staring at the error messages, I couldn’t help but feel a mix of excitement and frustration. We were doing something no one had done before in this company—building an entire Linux desktop environment from scratch.

But as the weeks went by, more problems kept cropping up. Users complained about network connectivity issues and printers not working properly. And then there was that one user who reported that his mouse would randomly stop working for a few seconds every hour. After hours of debugging, it turned out to be an issue with HAL’s daemon failing due to some obscure kernel module conflict.

At times like these, I couldn’t help but wonder if this experiment was worth the effort. But then again, how often do you get a chance to be part of something so new and uncharted? We were pushing the boundaries, trying to make Linux work in an environment where it had never been used before.

In the end, we managed to iron out most of these issues, but not without a few sleepless nights. The experience taught me valuable lessons about the importance of thorough testing and understanding all the moving parts involved in setting up a modern operating system. We eventually succeeded in making Linux work on our machines, although it was definitely a bumpy ride.

Looking back now, those days seem like a distant memory from another era—before cloud computing, before containerization, even before virtualization had truly taken off. But then again, that’s what makes these experiences so valuable. They’re not just about the technology; they’re about learning and growing as an engineer in a rapidly changing landscape.


The roll-out of Linux on our desktops was an experiment, and it almost went wrong. But through the struggles and successes, I gained invaluable knowledge that has stayed with me over the years. Those days taught me to be more patient and thorough when embarking on big changes, no matter how exciting they might seem at first.