Like many network engineers, I quickly fell in love with my field and worked hard to master it. I got into networking when I was working in desktop support. The behind-the-scenes stuff was way more interesting to me than the front lines. Back in the late nineties, I bought a library of books to learn this field. Perlman, Comer, and Stevens were the classics. I rounded it out with blue-and-white Cisco Press books by Doyle, Peplnjak, Williamson, and many others. I studied these books religiously, read through config guides, and practiced in the lab.
The network engineers on my team and I loved to debate the arcana of this mysterious field. We always tried to one-up each other, learning new technologies, new protocols, and attaining new technical certifications. I’ve worked with engineers who are smarter than I am, and better than I am, but that always motivated me to learn more.
I bring this up because I’ve had multiple conversations with multiple execs, for many years, in which they seem to decry the virtue of expertise. Network engineers “revel in complexity”, they don’t realize their time has passed, the build networks that need “armies of CCIEs to maintain”, and they hate simplicity. If only the pesky network engineers would get out of the way, the glorious MBAs could build us simple and elegant products, which is how the industry is going, don’t you know!
In short, our industry is suffering through a war on expertise. Those arcana we love to master have put a target on our back. If you want to learn those things, you must be reveling in complexity. Go find something else to do, ChatGPT will replace you!
The first mistake in this line of thinking is the assumption that network engineers want to build networks that are complex. We actually don’t. A couple of anecdotes:
When I was working for a Gold partner, I was sent to help out an IT manager of a rather small company, only four sites. She had contracted VPLS from two service providers, and asked me to implement a complex load balancing scheme she had conceived. I begged her not to make me do it, but she insisted. I ended up building a functional mess, a combination of PBR and EIGRP offset access-lists. Man, was it ugly. But it worked. Then I got laid off from the partner, and a year later she was calling me, begging me to come back because nobody could figure out how it worked. I didn’t want to build something that ugly and she didn’t need it.
Second anecdote. My wife had to go in for a surgical procedure a few years ago. We went to the best doctor in San Francisco. When he got into the procedure he found that her anatomy is not conventional, and it was a very difficult procedure. In the recovery room, he told us most doctors would have stopped. My wife wanly smiled and said, “well, I’m sure you like a challenge.” He looked back at her and said, “no, we like it when it’s easy.”
I think this is where the execs misunderstand expertise. 99.9% of the time, your airline flight could be handled by a low-time pilot who can work the automation systems. But when the engines fail, you want Sully at the controls. Just because some people understand complexity and study difficult concepts, it doesn’t follow that they want complexity. When I administered networks, I wanted it to be easy. But I was ready for when it was hard.
The war on expertise seems to me to be a war on the human spirit. The CCIE exam, whatever you think of it, was a heck of a challenge, and passing it was one of the proudest days of my life. Human beings want to learn, to grow, to push their limits, and to test themselves. That’s why we spend hours in the lab. We should encourage this behavior. We should want people in our business who seek subject matter expertise and mastery. We can make things simpler, fine, but we should still encourage expertise.
At the end of the day, networks are inherently complex. A network is a large distributed system, connecting numerous devices running numerous operating systems over diverse transport mechanisms using a wide variety of protocols. You can simplify the protocols a bit, but ultimately most simplification of networks is done one of two ways: reducing the number of choices an administrator can configure, or abstracting and hiding the underlying complexity. In the first case, you may close out necessary use cases. In the case of abstraction, well, it works great until something breaks. Then you need to call a network engineer.
I’m not in any way saying the new tools, from programmability to automation systems like Ansible, to “controllers” are unnecessary. Far be it. Any tool that makes an engineer’s job easier will be embraced by engineers. I am saying that we need to stop blaming complexity on those who manage to understand it.