Skip navigation

Tag Archives: mba

Back in the lab

We all have to make a decision, at some point in our career, about whether or not we get into the management track.  At Cisco, there is a very strong path for individual contributors (IC).  You can be come a principal (director-level), a distinguished (senior director-level), and a fellow (VP-level) as an IC, never having to manage a soul.  When I married my wife, I told her:  “Never expect me to get into management, I’m a technical guy and I love being a technical guy, and I have zero interest in managing people.”

Thus, I surprised myself back in 2016 when my boss asked me, out of the blue, to step into management and I said yes.  Partly it was my love of the Technical Marketing Engineer role, partly my desire to have some authority behind my ideas.  At one point my team grew to fifty TMEs.

All technical people know that, when you go that route, your technical skills will atrophy as you will have less and less hands-on experience.  This is very true.  In the first couple of years, I kept up my formidible lab, then over time it sat in Cisco building 23, unused and consuming OpEx.  I almost surrendered it numerous times.

Through attrition and corporate shenanigans, my team is considerably smaller (25 or so) and run by a very strong management team.  Last week, I decided to bring the lab back up.  I’ve been spending a lot of time sorting through servers and devices, figuring out which to scrap and which to keep.  (Many of my old servers require Flash to access the CIMC, which is not feasible going forward.)  I haven’t used ESXi in years, and finding out I can now access vSphere in a browser–from my Mac!!–was a pleasant surprise.  Getting CIMCs upgraded, ESXi installed, and a functional Ubuntu server was a bit of a pain, but this is oddly the sort of pain I miss.

I have several Cat 9k switches in my lab, but I installed Cisco Modeling Labs on one of my servers.  (The nice thing about working for Cisco is the license is free.)  I used VIRL many years ago, which I absolutely hated.  CML is quite slick.  It was simple to install, and within a short time I had a lab up and running with a CSR1kv, a Cat 8k, and a virtual Cat 9k.

When I was in TAC I discovered IOS on Unix, or IOU.  Back then, TAC agents were each given a Sun Sparc station, and I used mine almost exclusively to run IOU.  (I thought it was so cool back then to have a Sun box on my desk.  And those of you who remember them will know what I mean when I say I miss the keyboard.)  IOU allowed me to define a topology in a text file, and then spin up several virtual IOS devices on the Sparc station in that topology.  It only supported sinulated Ethernet links, but for pure routing protocols cases, IOU was more than adequate to recreate a customer environment.  In 15 minutes I could have my recreate up and running.  Other engineers would open a case to have a recreate built by our lab team, which could take days.  I never figured out why they wouldn’t use IOU.

When I left Cisco I had to resort to GNS3, which was a pretty helpful piece of software.  Then, when I went to Juniper I used Junosphere, or actually an internal version of it called VMM, to spin up topologies.  VMM was awesome.  Juniper produced a virtual version of its MX router that was so faithful to the real thing that I could pass the JNCIE Service Provider exam without ever having logged into a real one, at least until exam day.

It’ll be interesting to see what I can do on virtual 9ks in CML–I hear there are some limitations.  But I do plan to spend as much time as possible using the virtual version over the real thing.

One thing I think I lost sight of as I (slightly) climbed the corporate ladder was the necessity of technical leadership.  We have plenty of people managers and MBAs.  We need leaders who understand the technology, badly.  And while I have a lot of legacy knowledge in my mental database, it’s in need of refresh.  It’s hard to stay sharp technically when reading about new technologies in PowerPoint.

The other side of this is that, as engineers, we love the technology.  I love making stuff work.  My wife is not technical at all, and cannot understand why I get a thrill from five little exclamation points when a ping goes through.  I don’t love budgets and handling HR cases, although I’ve come to learn why I need to do those things.  I need to do them so my people can function optimally.  And I’m happy to do them for my team.

On the other hand, I’m glad to be in the frigid, loud, harsh lighting of a massive Cisco lab again. It’s very cool to have all this stuff.   Ain’t life grand!

5

The death of CLI: A lesson from Swift

Update:  From Fred, who was the guy referenced in the first paragraph:

Actually it was a white button with a router icon on it and “make cli great again”, I know this because it was me. It was June 2016. Needless to say in my view that did not age well.

When I attended Cisco Live sometime around the election of Donald Trump, there was a fellow walking around with a red hat with white lettering on it:  MAKE CLI GREAT AGAIN.  Ha!  I love Cisco Live.  These are my people.

I remember back when I worked at Juniper, one exec looked at me working on CLI and said, “you know that’s going to be gone soon.  It’ll all be GUI.”  That was 8 years ago…how’s that going?  When I work on CLI (and I still do!), or programming, my wife always says, “how can you stare at that cryptic black screen for hours?”  Hey, I’ve been doing it since I was a kid.

The black screen won’t go away, I’m afraid.  I’ve recently been learning iOS app development for fun (not profit).  It’s surprisingly hard given the number of successful app developers out there.  I may be too used to Python to program in Swift, and my hatred of object-oriented programming doesn’t help me when there is no way to avoid it in Swift.  Anyways, it took me about a week to sort out the different UI frameworks used in iOS.  There are basically three:

  • Storyboards.  Storyboards are a graphical design framework for UI layout.  Using storyboards, you drag and drop UI elements like buttons and textfields onto a miniature iPhone screen.
  • UIKit.  (Technically storyboards use UIKit, but I don’t know what else to call this.)  Most high-end app developers will delete the storyboard in their project and write the UI as code.  They actually type in code to tell iOS what UI elements they want, how to position them, and what to do in the event they are selected.  Positioning is fairly manual and is done relative to other UI elements.
  • SwiftUI.  Apple is pushing towards this model and will eventually deprecate the other two.  SwiftUI is also a UI-as-code model, but it’s declarative instead of imperative.  You tell SwiftUI what you want and roughly how you want to position things, and Swift does it for you.

Did you catch my point?  The GUI-based layout tool is going away in favor of UI-as-code!  The black screen always comes back!

The difference between computer people and non-computer-computer-people (many industry MBAs, analysts, etc.), is that computer people understand that text-based interaction is far more efficient, even if the learning curve is steeper.

Andrew Tanenbaum, author of the classic Computer Networks, typeset his massive work in troff.  Troff is a text-based typesetting tool where you enter input like this:

.ll 3i
.mk a
.ce
Preamble
.sp
We, the people of the United States, in order
to form a more perfect Union...

Why doesn’t he just use Word?  I’ll let Dr. Tanenbaum speak for himself:

All my typesetting is done using troff. I don’t have any need to see what the output will look like. I am quite convinced that troff will follow my instructions dutifully. If I give it the macro to insert a second-level heading, it will do that in the correct font and size, with the correct spacing, adding extra space to align facing pages down to the pixel if need be. Why should I worry about that? WYSIWYG is a step backwards. Human labor is used to do that which the computer can do better.  (Emphasis added.)

I myself am not quite enough of a cyborg to use troff (though I use vi), but I have used Latex with far better results than Word.  (Dr. Tanenbaum says “real authors use troff,” however.)

One of my more obscure interests (I have many) is Gregorian Chant.  Chant uses a musical notation which is markedly different from modern music notation, and occasionally I need to typeset it.  I use a tool called Gregorio, where I enter the chant like this:

(cb3) Ad(d)ór(f’)o(h) te(h’) de(h)vó(hi)te,(h.) (,) la(g)tens(f) Dé(e’)i(d)tas,(d.)

The letters in parentheses represent the different musical notes.  I once tried typesetting the chant graphically, and it was far more tedious than the above.  Why not enter what I want and let the typesetting system do the work?

Aside from the mere efficiency, text files can be easily version controlled and diff’d.  Try that with your GUI tool!

It’s very ironic that many of my customers who use controllers like DNAC or vManage are actually accessing the tool through APIs.  They bought a GUI tool, but they prefer the black screen.  The controller in this case becomes a point of aggregation for them, a system which at least does discovery and allows some level of abstraction.

The non-computer-computer-people look at SwiftUI, network device CLI, troff, Gregorio, APIs, and rend their garments, crying out to heaven, “why, oh why?!”  Some may even remember the days of text-based editing systems on their DOS machines, which they could never learn, and the great joy that WYSIWYG brought them.  It reminds me of a highly incompetent sales guy I worked with at the Gold partner back in the day.  He once saw me configuring a router and said:  “Wow, you still use DOS to configure routers!”

“It’s actually IOS CLI, not DOS.”

“That’s DOS!” he densely replied.  “I remember DOS.  I can’t believe you still use DOS!”

It’s funny that no matter how hard we try to get away from code, we always come back to it.  We’re hearing a lot about “low code” environments these days.  It tells you something when the first three Google hits on “low code” just come back to Gartner reports.  Gee, have we been down this path before?  Visual Basic was invented in 1991.  If low code is so great, why is Apple moving from storyboards to SwiftUI?

In my last post I wrote about the war on expertise.  This is one of the fronts in the war.  The non-computer-computer-people cannot understand the black screen, and are convinced they can eliminate it.  They learned about “innovation” in business school, and read case studies about Windows 95 and the end of DOS.  They read about how companies like Sun Microsystems went belly-up because they are not “disruptive.”  They did not, however, read about all the failed attempts to eliminate the black screen, spanning decades.  I believe it was George Santayana who said, “If you don’t remember computer history, you’re doomed to repeat it.”

6

The War on Expertise

Like many network engineers, I quickly fell in love with my field and worked hard to master it.  I got into networking when I was working in desktop support.  The behind-the-scenes stuff was way more interesting to me than the front lines.  Back in the late nineties, I bought a library of books to learn this field.  Perlman, Comer, and Stevens were the classics.  I rounded it out with blue-and-white Cisco Press books by Doyle, Peplnjak, Williamson, and many others.  I studied these books religiously, read through config guides, and practiced in the lab.

The network engineers on my team and I loved to debate the arcana of this mysterious field.  We always tried to one-up each other, learning new technologies, new protocols, and attaining new technical certifications.  I’ve worked with engineers who are smarter than I am, and better than I am, but that always motivated me to learn more.

I bring this up because I’ve had multiple conversations with multiple execs, for many years, in which they seem to decry the virtue of expertise.  Network engineers “revel in complexity”, they don’t realize their time has passed, the build networks that need “armies of CCIEs to maintain”, and they hate simplicity.  If only the pesky network engineers would get out of the way, the glorious MBAs could build us simple and elegant products, which is how the industry is going, don’t you know!

In short, our industry is suffering through a war on expertise.  Those arcana we love to master have put a target on our back.  If you want to learn those things, you must be reveling in complexity.  Go find something else to do, ChatGPT will replace you!

The first mistake in this line of thinking is the assumption that network engineers want to build networks that are complex.  We actually don’t.  A couple of anecdotes:

When I was working for a Gold partner, I was sent to help out an IT manager of a rather small company, only four sites.  She had contracted VPLS from two service providers, and asked me to implement a complex load balancing scheme she had conceived.  I begged her not to make me do it, but she insisted.  I ended up building a functional mess, a combination of PBR and EIGRP offset access-lists.  Man, was it ugly.  But it worked.  Then I got laid off from the partner, and a year later she was calling me, begging me to come back because nobody could figure out how it worked.  I didn’t want to build something that ugly and she didn’t need it.

Second anecdote.  My wife had to go in for a surgical procedure a few years ago.  We went to the best doctor in San Francisco.  When he got into the procedure he found that her anatomy is not conventional, and it was a very difficult procedure.  In the recovery room, he told us most doctors would have stopped.  My wife wanly smiled and said, “well, I’m sure you like a challenge.”  He looked back at her and said, “no, we like it when it’s easy.”

I think this is where the execs misunderstand expertise.  99.9% of the time, your airline flight could be handled by a low-time pilot who can work the automation systems.  But when the engines fail, you want Sully at the controls.  Just because some people understand complexity and study difficult concepts, it doesn’t follow that they want complexity.  When I administered networks, I wanted it to be easy.  But I was ready for when it was hard.

The war on expertise seems to me to be a war on the human spirit.  The CCIE exam, whatever you think of it, was a heck of a challenge, and passing it was one of the proudest days of my life.  Human beings want to learn, to grow, to push their limits, and to test themselves.  That’s why we spend hours in the lab.  We should encourage this behavior.  We should want people in our business who seek subject matter expertise and mastery.  We can make things simpler, fine, but we should still encourage expertise.

At the end of the day, networks are inherently complex.  A network is a large distributed system, connecting numerous devices running numerous operating systems over diverse transport mechanisms using a wide variety of protocols.  You can simplify the protocols a bit, but ultimately most simplification of networks is done one of two ways:  reducing the number of choices an administrator can configure, or abstracting and hiding the underlying complexity.  In the first case, you may close out necessary use cases.  In the case of abstraction, well, it works great until something breaks.  Then you need to call a network engineer.

I’m not in any way saying the new tools, from programmability to automation systems like Ansible, to “controllers” are unnecessary.  Far be it.  Any tool that makes an engineer’s job easier will be embraced by engineers.  I am saying that we need to stop blaming complexity on those who manage to understand it.

27

Plain Language

I don’t know about the rest of the world, but here in America our police speak in a sort of code language.  Instead of saying “he got out of the car and walked away,” the police will say, “the subject exited the vehicle and proceeded on foot.”  It’s not that their language is any clearer–in fact it’s less clear.  When talking to each other, the cops like to use language like this because it seems to elevate them and make them sound more professional.

A friend of mine told me a story of his friend who had always wanted to be a cop, but was too much of a screw-up to make it to the academy.  Despite his failure he remained a police buff, perhaps a cop in his own mind.  This fellow witnessed a crime and the local sheriff showed up.  This non-cop started to describe the crime to the sheriff as cops do:  “The subject exited the vehicle and proceeded to commit a 4-15…”  The deputy cut him off and shouted, “speak English, boy!”  The poor police-wannabe never lived it down.

Corporatist-types have a language like this too.  Attempting to sound smart and professional, they speak in an often-inaccessible code language replete with b-school buzz words.  I must admit, as long as I’ve been in the corporate world, I’ve frequently been confronted by language that I simply couldn’t understand.

“We collaborated with engineering and cross-functional product managers across multiple time zones to groom and prioritize backlog to ensure efficient program delivery.”  Huh?  “We need to build a motion that creates value at scale.”  What? “The adoption journey enables us to innovate continuously.”  Speak English, boy!

Then there was this gem, from an MBA describing a customer problem:  “The customer is not in the mindset of extracting value from the product.”  A lot of words there.  How about three words:  “It don’t work.”  Or, “customer hates it.”  Oh no.   If the MBA spoke that way, he’d sound like he learned nothing at his prestigious business school.  Professionals speak professionally, you see.  If he spoke like a normal human being, some people might suspect he actually doesn’t really know anything.  Although to be honest, that’s exactly what I was suspecting when he started talking about mindsets.

We can all fall into this trap, I’m afraid.  I refused, for years, to use “ask” and “spend” as nouns, because they’re not nouns.  (I remember an internal thread at Cisco years back in which someone said, “shouldn’t we productize that?”  The snarky response came back from an engineer, “no because at Cisco we can’t turn nouns into verbs.”)  Alas, I’ve surrendered to the progress of business-speak and have replaced “request” with “ask.”  Saving one syllable with a frequently-used word has certainly given me hours back to do other things, don’t you think?

Technical people can certainly fall into this and we have our own jargon.  Some of it is necessary.  Here is a snippet of a Cisco doc:  “To overcome the limitations of the flood-and-learn VXLAN as defined in RFC 7348, organizations can use Multiprotocol Border Gateway Protocol Ethernet Virtual Private Network (MP-BGP EVPN) as the control plane for VXLAN. ”  This is wordy, and it is jargony.  That said, I can’t think of a better way to say it.  This sort of language is unavoidable for network engineers.

What I don’t like is technical people adopting MBA-speak because they’re surrounded by it.  “Our latest release provides flexible options to operationalize your business intent.”  Oh dear, even if you get into some good technical meat, you’ve lost me already.  The simple secret for me in winning Cisco Live Hall of Fame for my speaking is simply to state things in plain, clear language.  Technical, yes, but clear.

I used to think I was stupid, sitting in meetings in the corporate world and not understanding what on Earth people were saying.  Then I learned that in many cases, the speakers didn’t understand what they were saying either.  In the event that they actually do, a few pointed questions can usually cut through the fog of fancy words.  I’m convinced many of the mistakes made in the corporate world would never happen if people actually spoke like normal people.

If you feel tempted to obscure your language to sound like you’re oh-so-smart, remember the advice of the deputy sheriff:  Speak English, boy!

8

The problem of permanence

Some years ago, I worked for a company with a CEO who had a background in marketing.  It was 2010, and he decided to use his marketing skills to launch a huge new campaign called “Mission 10”.  Our goal:  to become the next $10 billion company.  At the time I think I revenue was less than five billion.  Slick slides were drawn up, pep rally company meetings were held, and everyone in the company began pivoting their work to fit the new agenda.  Anyone who has worked in the corporate world has been there more than once.  Suddenly every initiative had to have a “Mission 10” theme.

The problem?  Despite the rah-rah of our CEO, we never achieved even close to $10 billion in revenue.  In fact, that company is still below $5 billion last I checked.  The bigger problem?  The CEO moved three years after that, having never really achieved this or any other goal he set.  He later ended up CEO of an even larger and more famous company that has nothing to do with technology.  This is known as “failing upward”.

In light of the “great resignation” I’d like to write a little about permanence, or the lack thereof.  We live in a temporary world.  People pick up a job and stay for two or three years, and then move on.  This was true even before COVID.  I myself have several two-year stints on my resume.  The longest I’ve worked anywhere is six.

Three years is just long enough to kick off some major initiative and get out at the peak, just before the whole thing crashes.  The damage done by corporate executives pursuing this short-term strategy is massive.  It works like this.  An exciting new executive is hired on from a big company.  The new executive launches a new product, architecture, marketing campaign, acquisition, whatever.  Everyone rallies around it because, well he’s the boss, and because if you want funding for anything it needs to be tied to the boss’ initiative.  The new initiative (let’s say it’s a product) is pumped up with cash, the marketing engine kicks in, the company oversells the product, and then customers start snapping it up.  It doesn’t perform as expected.  Things start to crash.  Money dries up.  The executive exits.   And whoever decides to stay is left picking up the pieces of the mess that this guy created.

In Ancient Greece, you faced consequences for this sort of thing, usually exile, sometimes death.  While I’m not advocating the death penalty for corporate screw ups (although in some industries they do cost lives), what’s fascinating is that in the corporate world, the consequences are the opposite.  Said executive who just screwed up royally walks away with huge bonuses, lots of stock, has a nice sabbatical, and begins the cycle again somewhere else.

If you think executives are the only problem, think again.  It happens at every level of the corporate world.  When a junior IT guy messes up a new system and then bolts for another job, you have the same issue at a smaller scale.  He just doesn’t get the bonus and sabbatical.  As a leader of technical marketing engineers, we face all sorts of challenges when an experienced TME leaves and takes knowledge with him.  Features can be stalled when the people who were working on them leave.

In my grandfather’s era, and even my father’s, it was expected that you would start and end your career at the same company.  There was an expectation of permanence.  People were proud of their companies and how they were treated, and bragged about the excellent pension they’d receive when leaving.  Now, we spend three years and jump ship to boost our salary.

Companies, are of course, largely responsible.  Often they don’t create the sort of employment experience that anyone would want to tolerate for long.  People stopped being human beings and started becoming human “resources”.  Executives, under various pressures, began to see their workforce as mere “metrics” to be manipulated as they learned in their B-school classes.  Times are good?  Dial up the workforce.  Times are bad?  Lay off 3%.  People are just numbers on a slide to many execs, and the difficulties of terminating employment are a remote problem to be dealt with by line managers.  As a result, employment is not a long-term commitment but a short-term business transaction on both ends.

The temporary workforce has an interesting effect on longer-term employees as well.  Someone who has worked at the same company for 15 or 20 years sees executives and initiatives come and go, ebbing and flowing like the tide on a beach.  They often develop an apathy and callousness that makes their own work unproductive.  They tend to focus on the day-to-day instead of the long-term, and often dismissively ignore the plans of new leadership, figuring the leaders will just be replaced and the cycle will start over.  Thus, while they have a long-term career, they often have a short-term level of focus.

We all live in a temporary world now, and permanence is in short supply.  If you want to understand why companies build bad products, why executives start disastrous programs and leave, and why there never seem to be consequences, this is a huge part of it.  I don’t really have a solution I’m afraid. Some of the causes are:  greedy hedge-fund finance people who take over corporate boards, an undisciplined corporate press/media, the instant availability of information leading to a lack of deliberation, and the rise of a management class who are not actually experts in anything other than management itself.  We can all point fingers at ourselves for up and going when the going gets tough.

The Greek philosopher Heraclitus famously said that you cannot step in the same river twice.  He meant that, if you cross a river, each time you take a step the water you were originally standing in has passed on, and you’re in new water.  Thus, there’s really no river.  Sometimes the tech world, and the corporate world in general, feel like Heraclitus’ River.  Even if you stand in one place, everything just moves on.

5

Technical Confusion

How often have you learned about a new technology, and couldn’t understand it?  How many trainings and presentations have you sat through that left you in a mental fog?  It amazes me how many technologies we are supposed to master in our industry, and how many we never do.

Let me give an example.  When I heard about “Cloud Computing” I could not, for the life of me, understand what it meant.  I went to meeting after meeting where we talked about “the Cloud” without any understanding of what it actually was.  I knew I used clouds a lot of Visio diagrams, but the MBA-types who were telling me we needed to migrate to the cloud would never be able to understand the Visio diagrams that network engineers make.  It seemed to involve using centralized computing resources, but I’d been doing this for years.  My first ISP accounts were shell accounts.  My email and other services were hosted on their computers.  Nothing was new about this.  In fact, Larry Ellison gave a hilarious talk in which he asked “What the hell is Cloud Computing?”

We all know the “cloud” has in fact made significant changes in how we engineer computing resources, but the truth is, the idea of centralized “compute” is not a new one.  (Side note:  I hate turning nouns into verbs.  “Compute”, “spend”, and “ask” are verbs, not nouns.  The MBAs who invent these terms apparently don’t have to study grammar.)  The scale is certainly different, but we all know that mainframes had both centralized computing and virtualization long before anyone said “cloud.”

SDN is another one.  I was told we needed SDN, but I couldn’t figure out what it meant.  I was a hard-core routing protocols guy.  BGP and OSPF are software.  Ergo, networks are already software defined.

Someone sent me a video from Nicira, later acquired by VMware.  The vague video described slicing networks into pools, or something like that.  I couldn’t understand what this meant.  Like a VLAN?  I finally found a document that described SDN as separation of the control plane from the data plane.  OK, but we already had been doing that in routers and switches for years?  Yes, but SDN was a centralized control plane.  Kind of like BGP route reflectors?  I couldn’t figure it out.  I spent some time getting OpenFlow up and running to try to understand it from the ground up.  What a waste that was.  Whatever SDN has become, it’s certainly not what it was originally defined to be.  And don’t get me started on SASE.

I used to think maybe I was stupid, but now I realized all of these things confused me because they were (a) confusing in themselves, or (b) so badly explained that nobody really understood them.  A little more detail:

  • Some technologies are simply vague marketing terms.  They don’t correspond to anything precise in reality.
  • Some technologies do correspond to reality, but they are simply bucket terms.  That is, the marketers took five, six, ten technologies, and slapped a new label on them.  In this case, you’re looking for some precise definition of term X and you realize term X refers to ten different things at once.
  • Sometimes new technologies are invented, and the inventors don’t want to cough up too much proprietary information.  So the produce vaguely worded marketing content that appeals to “analysts” with MBAs in marketing, but which technical people realize are meaningless.  Said “analysts” now run around creating hype (“You need software-defined cloud secure-access zero trust!”) and now we’re told we to implement it.
  • A lot of technical people are really bad explainers.  Sometimes there is a new technology which is clear and well-defined, but the people sent to explain it are completely incapable of explaining anything at all.

My point is, it’s ok to be confused.  A lot of times we’re in the room and everyone seems to be getting it, but we have no idea what is going on.  Chances are, nobody else really understands what is being said either.  Ask questions, drill down, and if you don’t understand something, chances are it’s hot air.  In a world where we prioritize talk over reality, there seems to be an abundance of that.

6