Feeling like a Puppet

Starting a new job is always a challenge.  It’s one thing to do an internal transfer within a company, like I did when moving from Technical Marketing Engineer to Product Management.  I even stayed in the same team.  But moving to a new company and new role is overwhelming and risky.

I experienced this when I went to Juniper.  I’d defined my entire existence in terms of Cisco.  When I worked at a Gold partner, I refused to touch anything–servers, desktops, etc.–that was not Cisco-branded.  I’d barely even touched a Juniper device.  Now I was expected to coherently articulate a network architecture when I didn’t even understand the products that were involved.  (One could argue network architecture should not be product-specific, but we all know better.)

Ironically enough, after six years at Juniper, going back to Cisco proved challenging.  I didn’t touch a single Cisco device in six years.  At the time I went to Juniper, Nexus products didn’t even exist.  Now I was expected to be doing technical marketing–at the principal level–for products I hadn’t even heard of.

My boss, the legend Carl Solder, assigned me to programmability.  I knew nothing about it.  During my interviews with Carl, I had mostly talked about controller-based architectures.  While I was at Juniper, Jeremy Pruitt, an automation expert, had managed to get Puppet working for managing Juniper devices.  He pitched it to me and I had no interest at all.  This was a server technology, not a networking technology.  I was a protocols guy.

Now I was assigned to YANG, NETCONF, Python scripting, Ansible, and (ha, ha) Puppet.  I was fortunate to end up on a team of low-ego engineers who helped me to learn this stuff and who were willing to share opportunities with me.  They are still friends to this day.

One of the opportunities they shared was a chance to present at Tech Field Day/Networking Field Day.  In fact, TFD is a high-stress presentation, and nobody really wanted to do it.  Thus, three months into my new job, knowing nothing about Puppet or Nexus, I was called upon to present Puppet management of Nexus to one of the most challenging audiences we have.  I think TFD delegates can be a bit nicer these days, but back then a few of them loved to eviscerate vendors, especially Cisco.  Oh, and Jason Edelman and Matt Oswalt, two automation experts, would be in the audience.  Also Terry Slattery, who is the first person ever to pass the CCIE lab.  And my boss Carl, one of the best public speakers at Cisco.  And it would be livestreamed and everyone (it seemed) in Cisco would be watching it.  Great.

A tough room to stand in front of

The only way out was through.  My team was very helpful in preparing me, and I was able to spend a lot of time with engineering.  Truth be told, it was just technology, and I’m a technology guy.  It was learnable.  Cisco’s marketing team was also very helpful, explaining possible pitfalls and traps in presenting to the TFD audience.  For example, if you put “Cisco Confidential” on your slides, they’ll make fun of you.  (It’s a default on our PowerPoint templates.)  They told us not to cite Gartner.  I made a fake slide with ridiculously large “Cisco Confidential” and “DO NOT SHARE”  and a magic quadrant and presented it at rehearsal.  The look of horror from the marketing team told me they didn’t get the obvious joke.

It’s a joke, really!

Puppet was particularly challenging because, unlike Ansible, it was agent-based.  (I’m not sure if this is the case anymore, I rarely hear about Puppet in networking contexts.)  This meant you had to bring up a guestshell and install the agent on any device that it managed.  To be honest, this is less than ideal.  It means that before you could use Puppet to manage the device, there was a fair amount of setup required.  This could be done through PoAP (Power-on Auto-Provisioning, Cisco’s version of ZTP for Nexus), but it required a lot more work than simply enabling a few CLIs, all that was needed for Ansible.

There was a non-technical challenge that faced me as well:  I had been working on overcoming a terrible fear of public speaking.  I had melted down in one presentation in my early years at Juniper, and had spent the last several years doing training, Toastmasters, and anything I could to get over this problem.  I had successfully delivered some major presentations at Juniper before I left, but this was the highest pressure I had ever been under.  Would I panic?  Would I crack?  Would I be shown the door only a few months into the new job?  Public speaking is risky, because when you fail, you fail in front of a lot of people.

I’m still here, of course, and aside from a couple minor stumbles, it went off quite well.  Watching the video eight years later, I do cringe a bit at some of my slides.  I’m not a genius when it comes to PowerPoint, but I’ve done a lot of work to improve the quality of my decks.  Given the time constraints, I had to borrow some slides and hack together others.  I do hear some of my “signature” lines taking shape.  In nearly all my presentations on automation, I joke about Notepad being the most common automation tool used by network engineers, and this must be the first place I used that line.  And while Jason and Terry did ask questions, they didn’t break me and I was able to answer them quite assuredly.

My demo was a bit simplistic.  I provisioned a few VLANs and interfaces, and pushed a patch to a Nexus 9k.  As I started to take over lead on programmability, I changed my approach to it.  For example, when I showed network engineers how to provision a VLAN with NETCONF, their eyes would glaze over upon seeing the massive block of Python and XML needed to make it work.  Why not just type one line of CLI?  I became convinced we needed to show the value of programmability by showing how we can use it beyond just simple tasks.  Configure a VLAN?  Meh.  Use Webex Teams (or Slack) to interact with a router, better.  We did some silly demos like using Alexa to configure switches, but the point was actually to show that programmability opens up possibilities that were not there before.  Nonetheless, when I did this TFD, it was still early days.

My TFD session also ended up generating a popular meme, although it wasn’t on my account.  Jason Edelman and Matt Oswalt were sitting next to each other.  Jason was dressed in a sportcoat, clean cut, with Windows PC, and no stickers on it.  Matt was sitting next to him with unkempt hair, using a Mac covered with stickers.  Someone screenshotted it and posted it to Reddit with the caption, “the two types of network engineers”.  It’s funny because Matt and Jason wrote a book on network automation together.  (In the photo, my boss Carl looks on in the background on the right.  Dead center in red is Lauren Friedman, with whom I worked many times as our TFD coordinator in marketing.  Stephen Foskett, one of the TFD organizers, is between Matt and Jason.)

The two types of network engineers

 

In our careers as network engineers, we’re often thrown into things we know nothing about and are expected to quickly turn around and become experts. It happened every week in TAC. Working in product marketing, we’re often on the cutting edge and constantly fighting to stay above water. I had only a few weeks to learn Puppet, get a demo up and running, and present it to a challenging audience. The most important thing is to not get cocky, to understand what you don’t know, and to spend the hands-on time to plug those gaps in your knowledge.

Every public speaker knows the feeling of relief when you finally finish and the stress unloads. As I closed my talk and walked to the back of the room, Carl, my new boss, looked at me and gave me a thumbs-up. At that point if you told me I’d win Hall of Fame at Cisco Live for talking about programmability, I’d have thought you were crazy.

As for Puppet, I never touched it again.

Programmability for Network Engineers

Since I finished my series of articles on the CCIE, I thought I would kick off a new series on my current area of focus:  network programmability.  The past year at Cisco, programmability and automation have been my focus, first on Nexus and now on Catalyst switches.  I did do a two-part post on DCNM, a product which I am no longer covering, but it’s well worth a read if you are interested in learning the value of automation.

One thing I’ve noticed about this topic is that many of the people working on and explaining programmability have a background in software engineering.  I, on the other hand, approach the subject from the perspective of a network engineer.  I did do some programming when I was younger, in Pascal (showing my age here) and C.  I also did a tiny bit of C++ but not enough to really get comfortable with object-oriented programming.  Regardless, I left programming (now known as “coding”) behind for a long time, and the field has advanced in the meantime.  Because of this, when I explain these concepts I don’t bring the assumptions of a professional software engineer, but assume you, the reader, know nothing either.

Thus, it seems logical that in starting out this series, I need to explain what exactly programmability means in the context of network engineering, and what it means to do something programmatically.

Programmability simply means the capacity for a network device to be configured and managed by a computer program, as opposed to being configured and managed directly by humans.  This is a broad definition, but technically using an interface like Expect (really just CLI) or SNMP qualifies as a type of programmability.  Thus, we can qualify this by saying that programmability in today’s world includes the design of interfaces that are optimized for machine-to-machine control.

To manage a network device programmatically really just means using a computer program to control that network device.  However, when we consider a computer program, it has certain characteristics over and above simply controlling a device.  Essential to programming is the design of control structures that make decisions based on certain pieces of information.

Thus, we could use NETCONF to simply push configuration to a router or switch, but this isn’t the most optimal reason to use it.  It would be a far more effective use of NETCONF if we read some piece of data from the device (say interface errors) and took an action based on that data (say, shutting the interface down when the counters got too high.)  The other advantage of programmability is the ability to tie together multiple systems.  For example, we could read a device inventory out of APIC-EM, and then push config to devices based on the device type.  In other words, the decision-making capability of programmability is most important.

Network programmability encompasses a number of technologies:

  • Day 0 technologies to bring network devices up with an appropriate software version and configuration, with little to no human intervention.  Examples:  ZTP, PoAP, PnP.
  • Technologies to push and read configuration and operational data from devices.  Examples:  SNMP, NETCONF.
  • Automation systems such as Puppet, Chef, and Ansible, which are not strictly programming languages, but allow for configuration of numerous devices based on the role of the device.
  • The use of external programming languages, such as Ruby and Python, to interact with network devices.
  • The use of on-box programming technologies, such as on-box Python and EEM, to control network devices.

In this series of articles we will cover all of these topics as well as the mysteries of data models, YANG, YAML, JSON, XML, etc., all within the context of network engineering.  I know when I first encountered YANG and data models, I was quite confused and I hope I clear up some of this confusion.

1
1