Skip navigation

Tag Archives: language

ChatGPT and the problem of authority

I’ve been reluctant to comment on the latest fad in our industry, generative AI, simply because everybody has weighed in on it.  I do also try to avoid commenting on subjects outside of my scope of authority.  Increasingly, though, people are coming to me at work and asking how we can incorporate this technology into our products, how our competitors are doing it, and what our AI strategy is.  So I guess I am an authority.

To be honest, I didn’t play with ChatGPT until this week.  When I first looked at it, it wanted my email address and phone number and I wasn’t sure I wanted to provide that to our new AI overlords.  So I passed on it.  Then Cisco release an internal-only version, which is supposedly anonymous, so I decided to try it out.

My first impression was, as they say, “meh.”  Obviously its ability to interpret and generate natural language are amazing.  Having it recite details of its data set in the style of Faulkner was cool.  But overall, the responses seemed like warmed-over search-engine results.  I asked it if AI is environmentally irresponsible since it will require so much computing power.  The response was middle-of-the-road, “no, AI is not environmentally irresponsible” but “we need to do more to protect the environment.”  Blah, blah.  Non-committal, playing both sides of the coin.  Like almost all of its answers.

Then I decided to dive a bit deeper into a subject I know well:  Ancient Greek.  How accurately would ChatGPT be on a relatively obscure subject (and yet one with thousands of years of data!)

Even if you have no interest, bear with me.  I asked ChatGPT if it knew the difference between the Ionic dialect of Herodotus and the more common dialect of classical Athens.  (Our version, at least, does not allow proper names so I had to refer to Herodotus somewhat elliptically.)  It assured me it did.  I asked it to write “the men are arriving at Athens” in the dialect of Herodotus.  It wrote, “Ἀφίκοντο οἱ ἄνδρες εἰς Ἀθήνας,” which is obviously wrong.  The first word there, “aphikonto“, would actually be “apikonto” in the dialect of Herodotus.  He was well known for dropping aspirations.  The version ChatGPT gave me would be the classical Attic version.

I let ChatGPT know it was wrong, and it dutifully apologized.  Then I asked it to summarize the differences in the dialects.  It said to me:

Herodotus and Ionic writers typically removed initial aspirations, while the Attic dialect retained them. For example, “Ἀφίκοντο” (Herodotus) vs. “ἔφικοντο” (Attic)

Uh, you don’t need to know the Greek alphabet to see it made exactly the same mistake, again.  It should have said that Herodotus would use “Ἀπίκοντο” (apikonto) whereas in Attic the word would be “Ἀφίκοντο” (aphikonto.)

Here’s a little more computer-person-friendly example.  I decided to ask about my other favorite subject, retro computing.  I asked, “What is the difference between the Apple IIGS ProDOS 8, ProDOS 16, and GS/OS operating systems?”  Among other things, ChatGPT responded that GS/OS ” introduced features such as a desktop metaphor, drop-down menus, and a hierarchical file system.”  Which happens to be wrong.  ProDOS 16 actually introduced those features.  (In fact, ProDOS 8 introduced the hierarchy.)  I pointed this out.  “You are correct, and I apologize for the oversight in my previous response. I misspoke in stating that GS/OS introduced [those] features.”  Hmmm.

I’m testing it and I know the right answers.   The problem would arise if I were trusting ChatGPT to provide me with a correct answer.  There have been plenty of examples of mistakes made by ChatGPT, such as adding a “permit any” to the top of access-lists.

The issue is, ChatGPT sounds authoritative when it responds.  Because it is a computer and speaking in natural language, we have a tendency to trust it.  And yet it has consistently proven it can be quite wrong on even simple subjects.  In fact, our own version has the caveat “Cisco Enterprise Chat AI may produce inaccurate information about people, places, or facts” at the bottom of the page, and I’m sure most implementations of ChatGPT carry a similar warning.

Search engines place the burden of determining truth or fiction upon the user.  I get hundreds or thousands of results, and I have to decide which is credible based on the authority of the source, how convincing it sounds, etc.  AI provides one answer.  It has done the work for you.  Sure, you can probe further, but it many cases you won’t even know the answer served back is not trustworthy.  For that reason, I see AI tools to be potentially very misleading and potentially harmful in some circumstances.

That aside, I do like the fact I can dialog with it in ancient Latin and Greek, even if it makes mistakes.  It’s a good way to kill time in boring meetings.

7

Plain Language

I don’t know about the rest of the world, but here in America our police speak in a sort of code language.  Instead of saying “he got out of the car and walked away,” the police will say, “the subject exited the vehicle and proceeded on foot.”  It’s not that their language is any clearer–in fact it’s less clear.  When talking to each other, the cops like to use language like this because it seems to elevate them and make them sound more professional.

A friend of mine told me a story of his friend who had always wanted to be a cop, but was too much of a screw-up to make it to the academy.  Despite his failure he remained a police buff, perhaps a cop in his own mind.  This fellow witnessed a crime and the local sheriff showed up.  This non-cop started to describe the crime to the sheriff as cops do:  “The subject exited the vehicle and proceeded to commit a 4-15…”  The deputy cut him off and shouted, “speak English, boy!”  The poor police-wannabe never lived it down.

Corporatist-types have a language like this too.  Attempting to sound smart and professional, they speak in an often-inaccessible code language replete with b-school buzz words.  I must admit, as long as I’ve been in the corporate world, I’ve frequently been confronted by language that I simply couldn’t understand.

“We collaborated with engineering and cross-functional product managers across multiple time zones to groom and prioritize backlog to ensure efficient program delivery.”  Huh?  “We need to build a motion that creates value at scale.”  What? “The adoption journey enables us to innovate continuously.”  Speak English, boy!

Then there was this gem, from an MBA describing a customer problem:  “The customer is not in the mindset of extracting value from the product.”  A lot of words there.  How about three words:  “It don’t work.”  Or, “customer hates it.”  Oh no.   If the MBA spoke that way, he’d sound like he learned nothing at his prestigious business school.  Professionals speak professionally, you see.  If he spoke like a normal human being, some people might suspect he actually doesn’t really know anything.  Although to be honest, that’s exactly what I was suspecting when he started talking about mindsets.

We can all fall into this trap, I’m afraid.  I refused, for years, to use “ask” and “spend” as nouns, because they’re not nouns.  (I remember an internal thread at Cisco years back in which someone said, “shouldn’t we productize that?”  The snarky response came back from an engineer, “no because at Cisco we can’t turn nouns into verbs.”)  Alas, I’ve surrendered to the progress of business-speak and have replaced “request” with “ask.”  Saving one syllable with a frequently-used word has certainly given me hours back to do other things, don’t you think?

Technical people can certainly fall into this and we have our own jargon.  Some of it is necessary.  Here is a snippet of a Cisco doc:  “To overcome the limitations of the flood-and-learn VXLAN as defined in RFC 7348, organizations can use Multiprotocol Border Gateway Protocol Ethernet Virtual Private Network (MP-BGP EVPN) as the control plane for VXLAN. ”  This is wordy, and it is jargony.  That said, I can’t think of a better way to say it.  This sort of language is unavoidable for network engineers.

What I don’t like is technical people adopting MBA-speak because they’re surrounded by it.  “Our latest release provides flexible options to operationalize your business intent.”  Oh dear, even if you get into some good technical meat, you’ve lost me already.  The simple secret for me in winning Cisco Live Hall of Fame for my speaking is simply to state things in plain, clear language.  Technical, yes, but clear.

I used to think I was stupid, sitting in meetings in the corporate world and not understanding what on Earth people were saying.  Then I learned that in many cases, the speakers didn’t understand what they were saying either.  In the event that they actually do, a few pointed questions can usually cut through the fog of fancy words.  I’m convinced many of the mistakes made in the corporate world would never happen if people actually spoke like normal people.

If you feel tempted to obscure your language to sound like you’re oh-so-smart, remember the advice of the deputy sheriff:  Speak English, boy!

8