An Absolutely Abysmal Completion Percentage
Leading up to Oscar weekend (the beginning of March), I will conclude each newsletter with a piece of script from a favorite film or favorite scene dealing with “business.” Sort of. Probably.
Yesterday, I briefly mentioned that Aaron Sorkin won an Oscar for writing “The Social Network.” He could’ve easily won another one for writing the “Steve Jobs” screenplay.
Director Danny Boyle did a fantastic directing job with Sorkin’s script and called it, “It felt like it was like an action movie with words.”
From a PBS article on Transcendentalism from its “American Experience” documentary series:
Walt Whitman’s ego seemed impervious to criticism, and his self-promotion — writing anonymous reviews of his own book — suggests total self-assurance. There was, however, one man whose praise Whitman desired and cherished more than any other. He would refer to this man as his “master,” Ralph Waldo Emerson.
I was reminded of an Emerson quote from a Not Boring newsletter from last year:
There is a time in every man’s education when he arrives at the conviction that envy is ignorance; that imitation is suicide; that he must take himself for better, for worse, as his portion; that though the wide universe is full of good, no kernel of nourishing corn can come to him but through his toil bestowed on that plot of ground which is given to him to till. The power which resides in him is new in nature, and none but he knows what that is which he can do, nor does he know until he has tried.
I won’t harp on the life and times of Steve Jobs. That’s been done to death.
But I thought about Apple ($AAPL) because of today’s Dithering podcast episode, “Siri’s AI Ineptitude.”
As an accompanying piece, Dithering co-host John Gruber pounced into Siri and Apple Intelligence (spoiler: not so intelligent!) in his post “Siri is Super Dumb and Getting Dumber” on the Daring Fireball blog (read the link!).
But first, John’s friend Paul Kafasis wrote “Not So Super, Apple” on his One Foot Tsunami blog:
So, how did Siri do? With the absolute most charitable interpretation, Siri correctly provided the winner of just 20 of the 58 Super Bowls that have been played. That’s an absolutely abysmal 34% completion percentage. If Siri were a quarterback, it would be drummed out of the NFL.
Siri did once manage to get four years in a row correct (Super Bowls IX through XII), but only if we give it credit for providing the right answer for the wrong reason. More realistically, it thrice correctly answered three in a row (Super Bowls V through VII, XXXV through XXVII, and LVII through LIX). At its worst, it got an amazing 15 in a row wrong (Super Bowls XVII through XXXII). Most amusingly, it credited the Philadelphia Eagles with an astonishing 33 Super Bowl wins they haven’t earned, to go with the one 1 they have.
Okay, back to John Gruber and the Daring Fireball read:
It’s just incredible how stupid Siri is about a subject matter of such popularity. If you had guessed that Siri could get half the Super Bowls right, you lost, and it wasn’t even that close.
…
Super Bowl winners aren’t some obscure topic, like, say, asking “Who won the 2004 North Dakota high school boys’ state basketball championship?” — a question I just completely pulled out of my ass, but which, amazingly, Kagi answered correctly for Class A, and ChatGPT answered correctly for both Class A and Class B, and provided a link to this video of the Class A championship game on YouTube. That’s amazing! I picked an obscure state (no offense to Dakotans, North or South), a year pretty far in the past, and the high school sport that I personally played best and care most about. And both Kagi and ChatGPT got it right. (I’d give Kagi an A, and ChatGPT an A+ for naming the champions of both classes, and extra credit atop the A+ for the YouTube links.)
…
Old Siri — which is to say pre-Apple-Intelligence Siri — does OK on this same question. On my Mac running MacOS 15.1.1, where ChatGPT integration is not yet available, Siri declined to answer the question itself and provided a list of links, search-engine-style, and the top link was to this two-page PDF listing the complete history of North Dakota’s Class A boys’ and girls’ champions, but only through 2019. Not great, but good enough.
New Siri — powered by Apple Intelligence™ with ChatGPT integration enabled — gets the answer completely but plausibly wrong, which is the worst way to get it wrong. It’s also inconsistently wrong — I tried the same question four times, and got a different answer, all of them wrong, each time. It’s a complete failure.
…
But it’s even worse than that, because old Siri, without Apple Intelligence, at least recognizes that Siri itself doesn’t know the answer and provides a genuinely helpful response by providing a list of links to the web, all of which contain accurate information pertaining to the question. Siri with Apple Intelligence, with ChatGPT integration enabled, is a massive regression.
Maybe Apple will course-correct Apple Intelligence as it did with Apple Maps.
As a corollary, the general public might not anoint OpenAI’s ChatGPT as the eponym of AI search.
I still prefer Anthropic’s Claude to OpenAI’s ChatGPT. I may change my mind by the end of the year, or I may not.
From TechCrunch:
Anthropic has reportedly raised around $1 billion from Google as the AI company looks to deliver a number of major product updates this year.
First reported by the Financial Times, Google’s fresh investment brings the tech giant’s total stake in Anthropic to around $3 billion. Google poured $2 billion into Anthropic late last year.
Anthropic, which is also in the process of raising up to $2 billion from investors including Lightspeed at a $60 billion valuation, has an ambitious roadmap for 2025.
In a series of interviews yesterday, Anthropic CEO Dario Amodei said that the company plans to launch new AI models, bring “two-way” voice chat and web access to its chatbot, Claude, and roll out an AI system called the “Virtual Collaborator.” According to Amodei, the Virtual Collaborator will run on PCs, execute workflows, write and compile code (and verify the results), and interact with users through apps like Slack and Google Docs.
According to Crunchbase, the outfit has now raised $14.7 billion altogether.
I never cared for the dubbing of the Magnificent Seven stocks: Apple, Alphabet ($GOOGL), Amazon ($AMZN), Microsoft ($MSFT), Meta ($META), Nvidia ($NVDA), and Tesla ($TSLA).
It’s a lazy and dull name, while its film inspirations, “Seven Samurai” (1954) and “The Magnificent Seven” (1960), were incredible movies that were each never lazy or dull.
I won’t bet against a $3 trillion company like Apple.
But I’m not in the mood to bet in their favor, either.
Watch: “Steve Jobs” (2015):
Steve Wozniak: I get a free pass for life? From you? You’re the one who gives out the passes?! You give ’em to me?!!
Steve Jobs: You’re gonna have a stroke, lil’ buddy.
Steve Wozniak: What did YOU do?! What did you DO?! Why has Lisa not heard of me?
Steve Jobs: Shit, man, how many fourth-graders have heard of you?
Steve Wozniak: You can’t write code. You’re not an engineer, you’re not a designer, you can’t put a hammer to a nail. I built the circuit board, the graphical interface was stolen from Xerox PARC, Jeff Raskin was the leader of the Mac team before you threw him off his own project… everything! Somebody else designed the box! So how come 10 times in a day I read Steve Jobs is a genius? What do you do?
Steve Jobs: I play the orchestra. And you’re a good musician. (pointing) You sit right there. You’re the best in your row.
Steve Wozniak: I came here to clear the air. You know why I came here?
Steve Jobs: Didn’t you just (answer that?)
Steve Wozniak: (over) I came here ’cause you’re gonna get killed. Your computer is going to fail. You had a college and university advisory board telling you they need a powerful workstation for $2,000 to $3,000, you’ve priced NeXT at $6,500, which doesn’t include the optional $3,000 hard drive which people will discover isn’t optional because the optical disk is too weak to do anything and the $2,500 laser printer brings the total to $12,000 and in the entire world, you are the only person who cares that it’s housed in a perfect cube. You’re gonna get killed and I came here to stand next to you while that happens because that’s what friends do, that’s what men do, I don’t need your pass. We go back so don’t talk to me like I’m other people. I’m the only one who knows that this guy here is someone you invented. I’m standing by you ’cause that perfect cube that does nothing is about to be the single biggest failure in the history of personal computing.
Jobs wants to tear into Wozniak and he considers it for a quick moment but instead of doing that he says this…
Steve Jobs: (pause) Tell me something else I don’t know.
Jobs opens the door and walks out…
