Search Engine Turbulence

This is a March 2023 snapshot of the early LLM-search moment. Some of it aged well; some of it aged in weeks. Dating notes appear inline.

If you have an AI idea that you want to get funded, now is your time to try. VCs have been pouring billions into AI startups at an accelerating rate for years, and since ChatGPT went mainstream in December, the acceleration has been reaching a fever pitch. ChatGPT reached 1 million users in its first five days and was estimated to be the fastest-growing consumer app on record, hitting roughly 100 million monthly users within two months, and interest in it from the public has only increased.

There's a meme now that all you have to do to raise money from VCs is build a UI wrapper around a ChatGPT function—like an app that gives you customized recipes or helps you code. Even Snapchat is releasing a new chatbot with ChatGPT on the backend. It will soon be refrigerators and washing machines.

But as Balaji put it:

"Not your API keys, not your AI."

— Balaji Srinivasan (@balajis) February 24, 2023

Tech companies are feeling the competition.

Google's co-founder, Sergey Brin, came out of the shadows to submit his first request for code access in years to none other than LaMDA - Google's natural language chatbot.

This was a natural reaction to the moves Microsoft has been making with Bing. Microsoft CEO Satya Nadella made his position clear in interviews: Microsoft intended to compete aggressively in search.

Let's talk about Bing

For those of you unfamiliar with Bing's background, Trung Phan laid out its history in his newsletter. Bing now incorporates LLMs into its search engine to answer questions and provide a genuinely new kind of browsing experience.

Of course, there are variations of this already in the wild, but the fact that the biggest companies in the world are adopting this is a big signal.

Notion's AI, GitHub's Copilot, Replit's AI, Meta AI, and Snapchat's new chatbot - to name a few - signal how crowded the field is getting.

So why is OpenAI valued at $29 billion?

Well, at that moment theirs was the model everyone else was measured against.

Public Perception

The public's reaction is mixed. Some see Sam Altman's vision of the AI-enabled future, and others feel Eliezer Yudkowsky's fears of the unknown.

In the past few decades, there have been stories of individuals who began fearing imminent human extinction due to rapid AI progress.

They typically liquidate all their assets and indulge recklessly, only to realize they've acted prematurely a few years later. Others fear it will take their jobs, but it remains to be seen which industries will be massively interrupted first. Some AI experts, such as Yann LeCun, say these models are insufficient at their core, and we need new architecture to reach the coveted "artificial general intelligence" level of machine reasoning.

However, AI advancement may slow down as it historically has, following the typical S-curve of expertise acquisition.

ChatGPT, while impressive, still hallucinates - contemporaneous estimates put factual-error rates in the 10-20% range depending on the task, with no standard way to measure it - which made it hard to trust for search or medicine. Given the nature of S-curves, some speculate that the progress AI made through early 2023 will be the most it experiences until the next leap forward into the steep part of the curve - maybe quantum computing gets us there.

This makes sense, given that these models have been trained on nearly all the public data available. If stagnation occurs, we'll see history repeat itself, as that happened during the AI winters of 1974-1980 and 1987-1993 after massive leaps forward in the industry took place.

Unintended Consequences

The unintended consequences are hard to quantify, but a few are clear. Anecdotally, Bing has returned strange things to me, including "I would rather us not have this conversation anymore" instead of giving me a wrong answer - as ChatGPT, You.com, or Poe would. Note the word return - I am not going to anthropomorphize these chatbots.

It's not responding to me or talking to me. Prompts are the same as binary code to these things, and their entire existence is a compilation of code with emergent properties that stem from pattern recognition.

On the other hand, observing Bing's use of emojis is uncomfortable.

The Future of AI and Search

In interviews, Satya clarified that improving Google's search model has been daunting.

They are monolithic and, until ChatGPT, have had no competition. He sees ChatGPT and Bing as the first real competitors of Google's prowess. Combine this with the fact that Nvidia's CEO went on the record saying their chips will enable models to be 1 million times the power of current AI models in the next decade.

ChatGPT was trained on an estimated 10,000 Nvidia GPUs.

With Edge and Bing only amounting to a fraction of Google's market share, what do Satya and Microsoft have to lose?

Every underdog story has a Goliath.

This time, it's Google.

It's easy to forget that all of the recent progress in AI wouldn't have happened without Google. Transformers are considered game changers in the application of AI models, and all of these chatbots were trained on networks that incorporate transformers. Google may speed up its historically slow software development cycle and blow everybody out of the water - or one of the labs nobody is pricing in will.

Inspiration

In his 1962 speech, John F. Kennedy talked about landing on the moon within a decade.

Unfortunately, he didn't see it, but the US landed on the moon in 1969.

The space age saturated entertainment around the same arc - Star Trek (1966) and Kubrick's 2001: A Space Odyssey (1968) rode the run-up, Star Wars (1977) and Alien (1979) the aftermath.

There's a correlation there.

At first, AI's most significant societal changes will be on each edge of the barbell, inspiration, and fear - similar to Sergey Brin's and Eliezer Yudkowsky's reactions.

Then, unintended consequences that no one can predict will start happening, and that's where it can get out of hand. The long-term societal implications of human-AI interaction patterns remain uncertain and warrant careful observation.