When ChatGPT burst onto the scene in late 2022, it seemed to many like artificial intelligence had suddenly emerged, fully formed, to transform our world overnight. Headlines proclaimed the dawn of a new technological era, investors poured billions into AI startups, and both utopian enthusiasm and existential dread proliferated across social media.
But the truth is far more complex—and more interesting. Today’s AI breakthroughs aren’t the product of sudden inspiration or a single laboratory breakthrough. They represent the culmination of over seven decades of persistent research, punctuated by crushing disappointments, funding collapses, and the tireless work of researchers who toiled in obscurity during periods when artificial intelligence was considered a failed dream.
The Boom-Bust Cycles of AI
The field of artificial intelligence has not had a smooth path. There have been at least two major periods—the AI winters—where funding dried up, students were advised not to enter the field, and skepticism about the entire enterprise of AI was at its peak.
These “winters” weren’t minor setbacks but existential threats to the entire field. The first major AI winter descended in the mid-1970s, freezing a decade of optimism that had begun with the 1956 Dartmouth Workshop—the event where the term “artificial intelligence” was coined. Early AI researchers had made promising advances in areas like theorem proving and checkers-playing programs, leading to a surge of government funding, particularly from the Defense Advanced Research Projects Agency (DARPA).
The confidence in AI’s potential was extraordinarily high during this early period. Herbert Simon, a pioneer in AI, made bold predictions in 1965 that machines would be capable of doing any work that humans could do within twenty years. Such forecasts would come back to haunt the field.
By the mid-1970s, the limitations of existing approaches became painfully apparent. AI systems that worked in controlled laboratory settings failed in real-world applications. The fundamental issue wasn’t just insufficient computing power, but more importantly, the limitations of rule-based and symbolic reasoning approaches that dominated early AI. These systems couldn’t effectively handle uncertainty, learn from experience, or generalize their knowledge to new situations. The symbolic AI paradigm simply didn’t scale to handle the complexity and messiness of real-world problems.
DARPA’s frustration with the lack of practical results led to severe funding cuts. Meanwhile, a damning 1973 report by British mathematician Sir James Lighthill criticized the field’s progress, concluding that AI research had failed to live up to its promises. This assessment led the British government to cut funding for AI research at all but two universities.
A second boom began in the early 1980s with the rise of “expert systems”—programs that captured the knowledge of human specialists in fields like medicine and engineering. Companies invested heavily in this technology, creating a commercial AI industry for the first time. Japan’s ambitious Fifth Generation Computer project aimed to leapfrog American computing with massive investments in AI and advanced computing architectures.
But by the late 1980s, this wave also crashed. Expert systems proved brittle, expensive to maintain, and unable to learn or adapt. Companies that had invested millions watched their specialized hardware and software become obsolete as conventional computing advanced. By 1993, funding for broad AI initiatives had again collapsed, ushering in a second, deeper AI winter that would last until the early 2000s.
It’s worth noting that expert systems didn’t disappear entirely. Some applications remained successful, particularly in narrow domains such as medical diagnosis, financial assessment, and manufacturing quality control. However, the broader AI field began shifting toward statistical and probabilistic approaches that could better handle uncertainty and learn from data.
The Overlooked Pioneers
During these winters, a small group of researchers continued to advance AI, often with minimal funding and recognition. Their persistence would eventually enable today’s breakthroughs, yet many remain largely unknown outside technical circles.
Judea Pearl developed probabilistic and causal reasoning methods in the 1980s that enabled machines to reason with uncertainty—a fundamental capability for any system operating in the real world. While his work is essential to many modern AI applications, from medical diagnosis to autonomous vehicles, Pearl isn’t a household name like Elon Musk or OpenAI’s Sam Altman.
Even less recognized in popular accounts is the work of Kunihiko Fukushima, a Japanese researcher who in 1979 developed the “neocognitron,” a neural network inspired by the visual cortex that implemented many concepts central to today’s convolutional neural networks, which power modern computer vision. While Fukushima’s work was well-known in academic circles, it wasn’t practically implementable at the time due to hardware limitations. It would take decades before computing power caught up with his theoretical insights, allowing his ideas to be fully realized in modern deep learning systems.
Perhaps most tragically overlooked are the contributions of women pioneers in the field, many of whom made foundational contributions only to be forgotten by popular histories. Women have been at the forefront of computing and AI since the beginning, but their contributions are often minimized or forgotten entirely.
Take Mary Kenneth Keller, who in 1965 became the first woman in the United States to earn a PhD in Computer Science. While she didn’t develop BASIC (which was primarily created by John Kemeny and Thomas Kurtz at Dartmouth College), she made significant contributions to computing education and founded a computer science department at Clarke College. Or Annie Easley, an African-American mathematician who worked at NASA from 1955 to 1989, developing and implementing code for analyzing alternative power technologies and supporting the Centaur rocket stage.
There’s also Fei-Fei Li, whose ImageNet project created a dataset of over 14 million labeled images that proved crucial to advances in computer vision. While Li has gained recognition in recent years, for much of her career, her groundbreaking work received little public attention.
The public faces of AI today are often business leaders and entrepreneurs, but the foundations they’re building upon were laid by academic researchers, many of whom never sought or received recognition outside their specialized communities.
The Quiet Revolution Behind the Scenes
While AI disappeared from headlines during the winters, critical work continued that would eventually enable today’s breakthroughs. Three parallel developments—increasing computational power, growing datasets, and algorithmic innovations—were slowly converging to create the conditions for an AI renaissance.
The steady march of Moore’s Law meant that by the 2010s, the computing power needed to train large neural networks had become accessible. Meanwhile, the internet was generating vast amounts of data—text, images, videos—that could serve as training material. And researchers were refining algorithms that could take advantage of these resources.
Among the most important algorithmic advances was the backpropagation algorithm for efficiently training neural networks. Though its basic principles were understood in the 1960s, it was refined and popularized in a 1986 paper by David Rumelhart, Geoffrey Hinton, and Ronald Williams titled “Learning representations by back-propagating errors.”
Hinton, now sometimes called the “godfather of deep learning,” continued working on neural networks throughout the second AI winter when the approach was deeply unfashionable. For a long time, it was just Hinton and fellow researchers Yoshua Bengio and Yann LeCun who persisted in advancing deep learning when most of the field had moved in other directions. Their persistence was eventually recognized when they jointly won the 2018 Turing Award.
In 2012, the quiet revolution burst into public view when Hinton’s team won the ImageNet visual recognition competition, reducing error rates dramatically using deep neural networks. This moment is often cited as the beginning of the current AI boom, but it represented the culmination of decades of incremental progress rather than a sudden breakthrough.
Success in machine learning requires three components: the models (neural networks), computational resources, and data. The big breakthrough in 2012-2013 was that all three of these elements finally converged at sufficient scale. Critically, the use of Graphics Processing Units (GPUs), particularly those from NVIDIA, was a game-changing factor. Before this hardware innovation, neural networks were simply too slow to train at scale. GPUs, with their highly parallel architecture originally designed for video games and graphics, proved to be remarkably well-suited for the matrix operations underlying neural networks, making deep learning computationally feasible for the first time.
The Lessons of the AI Winters
Today’s AI landscape bears little resemblance to the barren funding environment of the winters. Investment is booming, with private investment in AI reaching $91.9 billion in 2023 alone. Tech giants and startups alike are racing to deploy AI systems in every industry.
But veterans of previous cycles urge caution. There are echoes of previous hype cycles in today’s discourse. We see the same pattern of overestimating what AI can do in the short term while potentially underestimating long-term impacts.
Indeed, some of the same challenges that triggered previous winters remain unsolved. While large language models like GPT-4 produce impressive results, they still struggle with reasoning, often generating plausible-sounding but factually incorrect information—a phenomenon called “hallucination.” The systems’ inability to explain their decisions creates challenges for critical applications like healthcare and criminal justice.
Researchers are also confronting the enormous computational resources required by current approaches. Training a single large language model requires significant energy and can be expensive, with estimates ranging from tens to hundreds of millions of dollars for the largest models. The environmental impact of AI training varies considerably depending on the energy sources used. Training centers powered by renewable energy have much smaller carbon footprints than those using fossil fuels. Additionally, these calculations often don’t account for potential efficiency gains AI might bring to other industries, which could offset initial environmental costs.
Despite these challenges, there are reasons to believe the current AI summer may be more durable than its predecessors. First, today’s AI systems are delivering real economic value in specific applications from content generation to customer service. Second, the technical foundation is broader, with progress occurring across multiple methodologies rather than a single approach. Finally, the commercial ecosystem is more diverse, with applications spanning virtually every industry rather than being concentrated in a few sectors.
Standing on the Shoulders of Giants
The story of AI’s development offers a powerful reminder that technological revolutions rarely happen overnight. ChatGPT and similar systems didn’t emerge from nowhere—they represent the culmination of decades of research across multiple disciplines, including computer science, linguistics, psychology, and neuroscience.
The fundamental ideas behind today’s AI systems date back many decades. Progress comes from persistence. The researchers who continued working on neural networks and machine learning during the AI winters, when it was unfashionable and underfunded, laid the groundwork for the current revolution.
As we navigate the opportunities and challenges of the current AI era, we would do well to remember this longer history. The overnight success of artificial intelligence was, in reality, seventy years in the making.
Disclaimer: This article represents reporting on the history of artificial intelligence and does not advocate for any particular approach to AI development or regulation. It is intended solely to provide historical context for current developments in the field.
