The History of AI Models - Part 1: Foundations & Early Neural Networks (1950s - 1990s)

Today, when we say Artificial Intelligence (AI), massive large language models like ChatGPT, Gemini, or Llama immediately come to mind. However, reaching this point is the result of a turbulent scientific journey that has spanned decades. In this four-part article series prepared by Canary Digital, we will examine the historical evolution of AI models step by step.

In our first part, we focus on the years where it all began, spanning from the 1950s to the late 1990s.

1950s: The Birth of the Idea and the Turing Test

The concept of artificial intelligence blossomed during the years when modern computers still occupied massive rooms. Published in 1950, Alan Turing’s paper “Computing Machinery and Intelligence” raised the question of whether machines could think. This concept, known as the Turing Test, aimed to measure whether a machine could exhibit intelligence equivalent to, or indistinguishable from, human intelligence.

In 1956, the term “Artificial Intelligence” was officially coined for the first time at the Dartmouth Conference. During this period, researchers believed that machines could solve logical problems and play games like chess.

1960s and the First Perceptron

Considered the ancestor of modern neural networks, the Perceptron was invented by Frank Rosenblatt in 1957. Inspired by the way neurons in the human brain work, this algorithm was the first attempt at an artificial neural network that could take visual inputs and make basic decisions (e.g., recognizing simple shapes).

Although the invention of the Perceptron created great excitement, it was soon realized that its capacity was limited. Published in 1969, Marvin Minsky and Seymour Papert’s book Perceptrons mathematically proved that these simple models could not solve non-linear problems like “XOR”. This situation caused a massive decline in interest in artificial neural network research.

The First “AI Winter”

Exaggerated expectations combined with the inadequacy of computer hardware at the time led to the cutting of AI funding in the 1970s. During this period, known as the “AI Winter”, research in the field almost came to a halt.

1980s: Expert Systems and Backpropagation

By the 1980s, interest in artificial intelligence was revived thanks to “Expert Systems”. These systems relied on coding the logical rules (if-then rules) of human experts in a specific field into the computer. However, these systems could not adapt to new situations and could only operate within the framework of the rules they were taught.

The real revolution occurred when neural networks returned to the stage in the mid-1980s. With the discovery of the Backpropagation algorithm (popularized by Geoffrey Hinton, David Rumelhart, and Ronald Williams in 1986), training neural networks became possible. This algorithm allowed the network to calculate the errors it made backwards and correct the connection weights. Multi-Layer Perceptrons could now solve complex problems.

1990s: The Rise of Machine Learning

The 1990s were the years when the transition from rule-based systems to “data-driven” systems accelerated. IBM’s Deep Blue computer defeating world chess champion Garry Kasparov in 1997 demonstrated the potential of artificial intelligence to the whole world.

During the same period, Yann LeCun developed the first successful Convolutional Neural Network (CNN - LeNet-5) capable of recognizing handwritten digits. However, what was needed for these models to reach their true potential was not yet available: massive amounts of data and powerful hardware to process that data.

These shortcomings were the harbinger of the revolution that would take place in the 2000s. In the second part of our series, “The Deep Learning Revolution”, we will examine the emergence of GPUs and how algorithms began to change the world.