Quick note on the definition of AI

First published on my old blog in May 2022.

As a final-year student with some experience in artificial intelligence (AI) and machine learning (ML), I was glad to have a chance to revisit the definition of AI through Stanford University’s introductory AI course, CS221, whose content is available online. It was also a chance to look back at the Introduction to AI course I took in my second year at Hanoi University of Science and Technology (HUST).

What AI and ML meant to me before this course

After many days of reading about the definition of AI in my early years at university, I finally settled on a definition that seemed general enough. In my own words: “Artificial intelligence is any kind of simulation that exhibits a feature of intelligence.” This definition works for me for a few reasons:

  • Some definitions equate intelligence with human intelligence, which I think limits them. Animals show some kinds of intelligence, and there may be kinds of intelligence that humans don’t have.
  • Simulations are generally created by humans, but “simulation” here simply means “artificial”: not occurring in nature.
  • There are many types of intelligence and abilities associated with it (Wikipedia lists abstraction, logic, understanding, learning, emotional knowledge and more), and there is no single definition of intelligence. So exhibiting one of those traits is enough.

Tom M. Mitchell’s definition of machine learning, from his book Machine Learning, is widely accepted:

“A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E.”

ML is often regarded as a subset of AI. The difference is that ML has an element called learning: performance improves with more data or experience.

My unanswered question

So what was the problem? I already had a definition I was happy with. What interested me next was this: how can we categorise AI and ML in a way that aligns with aspects of natural intelligence? There are already some common ways of categorising them:

  • By learning method: supervised, unsupervised and reinforcement learning, and so on
  • By type of task: computer vision, natural language processing, speech processing, and so on

These groupings are useful, but I wasn’t satisfied, because they feel unnatural:

  • They split AI and ML problems according to human-defined concepts (supervised learning, for example) or human functions (seeing becomes computer vision, speaking becomes speech processing). This ignores general aspects of intelligence, such as logic and abstraction.
  • There is little to compare between the groups. There doesn’t seem to be a scale from “low” to “high” on which to measure the categories against each other.

The Introduction to AI course at my university, and my doubt

These were the topics in my AI class:

  • Search problems: tree search (BFS, DFS), heuristic methods, constraint satisfaction problems (CSPs)
  • Logic: propositional logic, first-order logic
  • Machine learning: some common methods (kNN, naive Bayes, decision trees, neural networks)

Studying the course made me feel there was some kind of ordered categorisation here. Maybe the difficulty of the problems was the scale I was looking for? Or was the scale based on how abstract the problems are?

At the beginning of the course there was an overview of the history of AI. However, we students were never told why the course was structured that way; we only knew the topics.

What I learned and deduced from Stanford’s AI course

First, from the lecturer’s point of view, AI rests on three pillars: modelling (creating a mathematical or logical representation of the real world), inference (answering questions about models, such as finding a shortest path or a winning strategy) and learning (turning an abstract model into a concrete one with the help of additional data).

The course groups AI models into four types, sorted from “low” to “high” levels of intelligence:

  • A reflex-based model is a fixed sequence of computations that produces the output. This corresponds to everyday reflexive tasks we do very quickly, such as recognising an object in the house.
  • A state-based model considers the available states and the actions possible in each state, and plans the step-by-step actions to follow. Some of our favourite games, such as chess and Go, fit this model, as do puzzles like the classic river crossing with a farmer, a wolf, a goat and a cabbage.
  • A variable-based model assigns values to variables subject to some factors or constraints, regardless of order. Sudoku can be modelled well in this category.
  • A logic-based model can digest heterogeneous information into syntactic and semantic representations and reason deeply using inference rules. Much of the everyday logical reasoning we do can be represented with this type of model.

So there is an ordered categorisation here! Great news for me. To make the ordering clearer, I went through the course and tried to build scales that organise the problems and models along the three pillars. Here is my own summary, based on what I learned from the course:

Model type Modelling Inference Learning
Reflex Abstract model, a “massive lookup table of best actions” Follow a fixed sequence of steps to get the output Has to learn from (lots of) data
States Local interactions between states (like in a graph) Combine local interactions to find optimal solutions Gets data through exploration
Variables Solutions are assigned to variables (lots of possibilities) Find assignments to variables, or do probabilistic inference Needs little data; incorporates prior domain knowledge
Logic Syntax and semantics, highly expressive Draw (deep) logical conclusions from knowledge, but rules can be deterministic Rule-based; doesn’t allow fine-tuning with data
Trend, from reflex to logic Models get simpler and more expressive, moving from abstract to concrete (less learning) Inference gets more complex and handles less uncertainty Learning gets easier, but there is less ability to fine-tune with additional data

Conclusion

In the end, I found what I was looking for: an ordered grouping that aligns with aspects of natural intelligence. This categorisation also filled the gap in my understanding of how my university’s Introduction to AI course was structured.

There are many more topics in CS221. I think it’s a very good course for anyone who wants an overview of what AI is, and a starting point for exploring specific areas of AI and ML.

References