All About the Foundational Tech Behind the 2024 Turing Award Winners

Photo Source: Unsplash

The latest Turing Award went to two computer scientists who worked at the University of Massachusetts Amherst beginning in the 1970s. The Alan M. Turing Prize, often called the Nobel of Computing, comes with a $1 million award, and each year, it goes to pioneers in computing. Famously, Turing decyphered the German encryption machine dubbed the Enigma, which substantially helped the Allies defeat the Axis Powers in World War II. Turing also theorized the foundations for artificial intelligence and built one of the first general-purpose computers after the war. In commemoration of Turing's fundamental contributions to computer science, an award in his name goes to great contributors to computer science.

The 2024 Turing Award went to Professor Andrew G. Barto, University of Massachusetts Amherst Manning College for Information and Computer Sciences professor emeritus, and his then PhD graduate student, Richard S. Sutton, now a professor of computer science in Canada at the University of Alberta. The award goes to them for their pioneering work in a type of artificial intelligence called reinforcement learning. (cics.umass.edu) According to Barto, UMass Amherst, in the late 1970s, gave him and his graduate students the freedom to explore and develop their ideas on how machines can learn. Together, Barto and Sutton wrote the foundational papers on reinforcement learning and, in 1998, authored a book titled Reinforcement Learning: An Introduction, which is now in its second edition and published by Bradford Books.

Reinforcement learning forms a branch of AI that allows machines to learn from their successes and failures to improve their performance at a given task. In reinforcement learning, computers interact with an environment such as a game like GO or poker or a task such as picking stocks for investment portfolios. The program can make moves such as buying or selling stocks based on different criteria such as financial information and then monitor the stock's price over time. If the investments add value to the machine's stock portfolio, the program gets a numerical reward, and the machine can use similar criteria in picking future stocks. The portfolio goes down; the machine gets punished with a negative score. Either way, the machine learns and adjusts its next moves based on past results. More importantly, the machine can practice on historical data or simulated purchases. Because machines work millions of times faster than a person, the machines can gain significant experience with decades of data in hours or even minutes. Learning by doing sits at the core of reinforcement learning.

According to Barto, reinforcement algorithms did not catch on immediately, but the environment at UMass Amherst worked well for these pioneers. Today, reinforcement learning does more than win games such as GO. Combining knowledge from experience with other AI techniques, such as large language models (LLMs) like ChatGPT or Gemini, has opened the door to bridging the knowledgeable strengths of LLMs with practical, real-world challenges with reinforcement learning. For example, human evaluation of the quality of LLM responses can feed back into the LLM through the reinforcement learning scoring system. Such techniques help LLMs improve. In a more automated fashion, reinforcement learning will help LLMs to finetune medical conversations for more accurate capture of patient symptoms leading to better diagnosis and treatment recommendations. The possibilities are limitless and open the door to AI and robots to operate in the world and learn from their experiences with or without humans in the loop.   

Dr. Smith’s career in scientific and information research spans the areas of bioinformatics, artificial intelligence, toxicology, and chemistry. He has published a number of peer-reviewed scientific papers. He has worked over the past seventeen years developing advanced analytics, machine learning, and knowledge management tools to enable research and support high-level decision making. Tim completed his Ph.D. in Toxicology at Cornell University and a Bachelor of Science in chemistry from the University of Washington.

You can buy his book on Amazon in paperback and in kindle format here.