Article: How Computers Have Become Better Conversationalists Than Ever

Photo Source: Pixabay

When it comes to holding a conversation with a computer, things have truly changed. New computer programs called natural language processing models or NLPs have begun to excel in speech and question answering. Over the past few years, researchers have found that with these NLP models bigger is better, and the arms race continues with larger and larger models getting released every few months or so. For example, a company called OpenAI based in San Francisco developed a large-scale language model called GPT-3 capable of generating human-like text. GPT stands for “Generative Pre-trained Transformer,” and GPT-3 represents the third generation of OpenAI’s groundbreaking GPT models.

GPT-3 uses a transformer-based program to generate text from a given statement or question. In other words, a transformer learns from text by reading though millions of documents and periodically hiding words from itself and trying to guess what that word should be. When the computer guesses a word correctly it strengthens itself. Over the course of billions of challenges, the program better understands and replies to human language knowing which words make the most sense together. To put model size in perspective, GPT trained on 45 terabytes of text from such sources as the entire internet and a books database called Books1. One terabyte of text equals 83.3 million pages of text. (Alaska.edu) The Library of Congress, the largest library in the world, contains 10 terabytes of text, meaning that GPT-3 used more than four times the contents of the Library of Congress to learn human language. The author, Giani Statie of “Artificial Intelligence,” determined that one training run in the for GPT-3 with computer hardware and electricity cost $4.6 million dollars. (heits.digital)

GPT-3 has a public “Playground” that anyone can experiment with once they create an account. (beta.openai.com/playground) GPT-3 can perform a wide range of tasks, including question answering, translating plain speech into computer code, writing restaurant reviews with a few key words, tweet classification, make an essay outline, and much more. I tried the essay outline with the following question written to GPT-3:

Create an outline for an essay about Andy Warhol and his contributions to popular culture and art:

GPT-3 wrote back the following essay outline:

I. Introduction

A. Overview of Andy Warhol and his impact on popular culture and art

II. Early Life and Career

A. Warhol's childhood and upbringing

B. Warhol's early career and rise to fame

III. Warhol's Contributions to Art

A. Pop Art and its influence

B. Warhol's use of everyday objects and images

C. Warhol's use of color and repetition

IV. Warhol's Contributions to Popular Culture

A. Warhol's influence on fashion and design

B. Warhol's influence on music

C. Warhol's influence on film and television

V. Conclusion

A. Summary of Warhol's contributions to art and culture

The recommended essay structure shows clear organization and coverage of the different areas of culture Warhol influenced.

Large language models use a type of artificial intelligence called natural language processing and, more specifically, transformer-based programs. Such transformer-based programs require unbelievable amounts of text data to learn human language. Using a technique that the computer periodically leaves words out of sentences and then tries to guess the right one, allows the program to strengthen itself every time it guesses correctly. Such learning has generated models that increasingly produce natural, human-style questions and answers. However, such models require text of staggering size. GPT-3 from OpenAI used four times the number of pages of text than the largest library in the world contains. Since GPT-3’s release a few years ago, newer models have emerged such as Google’s PaLM that stands at five times larger than GPT-3. GPT-3 makes mistakes and even the CEO of OpenAI, Sam Altman, put a little water on the hype-fire in a tweet that said, “ The GPT-3 hype is way too much. It’s impressive (thanks for the nice compliments!) but it still has serious weaknesses and sometimes makes very silly mistakes.” (@sama, July 19, 2020) Weaknesses and all, give GPT-3 a try for yourself and you make a judgement on how well computers can talk to you.

Dr. Smith’s career in scientific and information research spans the areas of bioinformatics, artificial intelligence, toxicology, and chemistry. He has published a number of peer-reviewed scientific papers. He has worked over the past seventeen years developing advanced analytics, machine learning, and knowledge management tools to enable research and support high-level decision making. Tim completed his Ph.D. in Toxicology at Cornell University and a Bachelor of Science in chemistry from the University of Washington.

You can buy his book on Amazon in paperback and in kindle format here.