Article: Will AI Leave Us in a Post-Facts World?

Photo Source: Pexels

Will large language models (LLMs) such as ChatGPT, Anthropic’s Claude, and Google’s Gemini usher us into a post-factual world? Unless we can find robust and widely applied fact-checking systems, the nature of facts will change due to the incredible speed and reach of (mis)information across the world through social media, the internet, text messages, and email. Misinformation can present details about people, political parties, and events that look true due to the persuasiveness of the message, image, or video manipulation to look like a genuine or seemingly legitimate source. 

LLMs have created a wide variety of opportunities to spread false information. The following three examples demonstrate how LLMs have facilitated the spread of misinformation. First, LLMs tend occasionally to hallucinate or provide well-written answers to questions with completely false facts included in the response. For example, The Washington Post reported that the technology website CNET had used an internally developed model to produce 77 articles for their website, some of which contained significant errors such as an explanation of compound interest where 3% on $10,000 would pay $10,300 after one year. (washingtonpost.com) The errors emerged after a fact-checking group called Futurism flagged the article, prompting an audit that found a number of other CNET articles containing various errors injected by their AI. Second, users of LLMs can take advantage of the high-quality writing and suggestions to make more effective scams to defraud people of money or to create legitimate-sounding personas on social media. Researchers at Harvard used LLMs and other models to automatically develop scam emails that promised a Starbucks gift card to those who clicked a link in an email. (techtarget.com) They measured the effectiveness of the email by the number of students who fell for the scam with a combination of models that achieved an 80% click rate! Thirdly, some people, from authors to product sellers, have used LLMs to generate large numbers of fake reviews to improve the reputation of their products. While sock puppet accounts and faking reviews are not new, people can use LLMs to make more natural-sounding reviews that make it harder to differentiate between genuine and fabricated customers

Computer science researchers and companies such as Amazon and the parent company of Facebook and Instagram, Meta, have invested heavily in techniques to determine the authenticity of information generated by LLMs. One such tool, FActScores, analyzes generated text by breaking it down into little pieces and checking the integrity of each piece against a trusted source truth. (arxiv.org) For example, the authors looked at Bridget Moynehan, an actor whose profile was generated by ChatGPT. The profile correctly claimed that she is an actor but falsely attributed acting credits in Grey’s Anatomy and credit as a producer. The tool looked at each piece of the sentence, such as “Bridget Moynahan is a producer.” It then checked against a trusted Wikipedia profile to determine accuracy. Such a technique called atomizing works if the source of truth has integrity. However, with more content on the internet coming from LLMs, the sources of truth may continue to erode because tools like FActScores may have less reliable information to depend on, and it becomes costly to run fact-checkers on everything being added to the internet and social media. The great concern would come from a content generation containing hallucinations and misinformation growing faster than a fact-checking system can include it. That could leave us in a post-fact world. As more people leave unchecked misinformation on the internet, it will end up drowning out the actual truth. This means that we need to find new structures and ways to deal with misinformation and ensure that we put the verified facts first.

Dr. Smith’s career in scientific and information research spans the areas of bioinformatics, artificial intelligence, toxicology, and chemistry. He has published a number of peer-reviewed scientific papers. He has worked over the past seventeen years developing advanced analytics, machine learning, and knowledge management tools to enable research and support high-level decision making. Tim completed his Ph.D. in Toxicology at Cornell University and a Bachelor of Science in chemistry from the University of Washington.

You can buy his book on Amazon in paperback and in kindle format here.