Reddit Isn’t Just Suing Perplexity — It’s Challenging AI’s Entire Business Model

Reddit has become a top-cited source in artificial intelligence chatbots and a valuable repository for training their underlying language models. While the company has agreed to license its vast volumes of data to OpenAI and Google, it’s now taking legal action against AI companies that don’t pay up.

In June, Reddit sued Anthropic for scraping its data to train its chatbot, Claude. And in October it filed a separate lawsuit against AI search engine Perplexity, as well as middlemen who scrape data from the web and sell it to other AI companies: Texas-based SerpApi, Lithuania-based Oxylabs UAB and AWM Proxy, a former Russian botnet. SerpApi has advertised its relationship with Perplexity, according to the lawsuit.

In the lawsuit, Reddit’s lawyers allege the defendants swiped its content from Google search results  — an indirect means of accessing content on Reddit, which prohibits commercial use of its content without an agreement. Reddit and Google have both taken measures to prevent unauthorized scraping, but these data-scraping companies have found ways to circumvent Google’s tools.

“In a very real sense, these Defendants are similar to would-be bank robbers, who, knowing they cannot get into the bank vault, break into the armored truck carrying the cash instead,” Reddit’s lawyers wrote in the lawsuit.

Perplexity knew its actions were illegal, the complaint alleges, because Reddit sent the company a cease-and-desist order in May 2024. Perplexity denied scraping Reddit’s data and said it honors robots.txt files, which are embedded files that tell data-scraping bots what’s off-limits.

Since then, Reddit found a forty-fold increase of its content in Perplexity’s search results. To confirm Perplexity was scraping its data, Reddit created a post that only Google could see. Within hours, Reddit’s staff saw the post incorporated into Perplexity’s search results.

Publishers Fight Copyright Violations

In an earlier version of the internet, website publishers may have paid data scraping companies to help boost their rankings within Google search results. But AI companies — including Google itself — have destroyed this model by scraping and repackaging others’ content as their own, drawing users away from the original source and toward their subscription AI products.

Several online publishers — including The New York Times, Dow Jones and Getty Images  — are suing AI companies for stealing their copyrighted content. Social media companies, such as Meta, X and LinkedIn, have also filed lawsuits against data-scraping companies for using their data.

Meanwhile, online publishers have seen steep traffic declines due to Google’s AI Overview product, which produces AI-generated query responses above the regular search results. Since Google AI Overview launched May 2024, the percentage of searches resulting in no clicks or website visits have increased from 56 percent to 69 percent, according to a Similarweb study. At the same time, organic traffic has dropped from 2.3 billion visits to 1.7 billion visits.

The Rise of AI Licensing Deals

Many of the publishers that have sued AI companies for unauthorized scraping have also entered into paid licensing agreements with these AI companies. The New York Times, News Corp and numerous other publishers — including Reddit — have signed licensing deals with AI companies. 

Reddit, which has licensing agreements with OpenAI and Google, started blocking access and brokering licensing deals in 2023, when it realized how valuable its content would be to generative AI tools. Since ChatGPT launched in late 2022, new companies have poured hundreds of millions of dollars into computing power and AI researchers with the hopes of winning the competitive AI arms race. Training an AI model requires large troves of training data, though, which can be difficult to access. 

Reddit has become a valuable repository of knowledge for AI companies. For 20 years, its anonymous users have posted freely and candidly about their experiences, opinions and specialized knowledge on nearly every topic imaginable, with an upvoting system elevating content deemed most valuable to each community, or subreddit. One AI executive told The Wall Street Journal last year that Reddit’s corpus of content is “like manna from heaven. All they have to do is package the data set and hire salespeople.”

In fact, Reddit is the top-cited source on Perplexity and Google AI Overviews, according to a report by AI search analytics company Profound, and it’s the second-most-cited source on ChatGPT.

Instead of entering into a licensing agreement for Reddit’s data, the lawsuit alleges Perplexity “will apparently do anything to get the Reddit data it desperately needs.” In a Reddit post responding to the lawsuit, Perplexity claimed it would be “impossible” to sign a licensing agreement with Reddit, as it doesn’t train foundation models. The company said its chatbot summarizes and cites Reddit discussions in its responses, and that Reddit’s argument represents “the opposite of the open internet,” as it’s trying to stop Perplexity users from finding content through the chatbot.

Frequently Asked Questions