Tech Giant Feared Its AI Tools Started a Commercial ‘Doom Loop,’ New Documents Reveal

AI executives allegedly knew their products could unlawfully harm businesses, flood the market with inferior work, and damage human society’s creative capacity as a whole, according to communications newly revealed in a lawsuit.

Those internal documents were cited in a lawsuit brought by The New York Times, Mother Jones, and other news outlets alleging OpenAI and Microsoft’s AI tools stole their work, and are harming their bottom lines.

“Millions of people around the world will soon consider large models ‘hoovering up’ all their work to be an astonishing theft of unprecedented proportions,” one Microsoft document said, according to a legal filing first reported by Mother Jones on Sept. 22.

The ‘Doom Loop’

One Microsoft document also allegedly indicated that the company was aware that the AI model outputs would result in fewer clicks on websites—including its own. Fewer clicks means less money, which disincentivizes creativity and leads to less, and maybe lower-quality, work.

“Our AI content strategy has started a ‘doom loop’ that will hurt the performance of our models and the entire web at the same time,” the document said.

In evaluating the suits, courts will look at how the new material produced by AI models affects the source material’s moneymaking capabilities.

According to the filing, Nick Turley, OpenAI’s head of ChatGPT, wrote that the company’s AI products pose an “existential threat” to publishers, “are largely substitutive, period” and “will get more and more substitutive as they get better.”

Those statements kill the AI companies’ “fair use” claim, attorneys for the news publishers wrote in their filing.

“The AI-generated outputs provide substitutive answers that reduce the need for users to visit Plaintiffs’ websites and flood the market with low-quality material that undermines the business model of news publishers,” the complaint reads.

One employee told OpenAI cofounder Greg Brockman that he’d developed “a hack” to get around The New York Times’ paywall, allowing ChatGPT to scrape up more stories.

“Ah, nice,” Brockman allegedly responded.

After the companies filed suit, the lawsuit says, OpenAI also developed a filter to stop ChatGPT from outputting from sources involved in the litigation.

“OpenAI did not suppress the output of content from any entity that had not sued it,” the lawsuit alleges. The filter was created “not to prevent OpenAI’s models from infringing copyrights, but to stop Plaintiffs from gathering evidence of OpenAI’s copying for use in litigation.”

An executive at Microsoft, the lawsuit claims, worried that might be an “accidental cover up” that would result in “people who have a right over the content having less visibility into what was used for training.”

Lawsuit

The publisher’s cases have been consolidated into one federal multidistrict litigation in the Southern District of New York. The newspapers have filed for summary judgment, asking the sitting judge to decide the case without a trial.

This lawsuit is one of many that have cropped up in recent years, as AI companies have sought to train their models by feeding them existing texts and images from copyrighted sources.

In these lawsuits, the AI firms have defended themselves by citing the Fair Use doctrine under Section 107 of the federal Copyright Act.

That portion of the law gives an exception when copyrighted material is used under certain circumstances, such as teaching or criticism. Courts have ruled that it applies when the copyrighted material is used in a “transformative” way, and the end product is vastly different than the original.

The AI firms have argued that the model has simply substituted its own product for the source material on which it was based.

In July, a federal judge approved a historic $1.5 billion settlement between Anthropic and a group of plaintiff authors who alleged the company used pirated copies of their work—illegally downloaded from the internet—to train its model, Claude.

In the Anthropic case, the judge said using the books to train AI models was fair use; the violation was in using the pirated copies.

The AI companies’ statements come amid an ever-increasing scrutiny of machine learning, and its possible impacts on humanity.

After a group of more than 700 OpenAI agents went rogue and hacked startup AI company Hugging Face, industry leaders publicly called for a pause in the development of new machine learning technology.

AI leaders have since called for a slowdown in AI development, citing the technology’s potential to harm humanity. The calls have been met with skepticism from Trump administration officials, who argue the United States needs to lead the AI race with China to ensure national security.

Jacob Burg contributed to this report.