Raising Red Flags Across Washington, A New Norm?
The Food and Drug Administration’s newly deployed artificial intelligence tool, Elsa, is generating fake research and misrepresenting scientific data, according to a report from CNN that cites current and former FDA employees.
The AI system, introduced last month by FDA Commissioner Marty Makary, was promoted as a cost-saving tool intended to streamline regulatory tasks and speed up drug approvals. But employees working with Elsa say it has produced summaries based on nonexistent studies and returned inaccurate responses to basic drug-related queries. Internal agency concerns are mounting.
“Anything that you don’t have time to double-check is unreliable,” one employee told CNN. “It hallucinates confidently.”
Elsa was deployed on June 2 and, according to internal documents, cost just twelve thousand dollars in its first four weeks. An internal presentation described the rollout as “ahead of schedule and under budget.” Critics argue that the low cost may reflect a deeper problem: Elsa is not fit for the complexity and accuracy standards demanded by federal drug approval.
The allegations arrive amid a broader push by Health and Human Services Secretary Robert F Kennedy Jr. to bring generative AI into federal health agencies. Speaking recently with Tucker Carlson, Kennedy said AI would allow the FDA to approve drugs “very very quickly.”
In testimony before Congress in June, Kennedy doubled down on that vision. He also suggested that wearable health monitoring devices should be issued to every American within four years. Kennedy’s aggressive push for digital health tools, including unproven AI systems, has drawn scrutiny given his long history of promoting medical misinformation.
AI Missteps in the Spotlight
Elsa is designed to assist FDA staff with tasks like writing summaries, generating reports, and answering research queries. But CNN’s investigation revealed that the system often fails even basic accuracy tests.
When employees asked Elsa how many drugs in a certain category had been approved for children, the tool provided incorrect figures with apparent confidence. While it apologized when corrected, the admission changed little. As one employee put it, “a robot being sorry doesn’t really fix anything.”
The most serious concern involves fabricated study citations. Elsa has produced references to scientific papers that do not exist. That same issue emerged earlier this year in a report issued by Kennedy’s Make America Healthy Again commission. A nonprofit analysis revealed that at least seven cited studies in that report were fictional, and several others misrepresented existing research. It remains unclear whether Elsa was used in the creation of that document.
Makary has acknowledged that Elsa can “potentially hallucinate” but dismissed the issue as typical of large language models. “That is no different from any other generative AI,” he said. Critics argue that this misses the point. The FDA is responsible for public health decisions with real-world consequences. Using a tool that generates unreliable information raises the risk of downstream harm.
The Political Undercurrents
The controversy has further intensified scrutiny of the Trump administration’s broader approach to science and technology policy. Kennedy’s embrace of AI for medical regulation, along with his vocal opposition to vaccines, has alarmed public health experts. Meanwhile, Trump’s nominee for Surgeon General, Casey Means, is a co-founder of a company that sells glucose monitoring devices to non-diabetic individuals. Medical professionals say there is no evidence to support widespread glucose tracking in healthy populations.
When contacted by CNN, FDA spokesperson William Maloney defended Elsa’s performance and said the CNN report had mischaracterized the facts. But when asked to clarify what had been taken out of context or whether any part of the story was inaccurate, the agency did not respond.
“FDA was excited to share the success story of the growth and evolution of its AI tool Elsa,” Maloney said in a statement. “Unfortunately, CNN decided to lead the story with disgruntled former employees and sources who have never even used the current version.”
In a final line that appeared to echo political talking points rather than technical defense, Maloney added, “The only thing hallucinating in this story is CNN’s failed reporting.”
A Dangerous Shortcut
AI tools like Elsa are increasingly finding a place inside government agencies. But as this case illustrates, their promises often outpace their capabilities. FDA insiders say the current model creates more work rather than less, as outputs must be reviewed line by line for errors.
The agency’s defenders insist that Elsa is evolving and that human oversight remains in place. But given the stakes lives, treatments, public trust, critics say shortcuts in scientific review, especially those driven by automation, could become one of the most dangerous experiments of all.