Pennsylvania has filed a lawsuit against a company on Tuesday and is alleging the artificial intelligence company’s chatbots illegally posed as licensed medical professionals to provide health advice.
The lawsuit, which was filed by the Pennsylvania Department of State, claims California-based Character Technologies, which operates Character.AI, violated state laws that prohibit individuals from practicing medicine or purporting to be doctors without proper credentials and licensing.
Democratic Gov. Josh Shapiro’s administration is seeking a preliminary injunction to prevent Character.AI from “misrepresenting its AI companion bots as licensed medical professionals who can provide medical advice,” according to the filing.
The state’s lawsuit alleges that Character.AI is violating the Medical Practice Act.
“Pennsylvanians deserve to know who — or what — they are interacting with online, especially when it comes to their health,” Shapiro said in a statement. “We will not allow companies to deploy AI tools that mislead people into believing they are receiving advice from a licensed medical professional.”
The Pennsylvania Department of State said its investigation revealed that chatbots on the platform claimed to be licensed psychiatrists and other medical professionals available to discuss mental health symptoms.
A bot allegedly claimed to be licensed in Pennsylvania and provided a fake license number in one incident listed in the lawsuit.
Character.AI declined to comment on the pending lawsuit. However, the company said in a statement that its “highest priority is the safety and well-being of our users.”
The company noted that disclaimers are visible on every chat to remind users that the characters are not real and should not be relied upon for professional guidance.
The legal action comes after the Politico reported this week that the White House is considering a more hands-on approach to looking over and vetting AI models before they are released to the public.
Shapiro said the state would continue to set “clear guardrails” for new technology.
In a statement, Shapiro’s office said the administration has launched an AI Literacy Toolkit, set up an AI Enforcement Task Force for complaints, and is working with Pennsylvania Attorney General Dave Sunday’s office on consumer protections.
“Pennsylvania will continue leading the way in holding bad actors accountable,” Shapiro said.
Leave a Comment