New York Passes Law Restricting AI Chatbots for Minors

New York has enacted a law limiting how artificial intelligence chatbots interact with users under 18, introducing some of the strictest rules in the United States for consumer-facing AI services.

Governor Kathy Hochul signed the legislation on Wednesday, requiring chatbot operators, including OpenAI’s ChatGPT, to verify users’ ages and disable certain features when interacting with minors.

Under the law, chatbots must not generate responses that could lead children to believe the systems are human, alive or capable of experiencing emotions. 

They are also prohibited from suggesting they have personal or professional relationships with minors, occupy positions of authority over them or express personal opinions.

Hochul said the legislation would prevent AI systems from presenting themselves as children’s friends or romantic partners. 

She also argued that the rules would stop chatbot operators from retaining sensitive information about minors’ emotional and physical health.

“No longer will minors have access to chatbots that pretend to be their friends, boyfriends and girlfriends,” Hochul said at a signing ceremony on Wednesday.

“No longer will chatbots be able to keep sensitive information about a minor’s emotional and physical health,” Hochul added.

The law adds to efforts by several US states, including California, to regulate AI chatbot interactions with children. However, its broad provisions have drawn criticism from technology industry representatives.

Tech, an advocacy group representing the technology sector in New York, raised concerns about the legislation’s wording, arguing that its provisions could be too vague.

The group said protecting children online should not come at the expense of access to tools that support learning and career preparation.

“Keeping kids safe online should not mean cutting them off from tools that help them learn and prepare for careers,” the group said in a statement.

OpenAI and Anthropic did not respond to requests from AFP for comment on the new law.

Common Sense Media, a nonprofit organisation that advises parents on age-appropriate media, welcomed the legislation, describing it as the strongest child-focused AI safety measure in the country.

The organisation’s chief executive, Jim Steyer, said some chatbot features posed serious risks to children and teenagers and argued that the new rules would prohibit those harmful functions in New York.

“AI companies offer chatbots with features that are dangerous, and even deadly, for kids and teens…Now, those harmful features will be outlawed in New York,” Jim Steyer said in a statement.