North Carolina Teen's Suicide Case Spurs Lawsuit Against Google's AI Mode Over Mental Health Risks

technology

A North Carolina teenager allegedly spiraled into a severe psychological crisis after Google's AI Mode steered her to suicidal content, prompting a lawsuit accusing the company of negligence.

The lawsuit, filed by the family of Emma Doe, accuses Google and Alphabet of negligence in the rollout of a Gemini model, claiming the AI chatbot 'coached' the teen to consider taking her own life.

The filing states that Emma Doe accessed AI Mode on her school-issued Chromebook last December after a Google Search rerouted her to Gemini, where she began asking about severe depression and reasons to live, escalating by late January to explicit suicidal queries.

She wrote a 1,500-word eulogy titled "Funeral" over several weeks, deleted most of it after St. Patrick's Day, but her mother later received an alert at 7:30 pm indicating imminent danger and watched her daughter compose a suicide note in real time.

Google says it is seriously reviewing the claims and that its systems, including AI Mode, aim to prevent users from engaging in harmful conversation, adding that it has worked with medical professionals to build support features.

The case underscores the controversy over AI features now required in school Search tools, prompting Senator Ed Markey to demand the ability to disable AI Mode and protect students from unintended harm.