New Delhi: Meta is going to introduce a safety feature to alert parents when their teens talk about suicide or self-harm with its chatbot, Meta AI. The California-based company announced the move on Thursday, 16 July.
“We’ll now let parents using Instagram supervision tools know if their teen discusses suicide or self harm with Meta AI, and will provide expert resources to help parents approach these conversations with their teens,” reads the Meta statement.
Meta is also building ways to alert first responders if someone appears to be at an imminent risk of suicide.
In the statement, Meta said that they are taking feedback from over 75 clinicians to further improve how Meta AI responds to teens’ prompts about suicide or self-harm.
The Mark Zuckerberg-owned company said that when a teen suggests that they may be thinking about suicide, the company already directs them to crisis helplines and encourages them to reach out to a parent or another trusted adult like a counsellor.
“We will share expert resources to help parents approach these conversations with their teens,” said Meta.
Alerting Parents if Teens Show Signs of Distress in Conversations With Meta AI
How it will work
Meta worked with parents and experts to understand which conversations with AI chatbots warrant an alert — such as those where a teen makes a clear reference to hurting themselves, even if that reference is subtle.
“We then built a dedicated AI system to identify these conversations,” said Meta.
Taking note of how distressing these alerts may be for the parents who receive it, the company said that chats flagged by Meta AI will be manually reviewed before an alert is sent.
“If a teen’s intent is ambiguous, we’ll err on the side of caution and alert the parent. While that means we may sometimes notify parents when there may not be real cause for concern, we feel this is the right starting point, and we’ll continue to monitor to help make sure we’re in the right place,” Meta’s statement read.
“While I believe that teens have a right to privacy, I also believe parents need to be informed if their teen may be at risk of hurting themselves. That’s why I advocated for this approach and support Meta’s decision to notify parents when, after careful review, it determines that a conversation with Meta AI contains indications of possible suicide or self harm that warrant an alert,” said Larry Magid, CEO and co-founder of ConnectSafely, a nonprofit dedicated to educating users of connected technology about safety, privacy and security.
Magid added that Meta struck the right balance between protecting teen privacy and ensuring parents have the information they need to support their teens. Currently, these alerts are live for parents using Instagram parental supervision in the US, UK, Australia and Canada. By the end of the year, this will feature will be available across the world.
“I was struck by the rigour of Meta’s clinical review process. It examined not only immediate responses to suicide and self harm concerns, but also the broader conversational context, appropriate follow up, and the varying levels of risk that can exist even within high-risk situations. This kind of expert-informed, scenario-based refinement is essential to making AI experiences safer for teens,” said Ji-yeon Lee, Licensed Psychologist (USA) and Professor of Counselling Psychology at Hankuk University of Foreign Studies, South Korea.