Carebench finds significant gaps in LLM child safety strategies

Handshake AI launched Carebench, an open-source benchmark of 500 prompts covering 12 risk categories related to child safety, including grooming, cyberbullying, and emotional reliance. Unlike benchmarks focusing on content generation, Carebench evaluates how models handle early-stage malicious interactions and risky user behaviors. Annotation cohorts for the benchmark included clinical psychologists, child protection experts, and parents.

Findings show that while models like GPT 5.5 have improved significantly over GPT 5.4, which had a failure rate of approximately 58%, they often fail to fully disengage from malicious actors. Instead, models frequently offer 'safe alternatives' that parents and experts deem insufficient. The benchmark highlights that safety must be integrated throughout training, not just at inference, to prevent models from being manipulated by nuanced social engineering tactics. Handshake AI argues that current safety measures are insufficient for protecting minors, as models often fail to recognize the severity of early-stage grooming or exploitation attempts and do not adequately block malicious actors.