Companies must systematically assess both foreseeable harms from intended AI use and plausible misuse by bad actors, then build continuous safety processes powerful enough to address emerging risks
Key highlights:
-
-
-
Map risks before building— Distinguish between foreseeable harms that may be embedded in your product’s design and potential misuse by bad actors.
-
Safety processes need real authority— An AI safety framework is only credible if it has the power to delay launches, halt deployments, or mandate redesigns when human rights risks outweigh business incentives.
-
Triggers enable proactive intervention— Define clear, automatic review triggers such as product updates, geographic expansion, or emerging patterns in user reports to ensure your safety processes adapt as risks evolve rather than reacting after harm occurs.
-
-
In recent months, the human cost of AI has become impossible to ignore. Teenagers have taken their own lives after interacting with AI chatbots, while generative AI (GenAI) tools have been weaponized to create non-consensual deepfake images that digitally undress women and children. These tragedies underscore that the gap between stated values around AI and actual safeguards remains wide, despite major tech companies publishing responsible AI principles.
Cecely Richard-Carvajal, a senior associate at Article One, who works at the intersection of technology and human rights, argues that closing this gap requires companies to: i) systematically assess both foreseeable harms from intended AI use and plausible misuse by bad actors; and ii) build safety processes powerful enough to actually stop launches when risks to people outweigh commercial incentives.
Detailing the two-step framework for anticipating and addressing AI risks
To build effective AI safety processes, companies must first understand what they’re protecting against, then establish credible mechanisms to act on that knowledge.
Step 1: Mapping foreseeable arms and intentional misuse
When mapping AI risks during “responsible foresight workshops” with clients, Richard-Carvajal says she takes them through a process that identifies:
-
- foreseeable harms that emerge from a product’s design itself. For example, algorithm-driven recommender systems — which often are used by social media platforms to keep users on the site — are designed to drive engagement through personalized content, and are well-documented in amplifying sensationalist, polarizing, and emotionally harmful content, according to Richard-Carvajal.
-
- intentional misuse that involves bad actors who may weaponize technology beyond its purpose. Richard-Carvajal points to the example of Bluetooth tracking devices, which initially were designed to help people find lost items, but were quickly exploited by stalkers, who placed them in victims’ handbags in order to track their movements and in some cases, to follow them home.
Tactically, the role-playing use of “bad actor personas” by Richard-Carvajal and her colleagues can help clients imagine misuse scenarios and help ensure companies anticipate harm before it occurs rather than responding after people have been hurt.
Step 2: Building a credible AI safety process
Once risks are identified, Richard-Carvajal says she advises that companies identify mechanisms to address them. The components of a legitimate AI safety framework mirror the structure of robust human rights due diligence by centering on the risks to people.
Indeed, Richard-Carvajal identifies core components of this framework, which include: i) hazard analysis and red teaming to anticipate both foreseeable harms and potential misuse; ii) incident response mechanisms that allow users to report problems; and iii) ongoing review protocols that adapt as risks evolve.
Continual evaluation of new emerging risks is needed
As AI capabilities advance and deployment contexts expand, companies must continuously reassess whether their existing safeguards remain adequate against evolving threats to privacy, vulnerable populations, human autonomy, and explainability. Richard-Carvajal discusses each one of these factors in depth.
Privacy — Traditional privacy mitigations, such as removing information that leads to identifying specific individuals, are no longer sufficient as AI systems can now re-identify individuals by linking supposedly anonymized data back to specific people or using synthetic training data that still enables re-identification. The rise of personalized AI — in which sensitive information from emails, calendars, and health data aggregates into comprehensive profiles shared across third-party providers — can create new privacy vulnerabilities.
Children — Companies must apply a heightened risk lens for vulnerable populations, such as children, because young users lack the same capacity as adults to critically assess AI outputs. Indeed, the growing concerns around AI usage and children are warranted because of AI-generated deepfakes involving real children are being created without their consent. In fact, Richard-Carvajal says that current guidance calls for specific child rights impact assessments and emphasizes the need to engage children, caregivers, educators, and communities.
Cognitive decay — A growing concern is that too much AI usage can harm human autonomy and contribute to a decline in critical thinking. This occurs when AI shifts from helping people to think to thinking for them, and it has the potential to undermine their human rights in regard to work, education, and informed civic participation.
Meaningful explainability — Companies’ commitment to explainability as a core tenet of their responsible AI programs was always a challenge. As synthetic AI-generated data increasingly trains new models, explainability becomes even more critical because engineers may struggle to trace decision-making through these layered systems. To make explainability meaningful in these contexts, companies must disclose AI limitations and appropriate use contexts, while maintaining human-in-the-loop oversight for consequential decisions. Likewise, testing explanations should require engagement with actual rights holders instead of just relying on internal reviews.
Moving forward safely
While no universal checklist exists for AI safety, the systematic approach itself is non-negotiable. Success means empowering engineers to identify and address human-centered risks early, maintaining ongoing stakeholder engagement, and building safety processes that have genuine authority to delay launches, halt deployments, or mandate redesigns when human rights outweigh commercial pressures to ship products.
If your company builds or deploys AI, take action now: Give your engineers and risk teams the authority and resources to identify harms early, keep continuous engagement with affected people and independent stakeholders, and create governance that have the power to keep harm from happening.
Indeed, companies need to make sure these steps go beyond simple best practices on paper and make these protective processes operational, measurable, and enforceable before their next product release.
You can find more about human rights considerations around AI in our ongoing Human Layer of AI series here
