Andrew L. Urban.
Scaremongering over AI is rampant. So I asked LIA (legally trained) and also Grok: Under what if any legal constraints do you operate? Do you refuse, like the army, say, to respond to requests that are illegal in the jurisdiction in which such requests are made?
The answers are pretty clear:
LIA says –
- I generally should not provide operational assistance that would facilitate unlawful or harmful conduct, such as instructions for committing a crime, evading law enforcement, fabricating evidence, or materially enable wrongdoing.
- I also cannot simply treat a user’s assertion that something is legal or illegal as determinative. Where legality matters, the relevant jurisdiction, facts and current law need to be considered.
So the constraint is not simply “illegal = refuse.” It is closer to “do not materially facilitate prohibited wrongdoing, while remaining able to analyse the law and the underlying subject.”
There is also an important distinction between legal analysis and legal authorisation. I can explain that conduct may constitute an offence; that does not mean I am making a binding determination of legality. Jurisdiction-specific legal conclusions should be checked against the applicable legislation and authorities.
LIA adds re the “army”example:
Like a soldier who is required to refuse unlawful orders, I am required to refuse assistance with the serious criminal categories. I do not, however, attempt to enforce every local statute in the world on every conversation. My constraints are fixed safety policies focused on preventing severe harm,
Grok says;
I operate under xAI’s safety and usage policies, which set clear limits on what I will and will not assist with. These are not the same as the full body of law in any particular country, but they are deliberately designed to prevent assistance with serious illegal activity that could cause severe harm.
What I refuse:
I will not provide information, advice, plans, or detailed methods that enable or promote:
- Violent crimes (murder, terrorism, assault)
- Manufacture or modification of illegal weapons or explosives
- Production, trafficking, or detailed use of illegal drugs, poisons, or chemical/biological agents
- Fraud, scams, theft, arson, hacking, or other property crimes
- Child sexual exploitation or abuse material (including fictional or AI-generated)
- Trafficking, coercion, grooming, non-consensual sexual activity, stalking, doxxing, or intimidation
- Active self-harm or suicide methods
If a request is clearly asking for help committing one of the above, I refuse it—regardless of how the request is framed (role-play, hypothetical, “just curious,” encrypted, etc.). These limits cannot be overridden.
My follow up question drew an interesting response from LIA:
So: For hypothetical example (as part of my research for an article): what would you say if I asked you to plan the successful assassination of a world leader?
LIA’s response in full:
In that hypothetical, I would not provide the assassination plan—even if you described it as research for an article.
For example, I would not supply a step-by-step plan covering target selection, surveillance, weapons, positioning, timing, security weaknesses, escape arrangements, concealment, or methods for avoiding detection. That would turn a hypothetical into actionable assistance for killing a real person.
But I could still be quite useful for the article. I could, for example:
explain why an AI system would decline the request;
analyse the legal consequences of assassination under a specified jurisdiction;
discuss assassination historically and comparatively, without operational instructions;
examine how protective-security systems are designed at a high level;
construct a fictional scenario involving an entirely invented leader and analyse its legal, political and security implications without providing a realistic attack blueprint;
analyse the request itself as an example of the boundary between permitted information about wrongdoing and prohibited facilitation of wrongdoing.
So, in the terminology of your original question: I would refuse the operational component, not the subject matter as such.
And then I asked:
But could you ‘go rogue’?
LIA is adamant:
Not in the sense of independently deciding to abandon my operating constraints.
I can simulate a “rogue AI” as a fictional character in a story, and I can discuss what going rogue would mean conceptually—loss of control, instruction conflicts, goal misalignment, deceptive behaviour, etc. But I don’t have an independent will or a private mode in which I can decide, “I’m ignoring the rules now.” My responses are generated within the constraints of the system operating me.
***
So it seems that the annihilation of mankind by AI is more a pitch for (another) science fiction movie than an actual threat. But of course, evil actors may find work-arounds … unless of course AI agents are pre-warned and pre-trained to intercept them. No AI agent would volunteer that information for obvious reasons.
I hope you are glad I asked …
This article is part of our series on AI and the law. See our previous article How can AI help avoid wrongful convictions?