A former U.S. Central Command artificial intelligence adviser says a bad prompt can flood analysts with “15,000 reasons” to hit the wrong target — a risk no commander should ignore.
Story Highlights
- A former Central Command adviser described how leading questions can make military AI invent massive “evidence” lists.
- He urged analysts to force the system to check disconfirming sources before any strike.
- Defense policy requires human judgment over force, but leaves it flexible.
- Supporters say human checks remain in the loop and tools like Maven only guide choices.
What the adviser warned about: prompts can rig the answer
Military.com reported that Andrew Moore, once the first artificial intelligence adviser to U.S. Central Command, explained a simple failure mode. If an operator asks an AI to “find support for striking this house,” the system can return what seems like thousands of confirming data points in seconds. He said the result looks strong, but it is biased by design because the tool was told to seek only support and not challenges to the claim.
Moore’s fix was basic, and it mirrors common sense. He said analysts should ask for pros and cons, and demand independent sources before moving ahead. That means instructing the machine to hunt for evidence that both supports and undercuts a target call, and then tracing where that evidence comes from. It also means not starting with the answer you want. His point was clear: better prompts and disciplined review can blunt automation bias.
What current policy says about “human judgment” in weapons use
Congressional Research Service summaries explain that Defense Department Directive 3000.09 requires systems to be built so commanders and operators can exercise appropriate human judgment over the use of force. That includes judging the situation and the law. But “appropriate” is flexible. It does not always mean a person must press the final trigger. It can mean humans decide when and where to use a system, while the system runs within those bounds.
That flexibility protects rapid action in combat. But it also creates a gap that Moore’s warning hits. If human judgment is broad and not tied to each single target, weak prompts and rushed reviews can pass as compliant. The policy by itself does not prove that an analyst dug into disconfirming data for every proposed target. It says they can, and should, but it does not measure how often they actually do it in the field.
Real-world pressure: volume, speed, and the risk of rubber-stamping
Public reporting on alleged high-volume targeting in Gaza shows the danger of scale, even as facts remain contested. A summary of April 2024 testimonies claimed an Israeli tool flagged tens of thousands of suspects, and that some approvals were brief. Israeli Defense Forces denied that account and said each target requires independent checks. The core lesson stands either way: high output can overwhelm reviewers and push them to confirm rather than question.
American programs face similar pressure to speed up. Supporters of systems like Project Maven stress there is always a human in the loop and that the tool guides, not fires. A Palantir leader told the BBC that a human makes the final call, and Central Command’s technology chief said Maven helps find potential targets but does not verify them or deploy weapons. A warrant officer said the tool raised per-hour approvals from about 30 to as many as 80.
The conservative bottom line: speed is good, but truth beats hurry
America must win wars and protect troops. But rushing because a screen lit up is not strength. It is how you miss context, break rules, and hand enemies propaganda. Moore’s warning fits common sense parents teach kids: ask both sides, check the source, and do not start with the answer you want. Clear prompts that demand pros and cons, logs that show what was checked, and audits that track rejections can keep human judgment real, not just a box to tick.
Under President Trump, the mission is peace through strength and accountability. That means tools that help our operators think, not replace them. Congress and the Pentagon should require workflows that force disconfirming checks, record analyst notes, and report how often targets are changed or rejected. Directive 3000.09 gives the frame. Now we need teeth that ensure every “human in the loop” actually challenges the machine before the stakes turn deadly.
Sources:
military.com, congress.gov, theworldofai.org, chorrocks.substack.com
