🛠️ AI Tools

I Poisoned Contracts to Test AI's Legal Brain – Shocking Hits and Misses

What if AI couldn't spot a $3.65 million hidden fee? I built fake contracts to test it – and the results will make you rethink autonomous AI agents.

AI model responses failing and passing fake contract analysis tests

⚡ Key Takeaways

  • AI excels at syntax and basics but misses hidden math traps and commercial sanity. 𝕏
  • Unprompted models fail ghost fees; prompting or advanced like o1 fixes some. 𝕏
  • Agentic future: multi-agent systems will conquer contracts, but humans oversee now. 𝕏
James Kowalski
Written by

James Kowalski

Investigative tech reporter focused on AI ethics, regulation, and societal impact.

Worth sharing?

Get the best AI stories of the week in your inbox — no noise, no spam.

Originally reported by Towards AI

Stay in the loop

The week's most important stories from theAIcatchup, delivered once a week.