Generative AI Lies

Examples of generative AI making stuff up

Made-up witnesses

Yet another lawyer relied on GPT, resulting in yet another false court filing that no human checked.

But in this case, we have a video of a hearing, at which five New Mexico Supreme Court justices ask the lawyer what happened and why, and what he thinks his punishment should be.

The video is titled “Show Cause Hearing: State v. Sandoval.” The proceedings start around 25:00 in that video and go to around 50:00.

Some notes about the hearing:

According to an article about the issue, the lawyer, Stephen Aarons, filed “a legal brief that referenced fictional witnesses and testimony apparently fabricated by the artificial intelligence he used to help him write it.”

In the hearing, Aarons says that he had previously used GPT and hadn’t had any problems, so he assumed that it couldn’t cause any problems.

Justice Shannon Bacon, disbelievingly, says: “Do you read anything about what’s going on in the world?” And: “My thirteen-year-old nephew knows about [generative-AI] hallucinations. My 75-year-old stepmother knows about hallucinations.”

Aarons says that his situation is different from other lawyers-using-generative-AI situations, because in those other situations, the problem (he says) has been made-up citations, and that wasn’t what happened here.

He then suggests that the solution to this problem is for judges to tell lawyers that AI can hallucinate. (The false filing happened a year ago, and he somehow believes that at that point nobody knew about the problem.) He says that if only the court had told him to be careful about this, the whole problem could’ve been avoided. He later suggests that if only lawyers were shown more horror stories about AI, they would of course be more careful. And if only the court had granted his opponent’s motion to strike his filing, he would have fixed things. And that he hopes the mistake he made will prevent others from making the same mistake in the future. And so on and so on.

One of the justices points out that the main problem here is that Aarons didn’t check everything before he put his name on something: “You have presented briefing to the court that we cannot rely on, and your client is the one suffering from this.”

Aarons eventually reveals that he hasn’t told his client yet what he did. (He told the client’s relatives that there was a problem, but not about his use of GPT, and it’s unclear whether the relatives told the client.)

At around 50:00, the justices leave the room to discuss what happens next, and the video keeps running for another 20 minutes but then ends without anyone returning to the room.

The conclusion of the hearing is in a separate brief video, starting at 05:30 and running for two minutes. The justices tell Aarons that they’re referring him to the disciplinary board and sanctioning him $5,000, payable to the client protection fund.

Every time I see another case like this, I wonder how long lawyers are going to continue saying that they didn’t know that generative AI could make stuff up.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *