AI models answer questions about medical images they haven’t seen

 “AI models happily came up with answers to questions about a supposedly accompanying image — even if the researchers never even showed it an image.”

The article quotes the researchers as talking about this in terms of there being “information […] hidden in a sentence or a question” that allows the AIs to correctly answer the questions even without access to the images that the questions are about, but that seems weirdly implausible to me. Later in the article, there are implications of what seems to me far more likely: that the AI was trained on these questions and answers, and thus can provide the answers without having to have access to the images that accompany the text.

More from the researchers:

“Another implication is that, now that we know an AI can say ‘I see evidence of malignant melanoma on your skin’ without even having access to any images, how much can we trust it when it says the same while actually seeing the image?” Asadi posited. “We definitely need more effort being put in safety and alignment of such models, and might need to think twice before deploying them in user/patient-facing systems.”

[…] “The number one [takeaway] would be that just because the AI is saying, very convincingly, that it is seeing something, it doesn’t mean that it is actually seeing that.”