Before 2026, Will DL systems outperform humans at describing a picture in words to make human mental images match it?
11
Ṁ3911
2026
45%
chance

Before 2026, will experimental results be announced demonstrating a Deep Learning system that outperforms humans at something roughly like the following task?

  • A person spends less than one hour in an MRI machine visualizing pictures shown and described to them, and this is used to calibrate or funetune the DL system

  • Reference pictures are then shown to the DL system, which must translate them into written or audio descriptions given to the person in the MRI machine, who tries to visualize them

  • The reference pictures are then shown directly to the person, and brain activities from the description (including reconstructions of what they visualized) vs the picture are compared for their similarity.

In order for this market to resolve YES, the system must unambiguously outperform researchers or prepared humans at accurately conveying a visual scene, including in cases where the researcher gets up to an hour to practice with the tested person. If it is a clear common expectation that AI systems would outperform humans at this task if it were formally run, this market can also resolve YES.

This market attempts to evaluate the sophistication of multimodal models that can work between language, images, and human neural activations in the short term future.

I reserve the right to bet on this market because I believe the resolution criteria are sufficiently objective. If that impedes on peoples' willingness to bet, let me know and I may be willing to stop betting.

Get Ṁ1,000 play money
Sort by:

I would expect unless you have reason to think otherwise that this won’t be tested before 2026? Which if true puts the meaningfulness of the prediction into question

This market resolves NO if we don’t have good experimental evidence or it’s not obviously clear that AI systems can meet this threshold. However, I’m already pricing that in through my bet on YES.