New AI Model Reads Brain Scans to Reconstruct Images With Unprecedented Speed and Accuracy

Scientists at the Weizmann Institute of Science have unveiled Brain-IT, an AI model that can reconstruct images from brain scans with remarkable accuracy and speed. The system analyzes brain activity to recreate what a person is viewing, completing the process in approximately one hour—a significant improvement over earlier methods that required dozens of hours of scanning.1

1

How the Mind Reading AI Works

Developed in Prof. Michal Irani's lab, Brain-IT was trained using more than 70,000 images shown to eight participants while their brains were scanned. The researchers divided each brain scan into approximately 40,000 tiny sections called voxels, measuring how each responded to different visual stimuli.1

1

The team built an encoder model that identifies patterns shared across different people's brains, predicting brain activity from images. This encoder naturally identified 128 functional regions shared by all people that perform specific roles in image processing.1

1

Superior Accuracy in Image Reconstruction

Unlike earlier AI models that struggled with basic features like composition and color, Brain-IT excels at reconstructing both content and fine details. "The new model we developed outperforms them in reconstructing both the content of the image and its details," Irani explains.1

1

The system uses high-resolution brain scans that can show brain volumes as small as one cubic millimeter of neurons, enabling more precise reconstructions than previous attempts.2

2

Innovative Training Approach

To overcome the challenge of limited brain scan data, researchers developed a dual-system approach. The encoder model predicts brain activity from images, while a decoder model works in reverse, using brain activity to reconstruct images. By pairing these with a diffusion model and repeatedly training them together, the team generated synthetic brain scans that could be used to refine the AI tool to recreate images with increasing accuracy.2

2

This innovative method allowed them to work around the scarcity of high-resolution brain scans needed for training.

Discovering New Brain Functions

The research revealed unexpected insights into how brains process visual information. "During training, the encoder naturally identified 128 functional regions that are shared by all people and perform specific roles in image processing," Irani notes. "Some of them are familiar to neuroscientists, but others are entirely new."1

1

For instance, the team discovered a division within the PPA brain region, with one part responding to indoor scenes and another to outdoor scenes.

Potential Applications and Limitations

The technology behind Brain-IT could transform communication for people with paralysis, offering them new ways to express themselves. It also provides scientists with a more efficient method to study how the brain processes images.1

1

However, the system isn't flawless. Irani acknowledges occasional failures, such as reconstructing a dog in a bathtub as a goat in a bathtub.2

2

Growing Concerns About Brain Privacy

The breakthrough has sparked serious ethical concerns among neuroscience researchers. "The results seem very impressive," says Tommy Sprague, a neuroscientist at the University of California, Santa Barbara. "But if there's a way to surreptitiously extract information about what you're thinking about, then... 150 years of sci-fi can come true anytime, and that's worrisome in a lot of ways."2

2

Currently, the technology requires expensive fMRI machines, but experts warn that if similar techniques were applied to wearable EEG devices, it could enable companies to extract information from people's brains without consent. Marcello Ienca, a neuroscientist and philosopher at the Technical University of Munich, told MIT Tech Review that while the research is well-intentioned, "it's also pretty obvious that it could be co-opted for... ethically and societally problematic commercial uses."2

2

As this mind reading AI advances, the need for robust brain privacy protections becomes increasingly urgent.