Brain Scan Database Links Computer, Human Vision

(NIH.gov)

6 May 2019. A publicly-available database offers brain scan images to train machine learning programs for use with computer vision, a form of artificial intelligence. The Bold5000 database is a product of Carnegie Mellon University in Pittsburgh, and described in today’s issue of the journal Scientific Data.

Bold5000 is an effort by psychologists and computer scientists at Carnegie Mellon, as well as a team of 4 intrepid volunteers, to provide developers of computer vision applications a rich source of neurological data to train their algorithms. Computer vision is a key technology for robotics, enabling robots to accurately interpret and respond to images in their fields of view. Yet studies by neuroscientists of human visual perception for training computer vision systems are based on a relatively small number of images, about 100, according to the authors.

The researchers led by computer science doctoral candidate and first author Nadine Chang aim to fill the gap with brain scans from real humans viewing images in widely used computer vision databases. The brain scans are provided by functional magnetic resonance imaging, or fMRI, an extension of MRI that uses a magnetic field to produce images for diagnosing soft tissue injuries. With fMRI, the images show blood flow in the brain, an indicator of brain functions.

The Carnegie Mellon team recruited 4 volunteers who agreed to scan more nearly 5,000 images from 3 leading image databases used to train computer vision systems: ImageNet at Stanford University, Common Objects in Context or CoCo, and Scene Understanding at MIT. The images represent a wide range of scenarios, objects, visual features, and semantics to offer a diverse set of neural responses as shown in the FMRIs. The 4 volunteers, 2 men and 2 women in their 20s, each endured about 20 hours of FMRI scanning to produce the 5,254 images in the Bold5000 database. (Bold stands for Brain, Object, Landscape Dataset.)

The university says the Bold5000 database, made available to the public, so far generated a positive response from other computer scientists, as indicated by more than 2,500 downloads. The team also makes available its data-collection code and raw fMRI images. The authors point out that the 5,000+ image database may be much larger than others assembled so far, but it’s still too small for training many computer vision applications needing many more times that number to be effective. The small number of brain-scan volunteers is another limiting factor.

Nonetheless, as Chang notes in a university statement, the project shows the feasibility of linking the efforts of computer vision scientists and neuroscientists, who both can benefit from Bold5000. “Computer-vision scientists and visual neuroscientists essentially have the same end goal,” says Chang, “to understand how to process and interpret visual information.”

More from Science & Enterprise:

• Human-Friendly Robot Designed for A.I.
• Autonomous Robots Designed for Flood Projects
• Start-Up Creating Touch Sensing for Visual Images
• Micro Robots Made to Climb in Curved, Inverted Spaces
• Robotics, A.I. Employed for Personal Autism Therapy

*     *     *