Software Developed that Amplifies Video Frame Variations

The software can visualize vibrations of a guitar string (A. Kotok)

Computer scientists at Massachusetts Institute of Technology have written software that amplifies variations in successive frames of video that are imperceptible to the naked eye. The team of graduate students, alumni, and faculty from MIT’s Computer Science and Artificial Intelligence Laboratory will discuss their software in August at the next Siggraph conference in Los Angeles.

The software, say its developers, works something like an audio equalizer that boosts some frequencies and cuts others, although in this case the signals affected are frequencies of color changes in a sequence of video frames, not the frequency of an audio signal. The software makes it possible to visualize a human pulse, as the skin reddens and pales with the flow of blood, and exaggerate tiny motions thus, for example, making visible the vibrations of individual guitar strings.

The video amplification technique, called Eulerian Video Magnification, appears to work well with phenomena that recur at regular intervals, such as a pulse rates or vibrations of a guitar string. If the range of frequencies is wide enough, however, the program can amplify changes that occur only once, for example, to compare different images of the same scene. In a test of this single-change feature, the system was able to amplify the movement of shadows in a street scene photographed only twice, at an interval of about 15 seconds.

Graduate student Michael Rubinstein says the software was first designed to amplify color changes, but in early experiments the team discovered  it amplified motion as well. Using the system to amplify motion rather than color, the researchers say, requires a different kind of filtration, and it works well only if the motions are relatively small, which gives the software its value.

Rubenstein believes the software could be used for contactless medical monitoring of vital signs, such as pulse rate, or of symptoms such as skin coloration and ease of breathing. Infants, particularly those born prematurely, could benefit, since their fragility makes it sometimes difficulty to attach sensors.

Even for monitoring healthy babies, the system could have value. Rubenstein says he and his wife recently equipped their daughter’s crib with commercial pressure sensors intended to gauge motion. “Those are kind of expensive, and some people really complain about getting false positives with them,” says Rubenstein. “So I can really see how this type of technique will be able to work better.”

In the following video, MIT professor Fredo Durand tells more about and demonstrates the software.

• $2.25M Challenge Seeks Health Care Sensing Technologies
• Gates Award to Fund Tattoo-Like Pregnancy Monitor Device
• 4D Lung Imaging Technology Developed
• Non-Invasive Technique Developed to Measure Brain Pressure

*     *     *