Dung Beetle Video Enhancements – Nature Does it Again

Like I always say, keep looking at nature because your problem may have already been solved.  You just have to look a little closer.

Video cameras on your cellphone could soon be good enough to record a jazz concert, a nighttime street scene, or a candlelit dinner. A Swedish start-up has created an algorithm, inspired by dung beetles, that can be integrated into camera modules to offer high-quality video in extremely low light situations.

Source: Dung Beetles Inspire Video Enhancements for Camera Phones

“We are talking about shooting video in situations that seem almost pitch black,” Benjamin Page, business development manager for Nocturnal Vision told Wired.com. “We can offer an unbelievable amount of noise reduction and contrast enhancement at the same time.” Nocturnal Vision presented its technology at the ISE 2010 imaging conference in London Thursday.

Toyota, which financed a significant portion of the research and development, has secured exclusive rights to use the technology in night-vision systems for cars.

Nocturnal Vision says it is now working with mobile phone companies such as Sony Ericsson to test its technology and find a way to integrate it into phones.

As more consumers use the cameras on their cellphones for video and photographs, companies are looking for ways to improve the quality of the camera modules. Earlier this week, Palo Alto startup InVisage Technologies said it has developed a new technology using a nanomaterial called quantum dots that would offer four times the light-gathering performance of current silicon-based sensors.

Nocturnal Vision says its software can be complementary to hardware-based improvements.

The company’s algorithm is based on research by a Lund University zoologist Eric Warrant on dung beetles, bees and other nocturnal bugs. Dung beetles are remarkable because of their ability to see enough detail in the night to find food and escape predators.

Their night-vision capability is the result of their ability to “sum the visual signal locally in space and time,” says Henrik Malm, one of the creators of the algorithm in his research paper. It’s known as adaptive spatio-temporal smoothing. That means the brain analyzes what’s going on across each frame of an image and what’s going on from one frame to another. (See Malm’s research paper on noise reduction and image enhancement in low light video.)

In most digital cameras today, the short, one-time exposure (usually a fraction of a second) and imaging sensors that have uniform sensitivity across their area combine to produce pictures that have underexposed dark areas. Amplifying the dark areas uniformly means the low signal-to-noise ratio becomes pronounced, writes Malm. Instead, adaptive spatio-temporal intensity smoothing can even out the noise, while reducing motion blur.

To do this, Nocturnal Vision’s algorithm pools information from about seven frames before and after a shot to brighten, reduce noise and sharpen the video stream, says Page. The technology can work in real time as scenes are shot, or can be applied to video in post-processing. However, because it requires multiple frames, it won’t work with single-exposure still images.

For instance, a video on the company’s website shows a clip of a man walking in the night. The algorithm first enhances the darker pixels in the frame more than the lighter ones to reveal additional details. But that also introduces a noise into the frame. The algorithm then pools brightness information from adjacent frames to correct for the noise.

The challenge for Nocturnal Vision is that the algorithm sucks up processing power. Most smartphones today, including those featuring the 1-GHz Qualcomm Snapdragon processor, don’t have enough muscle to run the software.

“Currently, we are running it on test devices via GPU computation power,” says Page. “For a standard video with resolution of 640 x 480 it requires approximately 14 billion calculations per frame.”

Nocturnal Vision’s technology works best on uncompressed images. Since most camera phones compress photos as soon as they are taken, that means Nocturnal Vision’s technology would need to be integrated into a phone’s firmware — or directly into a new line of chips. The company says it is looking for chip makers to do just this.

Page says Nocturnal Vision hopes to see its software in the hands of consumers within the next two years. “If we can work with the chip makers, we could be in millions of smartphones,” he says.

And your next nighttime videos might not be quite so dark.