Collective Vision
by Mark Lenz

Recently I gave a talk on a few research papers that reflect an idea I like to call collective vision, the process of using the billions of images on the Internet with computer vision algorithms to learn information about our world. The text on Internet web pages has been scoured for years to infer many different things from the relevance of websites to the level of buzz for a product. The billions of photos and images found on the Internet provide another large resource of data that until recently hasn’t been tapped, and the potential for new insights is increasing with the proliferation of websites such as Flickr, Panoramio, and Facebook. Mobile phones with cameras make it easy for everyone to capture and upload photos, increasing the amount of available information every second.

There are a few research groups diving into the sea of Internet images, funded mainly by Microsoft, Google, and Nokia. I think the first novel use of Internet images was by Hays and Efros at CMU in 2007. They used a huge database of Internet photos to attempt to convincingly fill holes in photos, scanning the database for photos with similar regions to fill the hole. The Community Photo Collections team at the University of Washington developed the system they call Photo Tourism, which was the first system that can automatically reconstruct and visualize 3D scenes using photos gathered from the Internet. Further work has gone in to speeding up the reconstruction process to enable the reconstruction of full cities. Google uses photos from the Internet to power part of their Google Goggles search as well as a world-wide landmark recognition engine.

Collective vision could be used beyond reconstruction of the world and filling gaps in photos, and it can apply to other large sets of images that may not be collected from the Internet. In the short term, collective vision could aid in social tourism, something I’m investigating as a final course project, or it could help in disaster relief. I think some longer-term possible uses could be detection of infrastructure in need of repair or tracking and prediction of fashion trends. Other researchers and groups have begun investigating the potential of collective vision, and even more have shown an interest in it. So with tons of potential uses and lots of minds, it won’t be long until someone finds another novel use of collective vision, and then there’s video.

Originally posted on the Code of Intelligence blog.