Mirrors help you see objects outside of your line of sight, whether it’s a car passing you on the highway or an unfortunate rash on your face. And as it turns out, with some extra computer processing, almost any old shiny object can serve as a decent mirror. In new research, computer scientists at the University of Washington have exploited the light reflected from the metallic lining of a bag of snacks to create a relatively faithful reconstruction of its surroundings.
“Remarkably, images of the shiny bag of chips contain sufficient clues to be able to reconstruct a detailed image of the room, including the layout of lights, windows, and even objects outside that are visible through windows,” coauthors Jeong Joon Park, Aleksander Holynski, and Steve Seitz of the University of Washington wrote in a paper that has been accepted to this year’s Conference on Computer Vision and Pattern Recognition proceedings. Their research helps to resolve a technical hurdle for virtual and augmented reality technology, although some experts say the scope of its potential uses—and abuses—is much larger.
Technically speaking, the researchers didn’t actually use chips; they reconstructed a room using a Korean brand of chocolate-dipped corn puffs called Corn Cho. But whether it’s corn puffs or potato chips, the snack bag acts like a bad, warped mirror. A heavily-distorted reflection of the room is contained in the glint of light that bounces off the bag, and the team developed an algorithm that unwarps that glint into a blurry but recognizable image.
In one instance, the researchers were able to resolve the silhouette of a man standing in front of a window. In another, the bag reflections allowed them to see through a window to the house across the street clearly enough to count how many stories it had. The algorithm works on a variety of glossy objects—the shinier, the better. Using the sheen of a porcelain cat, for example, they could also reconstruct the layout of the surrounding ceiling lights.