The Box

Overview

Most consumers have no idea how the devices they buy handle their data. Privacy policies are buried, hard to read, and rarely consulted in the moment of purchase. This project explored whether augmented reality could change that. Commissioned through an NYC Media Lab seed grant with Consumer Reports, we built an AR prototype that lets users point their phone at a device in a store and instantly see its privacy rating. The key innovation was training custom vision models on Consumer Reports’ product image database and connecting it to Mozilla’s Privacy Not Included initiative to surface privacy information in real time.

My role

I wrote the initial grant proposal and led product design, defining the user experience and interaction model. I also helped train the prototype demo dataset, working with Consumer Reports’ image library to build out the vision model’s recognition capabilities.

Team

Brent Bailey

Matthew Ross

Key Technologies

Custom Image Classifier on CoreML, Swift

Working Prototype