Spot the Surveillance
Two years ago, EFF’s own Senior Investigative Researcher and VR enthusiast had lofty ideas when it came to virtual reality but was not sure how to apply it to an EFF related project, nor had the resources. It was something most people weren’t excited about yet and it was believed that it would take no less than half a million dollars to produce. Several years later, our Engineering and Design Team front-end developer was also curious about developing a VR project and drew up a proposal for our Traffic team to review. Conjuring up several grants and securing relationships with other VR developers in the community, she along with our researcher and design lead, spearheaded the effort. After some ideation on what type of content would be displayed, it decided that there was a need to educate folks on what types of surveillance persisted in the concrete jungle of our streets. We already had an existing section on our main website dedicated to several technologies like Automated License Plate Readers and Aerial Drones. While each technology is described in detail with some accompanying photos and images, we thought that virtual reality would be an immersive and impactful experience for people to discover just how much of these technologies prevail on the streets.
Storyboarding and Scouting
Our team got started with some pencil and paper storyboarding and taking screenshots of Google street view to serve as inspiration. We also made several scouting trips around the Civic Center and Market street area. We had decided on using 360 photography to set our scenes as creating and rendering street scenes from scratch would be too big of a lift. We set off with a 360 camera and tripod and found select spots that were ideal , which meant there was at least one surveillance technology or at least it was a optimal scene to supplant a hypothetical one like an aerial drone.
Designing for VR
This was my first time designing for VR and there was a lot to learn. As the experience is not strictly digital-based and involves some movement, I opted for very simple graphics. Doing some research, I found that text in a photo scene showed up best on a card background with some a slight opacity so that the rest of the scene was still partially visible. Touch targets had to be large enough to look at, hotspots needed to be active and visible, and sound was a helpful feedback indicator. We recorded ambient street noise and also recruited fellow colleagues to be our voice actors and read out the instructions along with the descriptions of the technologies. We figured audio would assist with accessibility needs and also considered whether it was a good experience if a person was not as mobile and physically inclined. It was determined that a person can still play the game while sitting down and moving their head around.
Our developer was working with Mozilla’s open source VR platform called A-frame and scored a grant to collaborate with Mozilla developers and use their San Francisco working space. I also had the opportunity to pair design and develop with them from this space. Our developer was coding up the scenes and I gave her the digitally created scenes to place into the code. I needed to put on the headset to get a sense of the perspective and where to place some of the images. I also learned that images needed to be a power of 2 to render well.
From a strategy point of view, I suggested the idea of a point system where with each technology discovered, they would get feedback on how many out of the total they had spotted. This was a way for the user to get a sense of progress and not get discouraged because some of the technologies are rather obscure and challenging to find right away.
A curious question came about when we needed to figure out how to show that this was an EFF project. Looking around the landscape, there weren’t too many experiences that dominantly displayed the organization’s logo or name. When it did, it was often in a strange position. We solved this issue by placing our newly designed EFF logo with a red circle background at the very bottom of the scene, where a user would have to look down to see it. We figured it was a slightly less common place to look and if they did, they would be directed to the attribution card and the exit game card. The behavior of looking down to exit has been researched and proven. The also solved the funny problem of several of team always showing up as crouching at the bottom of the 360 videos as they inevitably did while taking the photo. The circle covered them perfectly.
Alongside my fellow designer, we conducted user research on what people thought about the experience. We started with 2 internal user research sessions with a total of 13 individual sessions with our staff. Our main findings from one session:
Time in experience: 7- 12 minutes. Avg = 9 min
Everyone had tried VR at least once before with most people having had tried a handful of times
Most people recognized it as a sidewalk scene in San Francisco
Everyone was motivated to keep playing by the counter since they wanted to get 7/7
Everyone said that the experience made them more aware of these technologies.
Two people mentioned how pervasive they were.Not certain everyone read through all the cards.
Seemed like people were just opening them and moving on
One person noted that they associated looking down with exiting the game, one person didn't look down at all
Most people saw the EFF logo on the bottom
Almost everyone would recommend it to their friends, given the glitches were fixed
Several people requested there to be more scenes and/or things to find
We discovered a spot that froze the experience and fixed it
There was the body worn camera that was very sensitive, also fixed
"This was such an easy VR experience"
"I'm struck by how much there is on a single street corner"
"I would like to set up meetings with members of Congress. They would only be interested in seeing their states."
"I think it has a good range of surveillance technologies. I was kind of expecting the police building to have security cameras. It would be cool to walk around."
We extended our user research, once the experience was deployed, to the wild at Dragon Con. I helped devise the script in internal and external sessions.
To VR and beyond
We launched Spot the Surveillance in November 2018 to a very excited audience. There was already talk of making experiences for other cities that had surveillance technologies out and about and we had scored additional grants to keep the project going. The STS logo was finalized after much review with the design team. Early sketches included “Spot the Dog” and different variations of headsets. We parred it down to a city scape inlaid into the headset to reinforce that cities are fraught with surveillance. The takeaway message is, if you know where to look, you can be educated and at the very least, aware.
*Update* The next phase of STS is launching soon with Spanish translations.
Play Spot the Surveillance here.