Google is celebrated (and feared) for the volumes of user testing it conducts on each facet of its products’ designs. Across its properties, Google researchers have access to millions of user interactions per day. From the early days, they have taken a data-driven approach to design. Google’s extensive testing of shades of blue or its search screen is now legend.
But when I was at the Googleplex HQ in Mountain View, I was confronted with something I had last seen at a lemonade stand run by kids in my neighborhood. At the Google merchandise store I saw a sandwich board sign with a hand-scrawled appeal inviting people to answer a few questions about You Tube, in exchange for either a $10 credit at the Play Store or a plastic figurine of the Google Android character.
As someone who has been working with technology and customer engagement for decades, I could not resist the experience of being on the other side of this user testing exercise. And as a long-time user of YouTube, I was curious to find out what part of the app they were redesigning.
Inside the store on a couch among the Google t-shirts, pens, notepads, mugs and various other merchandise sat a person equipped with two phones, one laptop and a sheet of paper. In less than ten minutes he took me through an exercise that was decidedly low-tech. He showed me a beat up iPhone with a YouTube screen, asked me to save a video to view later, then showed me a sheet of paper with four different pop-up screen options to consider.. We repeated the exercise with a slightly different version. He took notes on his laptop. That was it. A high-tech firm doing low-tech testing.
The experience served as a stark reminder of the value of getting out and literally getting in front of your users. It’s something any organization – nonprofits, publishers, tech companies – can do. And if a mega-tech company like Google is still doing it, you should too.
With access to an average of 1.9 billion logged-in users each month. YouTube easily could have relied solely on automated user experience testing, sampling a much larger and geographically diverse audience around the world. Yet, they recognized the value of the face-to-face user testing, even if its methodology was based simply on a random sample of people walking in to buy a Google t-shirt and taking the time to answer a few questions. Of course Google has loads of cash and access, and I am quite certain that the product management team charged with figuring out the optimal flow of this piece of their apps’ user experience are also doing that sort of ‘high-tech’ user testing on a broader scale. My responses will undoubtedly be mixed in with loads of data on user behavior to develop a smoother YouTube experience.
Recent concerns about the destructive effects of YouTube algorithms, including concerns about privacy and scandals about data collection, might be driving the company to use more face-to-face methods, but it is also common sense. There are distinct merits to combining insights gained from both low-tech and high-tech approaches.
Leaders of early-stage companies know the value of getting in front of their customers. Founders stand on street corners and solicit input. As organizations grow, however, other priorities take the lead and that face-to-face connection is often lost.
Smart data collection and analyzing user behavior at scale matters tremendously for organizations to effectively adapt to changes in the market. Successful organizations must continue to expand their capabilities to efficiently mine the insights that come from tracking and analyzing the keystrokes of users. However, at the same time, leaders are wise to include face-to-face information collection in their research mix. Sometimes the most innovative and effective tools don’t require any algorithms at all.
