Understand people through real experiences, not words
Understanding people is core to solving any human-related problem. While people are complex and difficult to understand, coming up with solutions without understanding people’s behavior and attitudes is like stabbing in the dark. As a designer, I find that most types of qualitative research are flawed due to de-contextualized or scripted context and reliance on spoken or written words. What I find the most impactful in helping the team understand people is through natural observation and real-life experiences.
At GOGOX, product managers and designers value the product discovery and validation process more so than other teams I’ve worked with previously. With the help of our continuous discovery team, we conduct research on a regular basis to discover problems and opportunities in an ever-changing environment.
On the quantitative side, we use A/B testing to measure the impact and people’s behavior when naturally using our product. We also conduct surveys to understand people’s attitudes. On the qualitative side, though, it’s more challenging to conduct behavioral and attitudinal research during natural use. For example, interviews and focus groups are often conducted in a decontextualized environment, meaning the people usually aren’t using the product naturally while talking to you. Usability tests are great for identifying usability problems, however, the sessions are often scripted and limited by the prototype that’s being tested.
The problems with words
User interviews and focus groups rely on people’s words coming out of their mouths. When we share our research findings with the cross-functional team, we, too, rely on spoken or written words.
Hypothetical questions
As interviewers, we try to avoid “will you” questions because they tend to be less reliable than “have you” questions. The classic example is “will you hit the gym next week?” It’s easy to say yes, and then not go due to having a bad day at work, or last-minute events. Asking “how many times did you hit the gym last week?” would be a more reliable way of determining their future behavior.
However, we sometimes find it difficult to avoid “will you” questions because we’re tackling problems in a new way that is difficult to compare to past experiences. By doing so, though, it’s difficult to gauge how reliable the answers are.
Indirect negative feedback
In the book No Rules Rules: Netflix and the Culture of Reinvention, Erin Meyer and Reed Hastings talk about how cultures have different management styles. For example, people from European cultures tend to give direct negative feedback, whereas people from Asian cultures give less direct negative feedback.
This definitely applies to user research too. Even within Asia, we found that different cultures and people have a range of directness and honesty. For example, Hong Kong and South Korean drivers tend to be more direct than Singapore drivers. Our research team found that it takes a lot more time to break the ice and form a trust with them. Moreover, when incentives are involved, some people are less likely to give honest negative feedback.
Attached to a single pain point
While we try to avoid leading questions and be as open as possible, we found that people tend to always go back to their single biggest pain point. This pain point could be important to them, however, it may not be something that our team could tackle at the moment. There could be many pain points and opportunities that cannot be uncovered by simply talking to them. We can try to uncover these by observing them instead.
The power of observation
Naturalistic observation allows you to identify more behavioral problems. Back in 2019, our delivery team at GOGOX found that most high-frequency deliveries were from merchants who placed orders with similar package information. We came up with an idea that saves the package information, much like how we save the sender/recipient addresses so merchants can place orders easier. To validate this, we showed this prototype to merchants. However, merchants didn’t find it helpful. Instead, they wanted to tell us more about their important pain points.
So we ended up parking this idea, until half a year later, we went to a store and observed a merchant place a real order. To our amusement, this power user placed a delivery order very quickly on their phone. We noticed when he typed “11” or “111”, the package information fields filled up automatically. He was using the native keyboard shortcuts as a workaround. This gave us a signal to ship the saved package information idea, as well as a reuse order feature. The outcome is a 23% reduction in time spent placing an order, and a 93% increased in orders placed.
Observation unlocks pain points that may not seem important to people (or they may not be even aware of them) but may potentially create a great outcome. It’s important to observe people in their natural environment so that you can understand their natural use.
However, observations could be time-consuming, and people may not act 100% naturally when being observed. To really understand why they think or do something, we can dive into real experiences.
Real experiences
The best context is gained through real experiences. It’s easier to scratch your own itch than somebody else’s. When I worked at Hypebeast, most of my team members read about fashion and shopped online. This made it easy for us to gain context on problems were trying to solve because we can look at our own experiences.
After joining GOGOX, I realized I had much less context on the people I wanted to understand: drivers, bikers, walkers (couriers who traveled by walking or public transport), merchants, and people with personal delivery needs.
Fortunately, as part of my onboarding, I spent half a day as a GOGOX driver and a few hours as a customer service rep. Fellow new joiner Maggie and I sat in the passenger seats while our manager Vincent drove the van. We took real orders on the GOGOX driver app and interacted with the customers. In less than four hours, I gained more context than the days I spent walking through the onboarding materials with my coworkers in the office.
To understand small e-commerce merchants, we set up online stores to sell our company merch. And for walkers, we took days off to deliver real orders to experience what it’s like. By doing so, we gained more context and identified more problems—bits and pieces that we would never get by simply talking to people or observing them.
As product managers, designers, and engineers, we’re good at spotting problems. It’s even easier when we’re doing something new. For example, plastic utensils from a popular fast-food restaurant chain in Hong Kong. For many years, the restaurant group packed their takeaway utensils with the fork and spoon facing one direction, while the knife faces the opposite direction. The first time I opened the utensil packaging, it took some time to realize the knife is turned the other way and flip it over. After opening it a few times, I got used to it and forgot about it.
A few years ago, my dad ripped open this utensil pack from the fork and spoon’s handle side (as a normal person would) and accidentally cut by the sharp side of the knife. He started bleeding and got really pissed off, so he called the customer service hotline and complained. Since then, the restaurant chain has changed its takeaway utensil pack so that all the utensils face the same way.
When we do something new for the first time, we can identify problems easier because there’s more cognitive load initially. We can leverage this when doing real experiences. In fact, getting more teammates to partake in observation and experiences helped us identify, decide, and tackle pain points.
Multiply the effect
When you give a talent-dense team the best context, it results in more ownership and better ideas. When we speak or write about our research findings, it creates another layer of words between people. We found that involving the team in the research process increases their motivation and ownership of the solutions.
In our same-day delivery team at GOGOX, we encourage all team members to go out to be a courier for a day to understand the supply side. After their experiences, I found that they contributed more during the design sprints and workshops. It led to solutions that created more impact. For example, the team brainstormed ideas based on their pain points during the courier experience day: region filters, package size, and weight filters, order suggestions, order bundling… The filter and sorting idea that we shipped resulted in a 180% increase in orders taken by couriers, and an 83% reduction in response time.
To summarize, here are the steps to improve your qualitative research:
- Observe people in their natural environment to spot new problems
- Conduct real experiences to gain context
- Encourage your team to do the same to increase the outcome
By doing so, your team will unlock pain points and opportunities that others may not be aware of, which can potentially lead to a greater outcome when tackling problems. Furthermore, the team will learn to trust the design process instead of relying on words of mouth, especially when the culture is less direct when it comes to giving feedback.
This is not to say to neglect the other research methods. Observing people may uncover new behavior problems, but may not always explain their behavior. Conducting real experiences is time-intensive, especially when the experience is difficult to reproduce or requires prior knowledge. These drawbacks can be overcome by conducting a wider range of research methods. The different research methods complement each other and will give your team a more complete understanding.
To improve the current process, I would schedule more frequent observations or real experiences with the team instead of doing it yearly or one-off (like onboarding). During the pandemic, the supply and demand for our delivery service changed rapidly, and we were blind to pain points until we started receiving courier and customer feedback. I felt like we discovered the problems too late and could’ve identified them if we conducted real experiences during this time, which we didn’t do due to safety concerns. In the future, when we’re all vaccinated and safe, doing more frequent observations and real-life experiences will definitely benefit the team’s discovery in a fast-changing environment.
This write-up is based on my talk at Product Management Night Hong Kong. Special thanks to Vincent Chan, Robert Dong, waikit kan, the Delivery team, and the Design and Continuous Discovery team Gemmy Wong, Matilda Kan, Lili Chan, and Franco.
