Demystifying User Research: A Beginner’s Guide for Product Managers

Introduction – Who this article is for and what you’ll learn

User Research or User Experience Research has many similar names: Customer Insights, Product Discovery, Usability Testing, Usage Data. Over the last 20 years, it’s become synonymous with a systematic approach to tailoring your product to meet user needs, anticipate success or failure, and honing your business sense and street smarts. In this article, I use Customer Insights for the outcome, or the result; I use User Research, or User Experience Research, for the activity that’s leading to customer insights.

The goal of this article is to help product managers understand effort, risk and rewards involved in User Research, and to guide junior product managers towards their first user interviews. 

Similar to other activities like quantitative data analysis, user experience research is usually done by professionals. Qualitative and quantitative user research is a university level career in psychology, sociology, or human factors. So the goal is not to become or replace professional researchers, but to get going with a few tips.

The focus of User Research changes as product managers progress in their career:

• The target audience for this article are juniors – entry-level product managers. If you are at this stage, this article may be a start in understanding some of the first rules of the trade, and to understand your own knowledge gaps and weaknesses. There may be times when the likely effort isn’t worth the return. A list of further recommended reading is at the end of this article.
• Mid-level senior product managers and group product managers. There, the focus is on how to do product discovery that helps build a product from zero to one, or to help a company that needs to pivot their product market fit in the light of a changed market environment. Product managers on this level usually partner with in-house UX research teams or with a specialized vendor. If you lead a team, you may think about customer insight skills to hire better team members or to make smarter promotion decisions.
• Director of product management, VPs, and executives. At this level, the goal of user research is mostly about how to guide your team, and to set a high bar in terms of quality. Further topics include how to build and use an internal user research team, how to use user research to understand market trends, and how to get steeped in user behaviour to shape the next 3-5 years, as well as preparing and nailing radical product pivots. For this level, I recommend Katie’s excellent post: ‘How do you set up a Design and User Research function properly?’ 

Why bother with User Experience Research and Customer Insights?

“Delivering features doesn’t mean you are delivering value, just like telling a joke doesn’t mean people will laugh. It’s all about how customers receive your feature and if it helps them to meet their goals.”

– Maarten Dalmijn, former Head of Product at Rodeo, in Why roadmaps reflect the level of Agile inadequacy 

User experience research is about finding and delivering value to users. If your joke doesn’t land, why bother telling it? 

User experience research goals change for different phases of the product life cycle.

Early Stage: Run product discovery in order to find product-market fit

Mid Stage: Expand beyond the initial customer base and business models; be aware of risks and dangers.

Maturity Stage: Run usability tests to launch incremental improvements to maintain the market position.

Crisis: Use in-depth user experience research to radically pivot your business model and product to accommodate a different user goal, create value for a different user group, or both.

There are many other benefits of running user experience research. One major one that’s easily overlooked is that in a product team, everyone is locked in and sees their product and market potential in a narrow and restricted view. User research adds to the diversity of viewpoints, it allows product managers to get the point of view of people who are new to your product, have never heard of it, are happy users, adjacent to happy users and potential new customers, disgruntled customers with a beef, etc. In my professional life, I’ve experienced first hand that sitting in and conducting user interviews has prepared me to predict economic success or the downfall well in advance.

User and Customer Research – one of the key skills for product managers 

Listening to users and customers, then deriving meaningful insights out of it, has long been a key skill for successful product managers. It is mentioned in a series of advice for building a powerful product management organization. 

For instance, User and Customer Knowledge is the very first skill mentioned in the influential article by silicon valley product group co-founder Marty Cagan.

Similarly, Customer Insight is one of the four main skill sets the product management influencer Ravi Mehta has identified for product managers. User Research typically spans the ‘voice of the customer’ and ‘fluency with data’ categories, with a nod to user experience design. I’m using Customer Insights and User Research as loosely equivalent terms in this article, but I’m aware that there are nuances depending on the field and discipline.

Few product managers excel in all skill sets. It depends on their background, their industry, their seniority level, and connections to experienced mentors. However, not all skill sets are the same. In a 2023 survey, almost half of product managers said they have a technical background. Compared to other skill sets, psychology and sociology degrees are rare in software product management. 

Therefore, while it’s a key skill and mentioned first in many lists, many product managers don’t have hands-on knowledge of solid user research methods when starting out.

My list of 6 tips before you start

Here are my 6 tips for aspiring people who do user research:

1. Educate yourself. Read at least the book Rocket Surgery Made Easy | Steve Krug. It’s short and to the point. You can read it over a weekend and benefit from it for a lifetime. I also recommend watching Steve’s demo video on YouTube. In parallel, many will benefit from watching an short introduction to UX Research like Amanda Stockwell’s course on Linkedin Learning
2. Embrace coaching, invite criticism, and ask for mentorship. Asking unbiased questions is an art more than a science and needs practice and constant feedback by seasoned interviewers.
   For example, you could say to an experienced user experience researcher: “I’d like to avoid the most common pitfalls in user interviews. Would you mind observing me and giving me pointers where I can improve?”. Typically, it takes 5-10 interview session feedbacks to be calibrated. Most professional user researchers have conducted hundreds, some even thousands of interviews.
3. Differentiate between anecdotes, data, and insights. For instance, if you remember when a user expressed frustration with a feature, it’s an anecdote. Transcribing and organizing multiple user interviews provides data. Only analyzing this data to identify a common issue offers insights.
4. Garbage in – garbage out. Don’t taint your data by doing most of the talking yourself, by asking leading questions, or by letting other guests take over and explain the product to users. Listen, listen, listen.
5. Triangulate data (combine quantitative and qualitative data, different markets, target audiences, include internal teams).
   For example, you could compare user feedback from the US and Europe to understand regional differences.
6. You don’t have to be an expert at everything in the beginning; instead, stay humble, start slow, give it a go and continue improving as you go. See tip 1 and 2 above (read a book, ask for mentors).

Many Product Managers are in a situation where they don’t have access to a UX Researcher to help them. Practically speaking – how do you go about talking to users in a smart way so your effort isn’t wasted and has maximum impact? 

Here are 6 typical steps of a qualitative research study:

1. Define goals and expected outcomes, clarify expected impact on business and tech decisions, define target audience and means of recruitment; then, write down your methods of research as well as method of distribution of results.
   Example: You noticed many users abandon the flow after the ‘forget password’ flow; you’re interested in investigating the issue, correcting the flaw, and significantly increasing user retention. A series of interviews with users who abandoned the flow should bring light to the issue.
2. Recruit 5-7 users of the target audience;
   Example: From an invitation link on the step 2 page, you’ve successfully recruited users who are willing to talk about their experience. Oftentimes, it’s more interesting to talk to someone who is critical or who doesn’t even use your product at all than to a fan.
   There’s a history to why many UX professionals recommend 5-7 user interviews, but for now, it’s important to understand the difference between qualitative versus quantitative research. There’s no hard rule and indeed, there are long-term in-depth qualitative studies that only observe one or two users. Generally, I consider that there’s diminishing value if we talk to 8 or more users on one topic. Beyond, there’s a chance that we don’t spend enough time listening and re-listening to the recordings, or that we only care about the most superficial first impressions. Counterintuitively, the value of a study usually radically diminishes after that. However, a UXR professional might develop a study design that accommodates and works around these challenges.
3. Collect data. Be humble in conversations and make it a good and worthy experience for your interview partners. Your primary goal is to learn and listen, not to explain or brag. Before you get to a tool, ask about user goals, context and only then show a prototype and let them think out loud. Take loads of notes. Include their exact words as much as possible.
   This is the most daunting step for many newbies – make sure to be well prepared with some of the resources linked below.
   Example: A user may insist on calling the forgotten password flow “I don’t have time for this”. Put it down exactly like this, not your internal name, e.g. Password Reset Flow, or worse, the PRF.
4. Clean up data; Transcribe raw conversations; don’t over-rely on automatic transcription; Taking and relying on your own notes will speed up your analysis and understanding 10x. Nobody ever reads 100 pages of automated transcripts – at least, that’s my experience.
   Example: Organizing notes and reviewing videos takes time. Per one hour interview, think of three hours to collect your thoughts, correct typos, and bring notes to a comparable format.
5. Analyse – sit down with your raw notes and look for patterns. Steep in the data, surround yourself with your data, refrain from social media and mobile phone usage in your bed, bus, and bathroom. Think about the patterns in unusual places, not just at your desk. Combine insights with recommendations for change. The first customer quote is not always right and only time will tell if a user quote is important or part of a pattern.
6. Add recommendations and share, share in 2 different ways with 2 different target audiences, archive into long-term storage

What to do with User Research Insights?

Collecting user or customer insights is not a goal in and of itself – they should be used in conjunction with other aspects as well.

• Customer Insights can be used in Product Execution, informing Feature Specification and Product Quality Assurance stages, for example by defining meaningful KPIs and by setting alerts.
• Product Strategy – Customer Insights should inform possible business outcomes, provide quality gates, inspire and fact-check product vision documents.
• Influencing People – Customer Insights convince stakeholders in Tech, Sales, and Management that the product manager knows what they’re talking about and have their ears on the ground. It can convince both teams you work with as a lead as well as with managing up.

Last but not least, customer insights increase the product confidence and influence important product gates along the way. A project or feature idea may be born out of buzzwords, the opinion of a senior executive, or the opinion of a large potential customer.

Until there’s launch data, user experience research helps increase product confidence from zero to high along the way. 

It may even help you tell better jokes.