How necessary is UX Research? The field of UX Research has been growing steadily in the past 2 decades. More recently, it has come into attack because much of it is associated with “soft” science - subjective feelings and experiences. Frequently designers and engineers think that they are able to perform user research through a couple of interviews that can generate a journey map, but also because designs can be executed regardless of a comprehensive understanding of a problem or the user. They might not be good designs or good products, but they can be executed and pushed out to market. Why? Because with some packaging, they can seem “complete."
From an architect's perspective, the parallels between architecture and UX research are intriguing. Architects employ scalar thinking, examining designs at various scales, from the expansive urban landscapes down to the chair one might sit on, or the literal nuts and bolts of construction detail. This multi-scalar perspective is deeply ingrained in our architectural mindset, as it encompasses both the temporal and spatial scales. We contemplate the long-term implications of our designs, scrutinizing how a building might shape the development of a city, and the day-to-day interactions inhabitants have with a single room. And though architects could design a beautiful state-of-the-art building, it is not good design, unless it understands the context that surrounds it and the people who will inhabit it.
Scales of UX Research
In many ways, architecture and UX research share a common purpose: to create spaces or products that not only meet functional needs but also elicit positive emotional responses. Both disciplines require a deep understanding of the intended users, balancing aesthetics and functionality, while considering the minutiae of interaction details and the broader experience.
In our approach to UX Research, we borrow from our architecture design experience, and see UX Research in three main scales:
- Context (years): This involves understanding the past, present, and especially future of the industry the business is in, and understanding how a product or service may fit into the long-term trajectory of a company or industry.
- Journey (days): Research in this scale looks at the user and how they interact with the product or service. This is what is traditionally thought of as UX Research, journey mapping, empathy mapping, personas, etc, and much of what is seen in the industry today.
- Interaction (seconds): This type of micro-analysis of user research, can help uncover the interaction minutiae, and how different interactions affect usability, user emotions, etc.
Challenges of Existing UX Research Methods
Understanding how UX research exists in scales can help companies visualize how to propel forward by utilizing the different scales of UX research, and not get trapped in UX research as it exists currently. As mentioned before, though there has been considerable progress in the field of user experience (UX) research over the last decade, several challenges persist:
- Experience = Subjective: Despite the field being called User Experience Research, the actual experience is often dismissed as subjective and difficult to quantify, yet holding immense untapped potential for user insight.
- Indirect & Intangible Benefits: The UX industry is currently awash with storyboards, user journeys, and experience maps. These methods offer crucial insights, if executed correctly, their benefits are often indirect and intangible. This poses challenges when trying to demonstrate the tangible value of UX research and garner buy-in from stakeholders.
- Insufficient Quant Research on end-to-end user experience: Many existing methods are focused on delivering product, process, or service-specific metrics, and very few attempt to understand the user in a more profound and holistic manner. Quantitative research methods, while being instrumental in comparing performances of specific UX/UI interactions, fall short of providing a holistic quantification of the user experience. Both pre and post-design stages lack targeted quant efforts that dive into the broader tapestry of user experience with product or service.
Existing Quantitative UX Research
It might seem obvious that the way to move forwards is with more Quantitative UX Research. Here is the list of some of the most used Quant UX Research Methods
|Method||Typically Used for||Cost||Difficulty of Collection||Difficulty of Analysis||Type||Context of Use|
|Quantitative Usability Testing||Tracking usability over time comparing competitors||Medium||Medium||Medium||Behavioral||Task-Based|
|Web Analytics (or App Analytics)||Detecting or prioritizing problems Monitoring performance||Low||Low||High||Behavioral||Live|
|A/B Testing||Comparing two specific design options||Low||Low||Low||Behavioral||Live|
|Card Sorting||Determining IA labels and structures||Low||Low||Medium||Attitudinal||Not Using Product|
|Tree Testing||Evaluating IA hierarchies||Low||Low||Medium||Behavioral||Not Using Product|
|Surveys and Questionnaires||Gather information about your users, their attitudes, and behaviors||Low||Low||Low||Attitudinal||Any|
|Clustering Qualitative Comments||Identifying important themes in qualitative data||Low||Medium||Medium||Attitudinal||Any|
|Desirability Studies||Identifying attributes associated to your product or brand||Low||Low||Low||Attitudinal||Task-Based|
|Eyetracking Testing||Determining which UI elements are distracting, findable, or discoverable||High||High||High||Behavioral||Task-Based|
Quantitative methods in user experience (UX) research provide the framework for understanding the measurable aspects of a product's usability. These methods generate numerical data which can be statistically analyzed, providing clear, objective insights that are crucial for strategic decision-making. Current methods allow teams to:
- Put a number on the usability of your product.
- Compare different designs (eg. different versions, or competitor products)
- Improve UX trade-off decisions, to see if different designs and features are worth the change.
- Tie UX improvements back to organizational goals and key performance indicators
Though traditional quant methods, like the ones mentioned in the chart above, can provide data with a lot of usability insight, most are product/service-centric, asking questions like “How long did it take for the user to navigate to [Page A] instead of [Page B]?” they have a hard time quantifying questions such as “What, if anything, surprise you or frustrated you about the experience?”
Future of Quant UX Research Methods
Emotion related user experience questions like the one asked above are hard to quantify using traditional quant methods. Quant methods of researching subtleties in interactions, perception, and emotion are mainly used in academic studies, and scientific research, including using a myriad of sensing technologies to detect how various sensory inputs, spatial design, and interactions affect physical arousal. Usually seen in the fields of psychology, architecture, sociology, and neuroscience, sensing technology has been tested in fields from sports to retail, such as using eye tracking for desirability heatmaps, but is not yet mainstream.
Modified from NNG's Current Landscape of User Research Methods
The potentials of pairing different types of sensing technologies are infinite. Biometric methods like EDA, EEG, and eye-tracking, hold the promise of unlocking rich, nuanced insights into not just isolated user interactions but the entire user experience. This underutilization leaves a significant gap in our comprehension of the user experience, suggesting there is much room for growth and evolution in our approach to UX research, and for us to address deeper user needs, or currently unobserved pain points.
Future Landscape of User Research Methods using sensing technologies to quantify user experience.
Implementation of these methods addresses the three main challenges of traditional UX research
- We can quantify user experience, in terms of a spectrum of pain points that contribute to stress triggers, irritations, and pain points, and start to know not only what the pain points are, but how they affect the users, seen through acute or gradual changes in physiological responses.
- Benefits are tangible and direct because 1) this method can allow researchers to isolate and prioritize the biggest frustration, both observable and hidden and 2) the same sensing metric can be used to evaluate user experience before and after change.
- Experiences are evaluated from end-to-end. With this type of quantitative method, the research team is able to quantify the entire experience. This is extremely useful when companies are looking to understand how their users are interacting with multiple touchpoints for their service or product. Its ability to cover the entire journey also allows companies to start asking the right questions earlier.
Traditional UX research has frequently centered around post-design evaluation, analyzing results after key decisions have been made. While this approach certainly provides valuable insights, it could limit the potential for a proactive, data-driven design strategy. The future calls for a more anticipatory approach, one that uncovers statistically significant aspects of experiences ahead of time and enables teams to focus their efforts where it matters most. With the increasing integration of biometric micro-research and advanced quantitative methods, we're entering an era where UX research can not only guide post-design refinement but also own a greater role in steering the initial stages of design and decision-making. As we usher in this future-forward perspective, the role of UX research is expanding beyond analysis and evaluation, becoming an indispensable tool for strategic, user-centric innovation. It's no longer just about understanding what happened, but illuminating what could - and should - happen next.