Human - Technology Interaction Archives

October 3, 2023 — No Comments

The Magic: Illusion of Completeness (Generative AI Part 1)

If you are trained as an architect or designer, you understand the feeling of never-ending decision fatigue when it comes to design elements. Or even the decisions that go into crafting an email, where nuances in wording can convey subtleties in meaning. When Generative AI tools, such as ChatGPT or Dall-E, came along, leaving it up to “fate” was easy after typing in the input prompt. One is either wildly surprised and in awe of what it produced or extremely disappointed that it didn’t understand your intentions.

As the digital era progresses, businesses across various industries are continually exploring ways to harness the power of artificial intelligence (AI) to enhance their processes and outputs, whether that’s a business plan, a short movie trailer, or a sales email. By design, GenAI can create new content from scratch - be it written text, images, music, or any form of digital content - mirroring the style, context, and nuances of its training data, which can seem magical.

However, within this evolving landscape, it's essential to understand the intricacies of Generative AI and its role in shaping our perception of artificial intelligence outputs. At the heart of this is a phenomenon known as the 'Illusion of Completeness.'

The 'Illusion of Completeness' refers to the perceived accuracy or wholeness of AI-generated content, which might seem perfect to the untrained eye but may not exactly align with the specific requirements, subtleties, or context intended by the user. This illusion is influenced by various factors, including our brain's neurology, cognitive biases, and even our intrinsic appreciation for beauty. To understand this phenomenon, let's delve into some of the factors contributing to the Illusion of Completeness in Generative AI:

Value to Effort Ratio:

In this context, the value-to-effort ratio refers to the disproportionate relationship between the minimal amount of user input and the substantial amount of AI output. Generative AI can produce large volumes of content or highly complex outputs with just a short prompt or a handful of parameters. Years ago a speed drawing challenge asked artists to sketch the same drawing in 10 minutes, 1 minute, and 10 seconds. With Dall-E, any drawing or image can be produced in 20 seconds, the amount of time it takes to generate a batch of 10 photos.

A drawing of an eye in 10 minutes, 1 minute, and 10 seconds by rachelthellama

Example of an eye drawing in 10 minutes, 1 minute, and 10 seconds (by rachelthellama).

Left: DALL-E2 Prompt: 3d realistic render, maya, ambient studio light, splash page image, sci-fi, futurism, greenery, aerial view, A city of bikes, scooters, pedestrians friendly city,
Right: Dall-E2 Prompt: Future of mobility workshop and symposium poster without text (bad prompting).

The image on the left could be categorized as low effort high value because it is perceived to be more coherent and labor intensive. The image on the right seems to be low effort low value, as we are more able to recognize errors in texts, just like we can recognize the famous example of a 6-fingered hand generated by AI is not a hand we recognize. From a psychological standpoint, this abundance of output against a minimal input can heighten our perception of completeness. This is seemingly the opposite of the effort justification effect in cognitive dissonance, where the individual experiences wonder despite the seemingly insignificant input.

*The “Effort justification” is a phenomenon whereby people come to evaluate a particular task or activity more favorably when it involves something that is difficult or unpleasant (APA).

Perceived Coherence:

While generative AI models like ChatGPT and Dall-E can produce impressive outputs, they lack true comprehension and contextual understanding. However, despite their limitations, they can lead to outputs that might seem internally coherent visually complete, and contextually appropriate at times but can also fall short in other instances.

Many times, the perception of coherence lies in the ambiguity and conciseness of both the user's inputs and the AI-generated outputs. The input prompts provided to generative AI models can be concise and open-ended, leaving room for interpretation. Users may assume that the AI understands their intentions fully and will generate outputs aligned with their expectations.

Here’s an example:

In this example, the response attempts to convey coherence and relevance to the topic of AI's societal impacts. However, the content lacks depth, true understanding, and specific examples to substantiate the claims.

Example of ChatGPT output, achieving the illusion of completeness.

Example of how bolding and similar sentence lengths achieve the illusion of completeness.

Fill in the _______

From a neuroscientific viewpoint, our brains are naturally wired to fill in missing information. This survival-oriented mechanism aids us in interpreting the world around us. A phenomenon called "filling in" happens when the brain "fills in" missing information in a person's blind spot. Reality is a construction of our brain, and the brain has evolved for survival, not accuracy. Consequently, when we examine an AI output, our brain instinctively completes any apparent gaps, making the result seem 'whole' even if it lacks certain aspects. Many times, it's easier to believe that AI can do more than it can.

Emotional Attachment:

When users witness generative AI producing something remarkable and aligned with their desires, they may develop an emotional attachment to the output. Specifically, the effort the user exerted to produce the prompt that led to the generated output can create a certain feeling of “ownership” towards the generated output, whether that’s a text, animation, or image. This emotional response further reinforces the belief that the AI has grasped its intent comprehensively.

Confirmation Bias

Confirmation Bias is a cognitive bias that affects our interaction with AI outputs. This bias causes us to process new information in a way that affirms our existing beliefs or expectations. So, if the AI's output is somewhat aligned with what we expect, our brain is inclined to view the output as more precise than it might be. For example, the certainty at which ChatGPT generates false information can trick many. On the other hand, if the AI output is not what we expected, we might dismiss it and give it less weight in our minds or continue to edit the prompt until the desired output matches our input and coincides with the user’s preexisting beliefs and thoughts.

Summary: Illusion of Completeness in Generative AI

These cognitive biases are more complex than described in this article. These are merely a glimpse to showcase why AI feels magical. And is it too good to be true? How can individuals and teams mitigate the Illusion of Completeness in Generative AI and think critically about our usage of Generative AI?

Promote Awareness: Users and consumers of generative AI should be educated about the technology's limitations. Schools, governments, local communities, and companies should teach how generative AI works. Understanding that AI models lack genuine comprehension and can produce unpredictable and incorrect results will foster more realistic expectations.
Iterative Hybrid Initiatives: Encouraging collaboration between AI and human creators, where the AI assists and the human guides, can lead to more reliable and contextually accurate results. The human plays a critical role in providing feedback to fine-tune the machine. This will allow more diligent tracking of AI outputs and user control over generated content.
Clear Communication of Intent: Users should be educated about prompt design and encouraged to provide more specific and explicit prompts to avoid misinterpretations by the AI model. A bad prompt is misleading, unclear, and ambiguous, leaving much to be “misinterpreted” by the machine, leading to irrelevant, inappropriate, ineffective misleading, inadequate, or biased outputs.

The Illusion of Completeness in Generative AI stems from a combination of factors related to human cognition, AI limitations, and expectations. By being aware of these factors and adopting suitable strategies, we can harness the true potential of generative AI while maintaining realistic expectations about its capabilities. As AI continues to evolve, understanding the nuances of human-AI interactions becomes increasingly critical for creators and consumers of AI-generated content.

Published by Isa He in Human - Technology Interaction, Uncategorized

July 15, 2023 — No Comments

Unlocking the Future of UX with Biometric Research

The trajectory of innovation is never static; it curves and flexes, adapting to the nuances of our growing technological capability. As the era of digitization powers on, one tool is proving to be increasingly crucial in shaping the next generation of products, services, and processes: biometrics. In the intricate tapestry of user experience (UX) design, a myriad of sensing technologies can reveal the underlying threads that tie users to their spatial experiences. And it's micro-research that holds the magnifying glass to these threads.

Within the selection of sensing technologies, the two major types help illuminate how users interact with their environments: biometric sensors and spatial sensors. Biometric sensors including electrodermal activity (EDA), electroencephalography (EEG), and eye tracking, reveal users' automatic physiological responses. Sensor technologies measuring spatial experience include both environmental sensors and spatial sensors.

Measuring spatial experience

Spatial sensors are pivotal in deciphering the intricacies of spatial experiences by supplying data that augments our comprehension of human interactions in various settings. For instance, visual sensors record and assess individuals' motions and engagements, detecting behavioral patterns and responses to environmental elements. Audio sensors record ambient sounds, aiding researchers in discerning the auditory cues that mold users' sentiments and perceptions. Sensors measuring temperature and humidity offer a glimpse into a space's thermal comfort, which influences the occupants' overall well-being. These are only some of the many types of sensors that can be used to measure our spatial experience. By integrating data from these multifaceted sensors, designers can comprehensively examine the multi-faceted nature of spatial experiences, enabling them to make informed decisions in the design and optimize the experience for various purposes.

Measuring users’ physiological responses

Biometric research is a specialized form of UX research that drills down into the intricate details of user interaction with technology. While macro and middle-range research provide necessary strategic frameworks and understandings of product development respectively, micro-research delves into the specifics of technical usability and minute interaction points. It navigates the granular sea of users' emotional responses, physiological signals, and behaviors, bringing to the surface insights that are often invisible to traditional research methodologies. Some examples of biometric sensors include:

EDA

One potent tool in the micro-research arsenal is EDA. This biometric method tracks the changes in skin conductance linked to emotional arousal. By quantifying these responses, businesses can gain an in-depth understanding of how users react emotionally to their products, services, or processes. This can uncover invaluable data, allowing organizations to fine-tune their offerings and craft experiences that resonate more deeply with their target audience.

EEG

Similarly, EEG, which measures brain wave activity, provides a unique window into a user's cognitive processes. It provides real-time feedback on a user's engagement, concentration, and stress levels when interacting with a product or service. By leveraging this biometric data, businesses can understand their customers on a much more intimate level, identifying potential pain points and opportunities for improvement that would otherwise go unnoticed.

Eye Tracking

Eye-tracking technologies, another facet of micro-research, not only unveil how users visually interact with products or interfaces, but also can be used to pick out user arousal through pupil dilation triggered by the autonomic nervous system. They reveal when users are stressed or excited, as well as which elements draw the user's attention and which are overlooked, informing design decisions that can drastically improve usability and engagement. In a world saturated with visual information, understanding and optimizing for users' visual attention can be a game-changer.

Technologies that measure physiological signs of excitement or stress.

Examples of biometric data:

User-Centered Biometric Data Type	Relevance to research topics
Electrodermal Activity (EDA)	Used as a strong indicator of heightened physiological activity correlated with heightened emotional states.
Electro-encephalogram (EEG)	Used as a strong indicator of cognitive load, focus and attention, and confidence level in decision making.
Eye-tracking	Pupil dilation in response to emotional stimuli, a strong predictor of stress and excitement.

Examples of User-Centered Activity Data:

User-Centered Data Type	Relevance to research topics
Accelerometer (ACC)	Shows the amount and speed of bodily movement
Global Positioning System (GPS)	Shows exact geographical location which can be overlaid on maps and plans.
Audiovisual (AV)	Shows a first-person perspective view of what was visible to the research participant, ability to pick up t activity, human interactions,
Eye-tracking	Shows how users visually interact with products or interfaces, can reveal elements that draw significant user's attention and which are overlooked

Examples of Environmental and Spatial Data

Environmental/Spatial Data	Relevant Characteristics & Validation Studies
Temperature, Pressure, humidity	- Proven indicator of environmental comfort, productivity and efficiency, functional comfort, physical comfort and psychological comfort.
Analog gas sensor and ADS1015 analog-to-digital converter (ADC)
Light and Proximity	- LUX levels can provide information of the transition from indoor to outdoor spaces, or proximity to windows, and availability of natural light- Shows also the proximity of other objects or humans to the user.
Microphone (Audio)	- Audio levels in decibels, audio sources, frequency, pitch.
Particulate Matter (PM) Sensor	- Indicator of pollution, visibility, and building ventilation.
Visual Data (Camera)	- Shows first-person perspective view of what was visible to the research participant- Spatial Depth- Object/Human movement - visible color, visible spatial depth, spatial information, and audio information.
Spatial depth (LiDAR)	- Shows mapped structures, spatial characteristics, configurations, movement of objects, etc.

The above quantitative data types collected by the methodology and their relevance expanded from previous research by He. et al.

Harnessing the Power of Biometric-Research

These biometric tools, wielded within the context of UX research, empower businesses to go beyond traditional metrics and benchmarks. They provide a robust, nuanced understanding of user experiences that transcends what users say or think. It taps into what they feel, their automatic physiological responses to interactions and impressions, and what they do, the patterns throughout the user journey. Fine-grained micro-level research essentially bridges the gap between the human and the digital, enriching business strategies with human-centric data. By integrating biometric methods into traditional UX research approaches, businesses can gain insights that drive innovation, improve customer satisfaction, and ultimately, bolster the bottom line. The power of biometrics, specifically within the context of micro-level research, isn't just the future of UX research - it's the future of business.

As we navigate the vast landscape of technological innovation, we can use more biometric data to guide businesses to align their strategy, product development, and intricate usability with the needs and desires of their users. It's through these scales that we can truly unlock the power of user experience, propelling our businesses towards a future designed with the user at its heart.

The devil is in the details. By tapping into the potential of fine-grained, in-depth, micro-level research, businesses can ensure they are not just designing for users, but are designing with them.

Published by Isa He in blog, Design Ethnography, Human - Technology Interaction, Inclusive Design, Uncategorized

June 14, 2023 — No Comments

Future of UX Research

How necessary is UX Research? The field of UX Research has been growing steadily in the past 2 decades. More recently, it has come into attack because much of it is associated with “soft” science - subjective feelings and experiences. Frequently designers and engineers think that they are able to perform user research through a couple of interviews that can generate a journey map, but also because designs can be executed regardless of a comprehensive understanding of a problem or the user. They might not be good designs or good products, but they can be executed and pushed out to market. Why? Because with some packaging, they can seem “complete."

From an architect's perspective, the parallels between architecture and UX research are intriguing. Architects employ scalar thinking, examining designs at various scales, from the expansive urban landscapes down to the chair one might sit on, or the literal nuts and bolts of construction detail. This multi-scalar perspective is deeply ingrained in our architectural mindset, as it encompasses both the temporal and spatial scales. We contemplate the long-term implications of our designs, scrutinizing how a building might shape the development of a city, and the day-to-day interactions inhabitants have with a single room. And though architects could design a beautiful state-of-the-art building, it is not good design, unless it understands the context that surrounds it and the people who will inhabit it.

Scales of UX Research

In many ways, architecture and UX research share a common purpose: to create spaces or products that not only meet functional needs but also elicit positive emotional responses. Both disciplines require a deep understanding of the intended users, balancing aesthetics and functionality, while considering the minutiae of interaction details and the broader experience.

In our approach to UX Research, we borrow from our architecture design experience, and see UX Research in three main scales:

Context (years): This involves understanding the past, present, and especially future of the industry the business is in, and understanding how a product or service may fit into the long-term trajectory of a company or industry.
Journey (days): Research in this scale looks at the user and how they interact with the product or service. This is what is traditionally thought of as UX Research, journey mapping, empathy mapping, personas, etc, and much of what is seen in the industry today.
Interaction (seconds): This type of micro-analysis of user research, can help uncover the interaction minutiae, and how different interactions affect usability, user emotions, etc.

Challenges of Existing UX Research Methods

Understanding how UX research exists in scales can help companies visualize how to propel forward by utilizing the different scales of UX research, and not get trapped in UX research as it exists currently. As mentioned before, though there has been considerable progress in the field of user experience (UX) research over the last decade, several challenges persist:

Experience = Subjective: Despite the field being called User Experience Research, the actual experience is often dismissed as subjective and difficult to quantify, yet holding immense untapped potential for user insight.

Indirect & Intangible Benefits: The UX industry is currently awash with storyboards, user journeys, and experience maps. These methods offer crucial insights, if executed correctly, their benefits are often indirect and intangible. This poses challenges when trying to demonstrate the tangible value of UX research and garner buy-in from stakeholders.

Insufficient Quant Research on end-to-end user experience: Many existing methods are focused on delivering product, process, or service-specific metrics, and very few attempt to understand the user in a more profound and holistic manner. Quantitative research methods, while being instrumental in comparing performances of specific UX/UI interactions, fall short of providing a holistic quantification of the user experience. Both pre and post-design stages lack targeted quant efforts that dive into the broader tapestry of user experience with product or service.

Existing Quantitative UX Research

It might seem obvious that the way to move forwards is with more Quantitative UX Research. Here is the list of some of the most used Quant UX Research Methods

Method	Typically Used for	Cost	Difficulty of Collection	Difficulty of Analysis	Type	Context of Use
Quantitative Usability Testing	Tracking usability over time comparing competitors	Medium	Medium	Medium	Behavioral	Task-Based
Web Analytics (or App Analytics)	Detecting or prioritizing problems Monitoring performance	Low	Low	High	Behavioral	Live
A/B Testing	Comparing two specific design options	Low	Low	Low	Behavioral	Live
Card Sorting	Determining IA labels and structures	Low	Low	Medium	Attitudinal	Not Using Product
Tree Testing	Evaluating IA hierarchies	Low	Low	Medium	Behavioral	Not Using Product
Surveys and Questionnaires	Gather information about your users, their attitudes, and behaviors	Low	Low	Low	Attitudinal	Any
Clustering Qualitative Comments	Identifying important themes in qualitative data	Low	Medium	Medium	Attitudinal	Any
Desirability Studies	Identifying attributes associated to your product or brand	Low	Low	Low	Attitudinal	Task-Based
Eyetracking Testing	Determining which UI elements are distracting, findable, or discoverable	High	High	High	Behavioral	Task-Based

*NNGroup’s Table of Quantitative Methods

Quantitative methods in user experience (UX) research provide the framework for understanding the measurable aspects of a product's usability. These methods generate numerical data which can be statistically analyzed, providing clear, objective insights that are crucial for strategic decision-making. Current methods allow teams to:

Put a number on the usability of your product.
Compare different designs (eg. different versions, or competitor products)
Improve UX trade-off decisions, to see if different designs and features are worth the change.
Tie UX improvements back to organizational goals and key performance indicators

Though traditional quant methods, like the ones mentioned in the chart above, can provide data with a lot of usability insight, most are product/service-centric, asking questions like “How long did it take for the user to navigate to [Page A] instead of [Page B]?” they have a hard time quantifying questions such as “What, if anything, surprise you or frustrated you about the experience?”

Future of Quant UX Research Methods

Emotion related user experience questions like the one asked above are hard to quantify using traditional quant methods. Quant methods of researching subtleties in interactions, perception, and emotion are mainly used in academic studies, and scientific research, including using a myriad of sensing technologies to detect how various sensory inputs, spatial design, and interactions affect physical arousal. Usually seen in the fields of psychology, architecture, sociology, and neuroscience, sensing technology has been tested in fields from sports to retail, such as using eye tracking for desirability heatmaps, but is not yet mainstream.

Modified from NNG's Current Landscape of User Research Methods

The potentials of pairing different types of sensing technologies are infinite. Biometric methods like EDA, EEG, and eye-tracking, hold the promise of unlocking rich, nuanced insights into not just isolated user interactions but the entire user experience. This underutilization leaves a significant gap in our comprehension of the user experience, suggesting there is much room for growth and evolution in our approach to UX research, and for us to address deeper user needs, or currently unobserved pain points.

Future Landscape of User Research Methods using sensing technologies to quantify user experience.

Implementation of these methods addresses the three main challenges of traditional UX research

We can quantify user experience, in terms of a spectrum of pain points that contribute to stress triggers, irritations, and pain points, and start to know not only what the pain points are, but how they affect the users, seen through acute or gradual changes in physiological responses.
Benefits are tangible and direct because 1) this method can allow researchers to isolate and prioritize the biggest frustration, both observable and hidden and 2) the same sensing metric can be used to evaluate user experience before and after change.
Experiences are evaluated from end-to-end. With this type of quantitative method, the research team is able to quantify the entire experience. This is extremely useful when companies are looking to understand how their users are interacting with multiple touchpoints for their service or product. Its ability to cover the entire journey also allows companies to start asking the right questions earlier.

Summary:

Traditional UX research has frequently centered around post-design evaluation, analyzing results after key decisions have been made. While this approach certainly provides valuable insights, it could limit the potential for a proactive, data-driven design strategy. The future calls for a more anticipatory approach, one that uncovers statistically significant aspects of experiences ahead of time and enables teams to focus their efforts where it matters most. With the increasing integration of biometric micro-research and advanced quantitative methods, we're entering an era where UX research can not only guide post-design refinement but also own a greater role in steering the initial stages of design and decision-making. As we usher in this future-forward perspective, the role of UX research is expanding beyond analysis and evaluation, becoming an indispensable tool for strategic, user-centric innovation. It's no longer just about understanding what happened, but illuminating what could - and should - happen next.

Published by Isa He in Design Ethnography, Human - Technology Interaction, Uncategorized

August 22, 2022 — No Comments

How will we live in the future?

Design of everything affects how we live now and how we will live in the future - it revolves around our society, city, lifestyles and personal preferences. In a world where technology is constantly evolving, it's hard to imagine what life will be like in the future. But the unknown shouldn’t stop us from wondering how we will live. We should go beyond thinking about what we want in the future, but try to understand what future generations would want in their world. We cannot rely solely on the systems that have sustained us for generations. Instead we should think of speculative futures, where we live in a world with technology and gadgets that don’t exist yet. (If you are interested in the future and speculative design, you should read Speculative Everything by Anthony Dunne and Fiona Raby )

As our society, city and lifestyles become more complex, the design of everything around us has to take these changes into account. Our personal preferences are also increasingly important in the way things are designed, whether it’s the clothes we wear, the cars we drive or the homes we live in. The future technology that serves our everyday needs cannot only be about comfort and efficiency. Designers will need to go above and beyond to think how technology can be responsive and create delightful experiences that make our life more enjoyable.

So whatever the future holds, one thing is certain: design will continue to play a vital role in our lives.

Questions about the future

Telemedicine scene from Hanna-Barbera’s 1962 animated sitcom, The Jetsons. — A telemedicine scene from Hanna-Barbera’s 1962 animated sitcom, *The Jetsons*.

More than 50 years ago, the Jetsons featured a telemedicine scene that we are so familiar with now. Back then, it was hard to imagine how their hospital visits could change so drastically. Similarly, future generations will live in a world far more different than we know of today. Questions we should ask ourselves include:

How will our lives change, and what are the implications for our interactions with technology and the built environment?
What types of technologies now will impact how we live in the future? And how will these affect the development of future technologies?
Why do we need technology in the future? Will these reasons be different from why we need technology today?

As we think about what we want for our future, we should also try to understand what future generations would want in their world. How will they live? Will they have the same occupations and work schedules? What do they do for fun?

Thinking about speculative futures can help us break free from our dependency on possibly unsustainable systems. It can help us imagine different ways of living and being in the world. It can also help us create new systems that are more equitable, inclusive and just (If you are interested in Inclusive Design, check out our project with Microsoft).

Dive in:

Future Work
Future Living
Future Mobility

Published by Isa He in blog, Design Ethnography, Human - Technology Interaction, Inclusive Design