AI Moderators Uncover Deeper Consumer Truths Due to Perceived Neutrality

Original source: Humans of Martech Podcast
This article is an editorial summary and interpretation of that content. The ideas belong to the original authors; the selection and writing are by Streamed.News.

This video from Humans of Martech Podcast covered a lot of ground. 5 segments stood out as worth your time. Everything below links directly to the timestamp in the original video.

Ever wondered why people might tell more to a machine than a person? This insight reveals how AI's perceived neutrality is reshaping our understanding of human candor, especially on sensitive subjects.

AI Moderators Uncover Deeper Consumer Truths Due to Perceived Neutrality

Artificial intelligence moderators are proving uniquely capable of uncovering uncomfortable truths and deep-seated fears that human interviewers might not access. Individuals exhibit a greater willingness to disclose sensitive information, such as personal finances, insecurities, or medical situations, when interacting with an AI, perceiving it as a neutral and non-judgmental entity. This candidness allows researchers to gain deeper insights into motivations and blockers. This phenomenon suggests that the non-human nature of AI can bypass social inhibitions, leading to more honest and open responses, even though participants are aware that human experts will eventually review the data. The ability of AI to engage in dynamic follow-up questions further enhances its capacity to explore nuanced personal situations, offering a novel approach to qualitative research that prioritizes unvarnished truth.

"People have this tendency to be more willing to open up to an AI moderator for like things about their finances or things about their insecurities or about their medical situations because it's sort of neutral."

▶ Watch this segment — 42:07

Expert Warns Against Using LLMs for Pricing, Advocates Diverse Synthetic Personas

A critical aspect of effective synthetic user research involves moving beyond single, monolithic personas to represent a broad distribution of personalities and diverse traits. Rather than creating one "average nurse" for example, researchers should model a range of individuals with varying levels of openness to change, nervousness, or different family situations. This approach allows for a more comprehensive understanding of user motivations, supported by the ability to conduct thousands of simulated trials. Furthermore, the speaker emphasizes the continuous interrogation of "why" to uncover deeper motivations behind user behavior. However, a significant caution is issued against using Large Language Models (LLMs) like ChatGPT for magnitude-based questions, particularly those related to pricing, as these models lack fundamental grounding in concepts like time, space, or money, leading to unreliable quantitative judgments. Such tools are better suited for qualitative insights and generating hypotheses.

"The one question I heard that I'd be really concerned about is like how much would you pay for this? That's a classic thing of grounding where the LLMs just don't have a concept of of you know time, space, money."

▶ Watch this segment — 23:56

Synthetic Users Build Internal Buy-In for Deeper Human Research

Synthetic users serve as a crucial stepping stone in market research, functioning as a proof of concept that helps build internal organizational support and budget for more extensive human studies. By allowing stakeholders to quickly explore diverse market segments—even those traditionally hard to reach—these AI-driven tools can provide initial indications and generate hypotheses. This early exploration helps identify promising areas for deeper, resource-intensive human-centered research. Beyond generating initial insights, synthetic users also act as a powerful tool for refining stakeholders' understanding of their own research questions. When confronted with immediate, albeit synthetic, responses, stakeholders often clarify their true interests and objectives. This iterative process, which can include asking "impossible questions" that would never be posed to real individuals, helps uncover the core business problems to be solved and ensures subsequent human research is precisely targeted.

"It doesn't matter almost what that synthetic user said. It's more a tool to get us at really understanding the heart of the problem that we're after solving."

▶ Watch this segment — 11:47

Synthetic Users Uncover "Why" Behind Marketing Funnel Behavior

Synthetic users offer a powerful method for uncovering the underlying "why" behind behavioral data within marketing funnels, addressing a critical gap where observed actions (like dropping off a pricing page or unsubscribing) lack clear motivations. This approach involves leveraging AI to interrogate existing interview data, seeking deeper insights into user decisions and preferences. It is particularly effective for understanding hard-to-reach or extreme user segments, such as highly specialized hobbyists or high-net-worth individuals who are typically unresponsive to conventional research incentives. While synthetic users can generate hypotheses about potential motivations, the speaker cautions that these insights should be treated as "converging lines of evidence" rather than definitive truths. Researchers must maintain a balance of "honest curiosity" and healthy skepticism, cross-referencing AI-generated possibilities with other available data. The appropriate application of synthetic users varies significantly depending on the stakes, from consumer product advertising to more critical fields.

"These groups that you can't reach out to at all are a great example of let's get some thoughts about what might be the real possibilities here and then what evidence do we have to support or or against that."

▶ Watch this segment — 55:04

Non-Human "Synthetic Users" Boost Stakeholder Interest in Real Human Research

Contrary to intuition, the use of non-human "synthetic users" is effectively humanizing market research by significantly increasing stakeholder interest in engaging with real human subjects. Traditionally, static research reports offer limited interaction, but synthetic user tools allow stakeholders to ask continuous "what if" and "why" questions. This dynamic questioning process helps generate new hypotheses and uncover previously unimagined possibilities regarding product viability or market strategies. This iterative interaction transforms synthetic users into a "preview" for genuine human research, preparing stakeholders for the diverse range of responses they might encounter. By allowing leaders to explore numerous scenarios and refine their understanding of potential user behaviors, synthetic users ultimately lead to a realization: if these simulated insights hold true, then rigorous testing with real human participants becomes essential. This approach fosters a more informed and engaged approach to human-centered design and market validation.

"At a certain point, they're like, 'Wow, if that could be true, this could be a really interesting angle. We need to go test that with our real humans.'"

▶ Watch this segment — 8:05

Also mentioned in this video

• Synthetic users as a potential standard for testing ideas and exploring… (0:00)
• Dr. John Whan defines synthetic user research, suggesting alternative terms… (3:26)
• Dr. Whan discusses the current reluctance and lack of awareness surrounding… (6:28)
• Concerns that synthetic users might miss emotional depth by acknowledging… (14:49)
• Synthetic users are built using both professional SAS products and from scratch… (17:47)
• The process of building synthetic users involves analyzing interview data for… (22:04)
• To synthesize existing human research, behavioral data, and various internal… (30:48)
• The Maven course teaches participants to use SAS tools and build agentic… (35:29)
• Traditional average personas are becoming obsolete because new systems can hold… (41:05)
• A common insight from using synthetic users is the heightened engagement of… (44:42)
• The debate between quantitative and qualitative market research, suggesting… (47:36)
• The importance of a gradual and pragmatic approach to introducing synthetic… (52:17)
• Skepticism about analysis paralysis and the theoretical nature of synthetic… (57:53)
• The shift towards designing for both human and agent users, acknowledging the… (1:01:43)
• John Whan shares his personal system for staying aligned with happiness, which… (1:04:22)

Summarised from Humans of Martech Podcast · 1:07:29. All credit belongs to the original creators. Streamed.News summarises publicly available video content.