The increasingly technology-enabled environment in which companies operate is creating some unusual situations.
As companies leverage new AI tools to aid in an ever-widening range of tasks and customer interactions, and as consumers increasingly rely on these tools, it can be hard to tell what kinds of interactions are taking place.
Is a human interacting with a human?
Is an AI interacting with a human?
Is an AI interacting with an AI?
It’s that final scenario that is creating real discomfort as companies turn to AI agents to interact with customers to collect feedback, run pulse checks, flag renewal risks or validate assumptions.
Can a company be confident that what’s on the other end of that interaction is a human being rather than another AI? Often, the answer is that it doesn’t know. And that’s a problem.
The New Reality: AI-to-AI Interactions
Organizations can now easily and cost-effectively gather input and feedback from large customer populations through automated outreach, AI-conducted interviews and real-time sentiment capture. AI tools have made it possible to collect customer intelligence at scale across more accounts, more often, at a fraction of the cost of traditional research.
These tools, though, have also introduced new risks, including AI-generated responses that can look indistinguishable from human ones. Survey panels have been shown to carry significant risks of fraud and poor-quality responses. According to Kantar, researchers discard an average of 38% of survey data because of quality concerns, which can include fraudulent or automated responses.
As these tools improve, text alone can no longer reliably separate synthetic answers from human ones. The answers read well, they are coherent, and they look real. But some aren’t. And unless your AI agents are designed to check, they may simply process the inputs they receive.
When Data Quality Problems Become Decision Problems
AI contamination is generally addressed as a data quality issue. And it is. But it’s also a decision problem because the data being captured and acted upon is flowing into AI-driven conclusions that are updating your risk models, shaping your renewal strategies and determining the next steps your agents will take.
When, based on an AI-generated response, your AI agent determines that a customer is healthy, that interpretation may be carried forward. It can feed inaccurate signals into your revenue forecasts. It can deprioritize that account for action. And where agents can act autonomously, they may act without any evidence that a human was ever involved.
Confident isn’t the same as correct. That’s as true of the data going in as it is of the recommendations coming out.
Participant Verification
Participant verification, or the ability to confirm that a person responding is who they claim to be, is a critical first step in your process. But it’s not enough. Verified identity doesn’t verify the response. It doesn’t tell you whether a person was actually engaged, whether they answered candidly or whether they employed an AI agent to act on their behalf.
There are ways, though, to evaluate these interactions by focusing on:
- Expression rather than language. In voice and video sessions, how someone speaks can still carry useful signals. Human responses may include hesitations, variations in energy and shifts in tone. However, modern real-time voice systems can simulate breaths, micro-pauses and even hesitation. These cues may provide reasons to investigate further, but they cannot establish authenticity on their own.
- Provenance rather than content. Whenever you rely on a specific finding to make a decision, it’s important that you’re able to trace it back to a specific moment to determine who said it, when, in what context and with what expressed signal. If you can’t do that, you don’t have verifiable evidence. You just have an assertion. A well-governed AI system can help you answer those questions.
- Contradiction testing rather than confirmation. Human feedback can be contradictory. People are inconsistent. They say one thing and do another. They agree in principle but respond differently in practice. If your customer intelligence is unusually consistent, that may be a sign you need to pay closer attention, though consistency alone does not establish whether a response is synthetic.
- Methodology, not just technology. Your research design matters. How you frame questions, whether follow-up probes can adapt based on input and whether interviewers adjust based on what respondents actually say all make a difference. Synthetic responses can imitate these patterns, though producing them under live, unscripted conditions may be more challenging.
The Governance Question Every Executive Should Be Asking
Effective governance requires traceability. Can you trace your significant AI-driven customer decisions back to a verified human interaction, a specific customer and a specific conversation? If you can’t, you have an AI governance issue.
The executives who get ahead of this are asking a different set of questions than their peers. Not just: What did the AI conclude? But: Who actually said that? When? How did they express it? What was the intensity of the signal? And is this enough to reliably act on?
These are questions worth building decisions on. Yet many AI systems may struggle to answer them because they were never designed to provide that level of traceability.
