CX Methods
Four research shortcuts that produce junk science
Cheap, fast research looks like insight but misleads every decision it feeds. The invoice arrives later, as a bad quarter.
Four recurring anti-patterns produce junk science in CX: untrained staff doing research, shipping speed over quality, workshops instead of observation, and bargain-basement methods. The data looks legitimate and misleads every decision it touches. The cost never shows up in the research budget; it shows up downstream, in products built on bad evidence.
Every failed CX program has a moment where someone chose the cheap shortcut. Four show up again and again. The Anyone Can Cook fallacy: assuming customer research needs no professional training, so anyone with a survey tool can do it. The Feature Factory: shipping fast and never checking whether the experience got better. Solution-by-Workshop: a room of employees guessing what customers do instead of watching them do it. And Microwave Pizza Research: the fastest, cheapest method, usually an unmoderated survey, standing in for real insight. Each fills the slot. Each produces data that looks legitimate and is not.
Why it matters to the business
Junk research does not sit harmlessly in a drawer. It feeds roadmaps, pricing calls, and redesigns. The delusion it creates is measurable: Bain's "Closing the Delivery Gap" study found 80% of companies believed they delivered a superior experience while only 8% of their customers agreed. That gap is what unchallenged internal guessing produces at scale. Self-report alone is shaky ground too. A study by the analytics firm 84.51 compared what survey respondents claimed with their actual loyalty-card data and found 75% misstated their own purchase behavior.
Rigor is not the expensive option. Jakob Nielsen's usability research shows that about five users uncover roughly 85% of usability problems in a qualitative test. A week of real observation costs less than a quarter spent building the wrong thing.
How to use it
- Inventory the evidence behind your last three major product decisions: observation, survey, or workshop vote?
- Put a trained researcher on every decision above a set spend threshold, the way you put a lawyer on every contract.
- Run five-user usability tests before and after each significant release, not one big study a year.
- Validate what customers say against what they do, using behavioral data, support logs, and analytics, before funding anything.
- Use workshops to align on findings, never to generate them.
Where teams get it wrong
Leaders treat democratized research as free capacity: everyone can run a survey, so research costs nothing. But the cost was never in the research line. It compounds downstream as features nobody uses, redesigns that make tasks harder, and quarters spent building for a customer who does not exist.
Ask your team
- Which of our current priorities is backed by someone watching a real customer, and which by a workshop or a survey?
- Who on this team is actually trained to run research, and are they in the room when we decide what to build?
- If we deleted every unmoderated survey from last quarter, which decisions would we have to revisit?
Junk research is expensive. The invoice just arrives later, disguised as a bad decision.
Apply this
Reading about four research shortcuts that produce junk science is one thing. Seeing where it applies in your journey is the useful part.