This article was published on July 25, 2018

This UX researcher reveals how to spot a liar

A UX researcher reveals how to spot the people who can wreck the accuracy of any study


This UX researcher reveals how to spot a liar

I pay people to talk to me. As a researcher, I typically study a particular product or a particular industry, so I need to find people who have bought that product or who shop in that industry. Then, that person has to agree to take time out of their busy day to have a video call with me or meet me in person at a research facility. The details vary from project to project, but to get anyone to talk, I have to compensate them for their time, cover their transportation costs, and try my best to make the whole interview process a pleasant experience for them.

Most of the time, finding subjects is seamless. I can do the recruiting myself on a research platform or outsource the search to a recruiting company specializing in locating the exact type of people that I need to talk to. To make sure I have the right people in my study, I write a screener, a survey for potential participants designed to identify exactly which people qualify for my study and weed out those who don’t. Because we pay our participants, we run the risk of people trying to join the study, even though they might not be the exact type of person we’re looking for.

Oftentimes, the wrong people try to get on a study because they honestly think they fit the profile; other times, they’re what we call “career participants,” people who have discovered the market research industry and attempt to join as many studies as possible to collect the money. Regardless of their motivations, these aren’t bad people; they’re just not qualified to talk about the topics that we need to research, and as a result, they have to lie.

Illustration by Sara Osorio

When we encounter liars in research, most people aren’t intentionally lying but rather thinking optimistically. Let’s say we’re looking to talk to people who have visited a particular store, like a Whole Foods, within the last 30 days. Maybe the store was renovated, and the brand wants to see what customers think of it.

The people volunteering to participate in our study might have visited a Whole Foods a year ago, and might shop at other groceries more frequently, so they think that they can help us with our research, but ultimately, they can’t. We ask them questions about their recent trip to Whole Foods and what they thought about the renovation, and because they didn’t actually see it, that’s when they begin to lie.

The 💜 of EU tech

The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!

Here are the signs of a liar:

They can’t “exactly” remember

When you ask them when they visited the store, they’ll find that they suddenly can’t remember. They can’t pinpoint when they were there, but it was definitely recently. Maybe it was in the last three to six months, but it could have been a year ago. They’ve been so busy recently that they just can’t seem to remember.

Sometimes people are busy, and sometimes it’s hard to remember details, but honest participants remember everything. They can tell you what month it was, if it was a weekend or weekday, who they were with, and what they planned to buy at the store. Liars can’t talk about the timeline, because they don’t have one. They try not to go too specific and will occasionally blend pieces of information from similar grocery store trips hoping that this will provide the answers that the researcher is looking for.

They love everything

When you ask about their experience with the store, they loved it. They loved everything about it and can’t think of anything negative to say at all. People who have actually visited a store, or actually paid for something, can always find fault with it. The product they bought doesn’t work as well as they hoped, or maybe it was more expensive than they were expecting.

Honest participants can almost always tell you about their disappointments. They’ll tell you how hard it was to find parking, or how they hated the store’s new renovation because they couldn’t find what was on their shopping list. Liars can’t give specific details, so instead, they default to saying that everything was great.

They will try to talk about other things

When you ask about the topic or the store that you’re researching, the participant starts to talk about different topics or different stores. When you ask questions about Whole Foods, you’ll start getting opinions about Trader Joe’s. It’s not that the participant wants to lie to you; it’s that they can’t honestly answer the questions you’re asking, and so they try to give answers about other stores that they have visited to hopefully provide the answers they think you want.

Honest participants can talk about a single store or a single product for hours — I’ve listened to them do it. They talk about the minutiae, how helpful or unhelpful a particular sales associate was at a specific location, or how delicious the store-brand barbecue sauce is compared to that of competitors. It takes a lot of imagination to make up answers to questions that they don’t know, so liars talk about the things that they do know.

Just like a promising date that turns out to be anything but, a dishonest participant can seem attractive on paper and in online-dating pictures. Looking for the three signs outlined above, however, can help you disqualify a participant within the first ten minutes of an interview — shorter than it takes to sip a cocktail. Once you figure out that your participant is lying, you’ll then have to decide how long to continue talking to them before tactfully exiting the conversation.

The best response is usually to ask a few follow-up questions about the brand or find things that the participant is qualified to talk about, then end the session within 20 minutes. Be polite about it, smile, and deliver the researcher’s equivalent of “I have to go home to feed my cat”: “That’s all the time we have on the schedule today. Thank you so much for your participation.”

This story is republished from Magenta, a publication of Huge. Follow Huge down here:

Get the TNW newsletter

Get the most important tech news in your inbox each week.

Also tagged with