Today, I want to share with you some tips on how to handle a common situation in user research: when a participant does not seem to meet key criteria for your study.
As a UX researcher, I have encountered this situation many times. It can be challenging to deal with a participant who does not fit your recruiting profile or has some other feature that makes proceeding with the session a problem. For example, you may have recruited a participant who claims to be an expert in using a certain software, but during the session, you discover that he is actually a novice. Or you may have recruited a participant who is supposed to use your product in a specific context, but during the session, you find out that she does not use the product in that context.
So what can you do in such situations? Here are some suggestions based on the book The Moderator’s Survival Guide by Donna Tedesco and Fiona Tranquada:
What to do or say in that situation:
- First of all, do not panic or get angry. Remember that the participant is not trying to deceive you or spoil your session. He may have misunderstood the recruiting criteria, or he may have overestimated his own skills or experience. He may have agreed to participate without knowing the full details of the study, or he may have encountered some unforeseen circumstances that prevented him from using your product in the intended context.
- Try to assess the situation and decide whether you can still proceed with the session or not. Depending on your study goals and methods, you may be able to adapt your session plan and still get some useful feedback from the participant. For example, if you are conducting a usability study and the participant is less skilled than expected, you may be able to simplify the tasks or provide more guidance and support. If you are conducting a contextual inquiry and the participant does not have access to the context, you may be able to ask her to describe or simulate how she would use your product in that context.
- If you decide to continue with the session, explain to the participant why you are making some changes to the session plan and reassure him that his feedback is still valuable. For example, you can say something like: “Thank you for participating in this study. We are interested in learning how people like you use this software. I noticed that some of the tasks are more difficult than expected for you. That’s okay, we want to see how people of different skill levels use this software. I’m going to adjust some of the tasks and give you more help along the way. Your feedback will help us improve this software for future users.”
- If you decide to end the session early, thank the participant for her time and explain why you are unable to continue. For example, you can say something like: “Thank you for participating in this study. We appreciate your willingness to share your experience with us. Unfortunately, we are unable to continue with this session because we need participants who use this product in a specific context. This is not your fault, it’s just a mismatch between our recruiting criteria and your situation. We apologise for any inconvenience this may cause you. We will still compensate you for your time and effort.”
What not to do or say in that situation:
- Do not blame or criticize the participant for not meeting the criteria. This will only make him feel bad and damage your relationship with him. Remember that he is doing you a favor by participating in your study and sharing his feedback with you.
- Do not ignore or dismiss the situation and pretend that everything is fine. This will only lead to biased or invalid data and waste your time and resources. Remember that your goal is to collect reliable and relevant feedback that will help you improve your product or service.
- Do not lie or deceive the participant about the purpose or goals of your study. This will only undermine your credibility and ethics as a researcher and potentially harm your organization’s reputation. Remember that you have a responsibility to respect and protect the participant’s rights and privacy.
How to avoid that situation:
- The best way to avoid this situation is to recruit participants who meet your criteria as closely as possible. This means that you need to define your criteria clearly and communicate them effectively to your recruiters, participants, and stakeholders.
- Another way to avoid this situation is to prepare for it in advance and have a contingency plan in case it happens. This means that you need to anticipate potential mismatches between your criteria and participants’ characteristics or situations, and think of ways to adapt your session plan accordingly. You also need to inform your stakeholders about the possibility of this situation occurring and how it may affect your data collection and analysis.
I hope these tips will help you handle this common situation in user research with confidence. Thank you for reading and happy moderating!
