Interviewing for data collection is deceptively simple. Ask a question. Get an answer. But simple is complicated. There is nowhere to hide in simple. That’s why interviews are easy to get wrong.
I have bumbled through my fair share of interview blunders. And from working with peers in the evaluation field, I continue to see persistent problems in interviewing. There’s no judgment here. Making and revisiting mistakes is a great way to learn.
In the spirit of learning, here are my top six interview setbacks to watch for:
#1 - Treating the interview like a talking survey
Surveys and interviews are different tools that work differently and require different strategies to engage folks in meaningful ways. An interview is not an ideal way to ask people to rank or recount the frequencies of their experiences on a scale. This is a methodological meeting that could have been an email.
Interviews shine when exploring the non-quantifiable human factor - like experiences, perceptions, beliefs, values, aspirations, and memories.
#2 - Assigning just anyone to be the interviewer
The “tool” isn’t the interview guide; it is the interviewer. Not just anyone can do the job in every context. The interviewer must earn and sustain trust by offering context-specific techniques - which could include grounded expertise. Not sure what’s needed? Ask those being interviewed if they have particular preferences and are okay with whomever you have in mind. Why? Consent!
To avoid treating interviews as “grunt work,” it is important for evaluators to invest in building and refining this skill. This means ensuring that early-career, student, and inexperienced evaluators are adequately supervised and paid to do this work because it is valuable.
#3 - Not anchoring the evaluation question in the interview
Often, I see interviewers either:
Not ask questions that help answer the global evaluation question
Directly ask interviewees their evaluation question
The interview isn’t a free-for-all to ask anything or try to short-cut evaluation work. Instead, an interview should address multiple parts of the global question. Then, evaluators make sense of responses through suitable data analysis techniques (which can be collaborative).
#4 - Asking questions badly
The easiest way to make a question unanswerable is to make it confusing, irrelevant, and icky. Practice and refine questions ahead of time to avoid:
Answering your own question Ask the question and stop talking. If you answer your question, you shift the focus from them to you. (e.g., When was a time you felt most at home? You know, like for me, I feel the most at home when I’m with my family and can relax. Is it like that for you, or something else?)
Explaining instead of asking a question Questions should be simple. Over-explaining questions before the interviewee asks you to rephrase can introduce your interpretation and limit how much you can understand about their point of view. (e.g., When was a time you felt most at home? What I mean by that is if home feels comfortable and safe, when does that happen for you?)
Cluttering questions with more than one topic or component Ask one question with one focal point at a time. The more you give someone to think about, the more likely it will be that they will get confused and lose their train of thought. (e.g., When was a time you felt most at home and what does “home” mean to you?)
Asking a pointed or leading question Imposing judgment or requiring someone to agree or disagree with you deteriorates trust and creates low-quality data. (e.g., You come from a good home, right?)
#5 - Ignoring the flow and vibe
There is an art to crafting an interview structure that flows. It takes practice to understand when to linger or move on. Much of this work happens when paying attention to and using body language, tone, expression, and silence (read more about that here).
The interviewer’s craft is how they create and hold space (and for what, when, and why).
#6 - Assuming that questions are entitled to answers
Questions can be intrusive, irrelevant, and inappropriate. An interviewee tells the interviewer what is and is not okay to talk about with patterns in answering questions, body language, tone, and direct requests to move on. But, interviewers don’t always listen to how consent is being exercised in the interview.
Just because an interviewer asks doesn’t mean they have a right to know.
Of course, there are many other ways to mess up an interview. At AND, Jennica and I firmly believe that taking the scenic route can bring valuable learning. You can only evolve a skill by practicing, getting it wrong, and doing better.
If you want to see a part two with even more interview inadequacies, email us an example, and we will include it in a follow-up post. To learn more, check out our other blog posts about interviewing and professional development offerings.
Subscribe to our blog to have new content delivered right to you!