Designing a Survey Pt. 2 - Solid Structure & Flow

Survey Structure, Evaluation Method, Common Survey Mistakes, Reflections on Practice

Alright, you have your survey questions and you’re ready to publish. Wait! There is one more step. What’s going on with that structure?

Great questions are important, but without a coherent flow, participants can still stumble, get lost, or quit. Spending a few moments to ensure that participants can easily make their way through to the finish line will help ensure your survey has a high completion rate and quality data. 

Here are 5 common mistakes we see people make when structuring their survey, along with quick fixes to ensure it is easy to complete. 

Mistake # 1: A Shady Start

We see lots of surveys just jump right in with the first question. Hold up! What is the purpose of the survey? Who will see the responses? What will happen with the answers? How long will it take to finish? Is there an incentive or gift? Who can they contact if they have questions? 

Potential respondents likely have many questions before getting started - especially about risks and confidentiality. Be considerate and manage people’s fears and expectations before you ask for anything. That way, they can decide if it is worth their time, energy, and potential risk to move forward. 

Begin with a consent page before asking for any information. This should clearly and succinctly answer the following questions: 

• What is this survey about?

• How will the data be used? Will any of it be made public? 

• Who will see it?

• What level of anonymity will participants have? (i.e., will their identities be completely unknown to everyone, or will some staff know who participated while keeping responses confidential from others?) 

• How long will it take to complete?

• Why should someone bother doing the survey? How might they benefit? 

  • If you do have a gift, how/when will that be coordinated? 

• Who can they contact if they have questions or concerns? 
  

Mistake # 2: Continuous Scroll 

Continuous scroll is for social media, not surveys. Seeing an endless list of questions can be intimidating and cause people to get overwhelmed, lose steam, and give up. Breaking up questions across multiple pages and adding a progress bar can help people navigate the survey more meaningfully, while also providing assurance that the end is near. 

Pro Tip: 

If people want or need to see all of the questions upfront, consider making a plain-text document that is hyperlinked or downloadable.

This is especially helpful if you are using the survey as a registration form (rather than a feedback survey).

Mistake # 3: No Flow 

It isn’t enough for the individual questions to make sense. Each question has to make sense in relation to the one that comes before and after. When topics jump around, people struggle to understand what is being asked of them and to focus their ideas. Group all questions related to a specific topic, experience, or time period together. Add a page break to signal that you’re moving on. 

Think of the survey structure like a dinner table conversation. Topics come up. People take turns sharing a comment or related story. Then, the conversation moves on to new territory. When someone chimes in too late, their joke or comment can fall flat. Others around the table may cringe and say, “So, anyway…” to get away from an awkward moment. To avoid this vibe in your survey, keep topics organized and flowing in a clear direction.

Reality Check:

Often, a lack of flow stems from an unclear survey purpose. Too many competing interests or priorities across partners can cause problems. If you have too many distinct topics or do not have a good balance of questions between topics, it may be a good moment to pause. Return to the purpose of your survey. Prioritize the minimum viable questions that you need to achieve this goal. Remove anything that detracts from it. This may require you to gently remind interested parties that the survey requires a clear focus and may not include their favourite question this time (but there may be other measurement opportunities). 

Mistake # 4: Forgetting About Branching & Display Logic

Display logic shows people only the questions that matter to them. For example, if someone answers "I don't have children" early in your survey, they won't see the 10 questions about your childcare program. This keeps surveys shorter and more relevant for each person. 

Building familiarity and confidence using these functions improves the flow of a survey. When participants only see questions that apply to them, they don't have to work as hard to provide answers. No need to recall previous answers to respond to dependent questions (e.g., if you answered "Yes" above...), scroll past irrelevant sections, or decide if certain questions are meant for them.

Best of all, a personalized survey reminds people that you are listening to them and took the extra steps to give them a customized experience. When you do the work, people will be more likely to reciprocate with thoughtful answers. 

Pro Tip:

Before committing to a survey software for your organization, spend time playing with branching logic. Unfortunately, many free or low-cost platforms offer few options, limiting your ability to structure the survey effectively. This means relying on clunky questions, like “If you answered Yes above,” or “If you answered No, skip to Question 7.”

If you already paid for software that doesn’t have these features, don’t panic! Focus on questions that work for anyone and do not require qualifiers or follow-up. You could also consider creating multiple versions of a survey for key populations (e.g., Version A for people who completed your program and Version B for people who did not). 

Branching is important, but it shouldn't be stressful or unattainable. That's why, as a values-driven company, we invest in our own software subscription. If our clients do not have access to software that supports the kind of survey they need, we can design an effective survey using ours at no additional cost to them. That is how strongly we believe in the importance of good data collection.

Mistake # 5: Lack of Question Diversity & No Open-Ended Boxes

Great surveys mix question types throughout. This balances the workload, maintains engagement, and encourages thoughtful answers. Getting the balance right is key. Too many scales, drop-downs, checkboxes, or matrices can cause people to stop paying attention and start clicking things absent-mindedly. But, too many open-text boxes signal to people that you couldn’t be bothered to have a chat with them. So, they’re not going to write detailed and thoughtful answers; they’re going to skip to the end. To get the balance right, consider: 

• Minimizing how many of the same type of question you ask so each one feels unique and offers a novel way to engage 

• (If possible) spreading out similar question structures across different pages so people have a break between repetitive tasks

• Include only the minimum viable open-text boxes to cut down on the work required to participate

• Ensure you have one open-text box at the end of the survey for any additional comments or feedback a person hasn’t yet mentioned. In our humble opinion, a survey isn’t valid or useful unless you have at least one place for explanation or insights that don’t fit into a scale or tick-box. 

  
  

---

The Bottom Line

Survey structure is like good event planning - when it's done right, people don't notice it. They move smoothly from start to finish and leave feeling like their time was well spent. 

Remember

Every survey is asking people to donate their time and insights. Your participants' expertise, experiences, and knowledge are valuable. At the very least, you can make sharing as smooth as possible. 

Before you hit publish, go through your survey as if you're seeing it for the first time. Does it feel welcoming? Does each section flow logically? Can you easily find your way through? If something feels off to you, it will definitely feel off to your participants. Trust that instinct. 

The structural issues we see most often aren't technical problems - they're clarity problems. May not know why they are asking these questions or haven't considered how it feels to answer. 

Not sure if you are ready to launch your survey? Reach out if you'd like another perspective. We offer free 30-min consultations with our senior research and evaluation consultants to address concerns. As nerdy data people, we love helping folks get survey-ready. To book a free consultation, email us at admin@andimplementation.ca

---

Subscribe to our blog to have new content delivered right to you!

#collage #artsbasedmethods #artsbased #creative #qualitative #qualitativemethods #evaluation #skillbuilding #professionaldevelopment #trysomethingnew #evaluation #programevaluation #research #methodology #learning #creativepractice #creativeconsulting #tipsandtricks #tipsforpractice #methods