The design of an online survey may feel like a daunting task that will require books, classes, or a consultant. Fear not!
Try PsychData today – sign up for your free trial!
Whether you’re building your very first survey or your overseeing a complex longitudinal, grant-funded, randomized clinical trial, there are a core set of basic principles that will lead you to success:
- Preparation: before creating a single question in PsychData, every question should be researched, drafted, reviewed, refined, and triple-checked in an easy-to-edit document. After hosting tens of thousands of surveys and millions of participants, one thing is very clear: if you don’t ask the right question, with all of the right answers, in the right way – you won’t get the data you need. In particular, make sure that you select the right question type for the data you need.
- Parsimony: In the late 1990’s, response rates for online surveys were amazingly high because everyone was super excited to receive an email and doubly-honored to be invited to participate in an online survey (for free!). Today, online surveys are just as useful as ever – but they are also ubiquitous – so you should assume that your participant’s time is as valuable as the success of your project. For this reason, we recommend asking the following question about every item in your survey:
- “Am I absolutely certain that I will use the data from this question?” If the answer is not a firm yes, delete it. If you find yourself saying, “I might not use it…” or, “It would be really interesting to ask…” – delete it. Ask only what you need. Your goal is to get the data you need with the least amount of participant time.
- Organization: Sloppy surveys produce sloppy results. Once you’ve prepared all of your questions, using the right question types, and loaded them into PsychData, take time to carefully review every aspect of your survey including:
- Order and flow of all questions.
- Layout of instructions, spacing, and font treatments (e.g., bold) to highlight key portions.
- Page breaks – generally speaking, shorter pages are better. Separate each portion of your survey using page breaks (e.g., by questionnaire or section).
- Break matrix question groups into chunks of 10-15 questions. Your column header row (e.g., your answers) should be visible for each question in the matrix. If you can’t see the top of the table, then it makes sense to break the matrix into more than one group of questions.
- Etcetera. This isn’t rocket-science, but it does take time. Invite your friends and family to preview your survey and offer suggestions.
- Preview as if you were a participant: Before gathering real data, use the “Preview” function in PsychData to view your survey as if you were a participant. This feature won’t use up any participants, questions, or store data. Read every word, follow every instruction, and refine. Walk a mile in your participant’s shoes and make sure they are comfortable.
- Test your data: After previewing your survey as above, and determining that it is ready to go, turn it on and participate several more times (invite friends and family to help you out) by answering every question. Then, download the resulting data and examine it in the statistical package you will use. Make sure that the data produced by your survey matches your analytical needs. You would be surprised how many researchers never take this step and learn, after finishing data collection, that they used the wrong question type throughout and either can’t use their data OR face many hours of painstaking work.
- Know your participants: Over the years, we’ve seen many researchers struggle to meet their sample-size goals because they didn’t put enough effort into a recruitment strategy. Here are some critical points to consider:
- Emotional connection. Participants begin, persist, and complete surveys on topics they care about. As you develop your materials and determine who to invite – make sure to help participants develop and maintain an emotional connection to your project.
- Find a leader to help. If you can find a person who is well-known among your target sample, ask them to distribute your survey.
- Communication. Some participants will only respond to a snail-mail invitation, others via Facebook, others via txt or a personal invitation. Figure this out in advance and adapt your recruitment strategy to fit.
- Incentives. Surveys are ubiquitous and your participants are busy. Find something that is interesting and attractive to them to offer as an incentive. Gift-cards, entertainment, raffles, etc. We have repeatedly seen a struggling survey turn around when the right incentive is added.
- Understand online survey response rates. Responses to online survey invitations come in within 1-3 days. Anything after that will only be a dribble. Don’t send out one email and wait 2-3 weeks assuming that eventually everyone will respond. Deadlines can make a huge difference – especially if your participants are busy.
Building a high-quality online survey that delivers the results you need is a very doable project. Invest your time in pre-survey preparation, strategy development, and careful refinement of your survey and questions before launching.
If you’ d like some professional help, just ask!
PsychData's Blog 