Tips for Online Surveys Tips for Online Surveys 65 L Control the quality The danger in quality control checks that are too strict is that they exclude not only poor performing participants but many good performers as well. This not only impacts ESS THA feasibility but could result in a sample of people who are unusually attentive and not representative of the normal range of human attentiveness. Source: SSI The problem with so many attention-checking questions is that they do not measure inattention at all! In addition, even a single attention slip (a normal human error) might not be evidence of a general lack of attention. THE RECOMMENDATION IS TO USE FIVE OR MORE OF THESE CHECKS: SUGGESTION: - MULTIPLE LOW-INCIDENCE ITEMS Use at least five simple quality-control checks and - OPEN-ENDED ANSWER QUALITY (IF AVAILABLE AND TIME ALLOWS) only remove people if they fail more than one. - CONFLICTING ANSWERS IN A SHORT GRID - SPEEDER CHECK SSI tested 15 commonly used quality control checks and - DIRECT INSTRUCTION (E.G., CHOOSE ANSWER 5) IN A SHORT recommends only removing people if they fail more than GRID one. Some checks fail no one, others fail over 60%, so - FACT CHECK (VALIDATE A FACT ASKED IN THE BEGINNING OF they cannot be measuring the same thing. THE QUESTIONNAIRE VS. THE END) See the article AND REMOVE ONLY THOSE PEOPLE WHO FAIL TWO OR MORE. Source: SSI Source: SSI © 2017 GRBN Participant Engagement Handbook | 156 © 2017 GRBN Participant Engagement Handbook | 157