101 Tips to Improve the Research Participant User Experience

Table of Contents ENGAGE Introduction 10 Golden Tips 101 tips Online Communities Tips to improve the research participant Online Surveys Tips user experience Passive Metering Tips Premier Sponsor Qualitative Research Tips Gold Sponsor Afterwords © 2017 GRBN Participant Engagement Handbook | 1

Hear From the Experts Hear From the Experts Scott Miller, David Rothstein, Neil Marcus, Scott Miller, David Rothstein, Neil Marcus, Vision Critical RTi Research MetLife Vision Critical RTi MetLife Anna Hebbeln, Oriol Llarudó, Anna Hebbeln, Oriol LLarudó Wakoopa Netquest Wakoopa Netquest Kerry Hecht, Katrina Noelle, Nikki Lavoie, Kerry Hecht, Katrina Noelle, Nikki Lavoie, cho KNow MindSpark E Echo KNow MindSpark © 2017 GRBN Participant Engagement Handbook | 2 © 2017 GRBN Participant Engagement Handbook | 2

Customer relationships are marketing's problem. Right? Wrong. In a world where people can spend as much time completing a survey online as they do on a website, research has become part of the digital customer experience. Yet, we all know that Introduction the market research industry has failed to engineer our intelligence gathering activities into the personalized, valuable experiences that actually improve customer relationships. We must change our approach. Think about the crazy and disruptive reality of our industry in ƈWe need to treat recent years. Plummeting response rates. Misleading polls. participants with the Bots filling out surveys. An industry begging for public trust. same respect we Many challenges facing the industry today stem from our reliance on unknown samples of customers who see no treat our friends.Ɖ meaningful value in engaging with us. Imagine if you could deliver as much value to your research participants—your customers and potential customers—as you do to your stakeholders. Imagine if it enhanced the experience of the customer. Imagine if intelligence gathering was foundational to building long-term customer relationships. © 2017 GRBN Participant Engagement Handbook | 4 © 2017 GRBN Participant Engagement Handbook | 5

Become a leader of change Seeking to understand customers solely to sell them more is not customer centricity. In fact, it is the opposite. It’s exploitative and damaging to customer relationships. But, delivering a personalized digital survey experience based on customer preferences and the data you already have—that boosts engagement. Sharing what you’ve learned and empowering your customers with the insight they help uncover—that makes the interactions meaningful and valuable. Proving that the value of those interactions is worth as much to your customers as it is to your company—that builds trust. That is what builds relationships. That is the approach leaders in our industry must embrace if we are to strengthen our role in the intelligence ecosystem. Learn more at www.visioncritical.com/relationships Scott Miller, CEO © 2017 GRBN Participant Engagement Handbook | 6 © 2017 GRBN Participant Engagement Handbook | 7

We believe improving the research participant user About the Handbook experience is crucial for the future success of our sector, and think this Handbook can be a valuable tool to achieve this goal. This Handbook presents 101 expert tips for improving the This Handbook is not intended to be a definitive industry research participant user experience. guideline* and the advice is not intended to be exhaustive. The Handbook is the result of 12 months of hard labor by The aim of the Handbook is to inspire and enable researchers GRBN and our partners. We cannot express enough our to take concrete action today and tomorrow to improve gratitude to our partners—without their support and hard participant engagement and the user experience. work, this Handbook would not exist. The tips contained in this Handbook are drawn from On the following pages, you can read the story of why and research-on-research, as well as from the collective how this Handbook was created and read about the next experience of numerous experts in the field who work for steps. companies that interact with participants daily. These experts The primary objective of the Handbook is simple: see what works and what doesn’t both in terms of driving a positive user experience, as well as creating a positive impression on the client commissioning the research. We recommend that you start out by reading the 10 Golden Tips, which highlight actions that can be taken to improve the TO PROVIDE CLIENT- AND AGENCY- user experience, irrespective of the methodology. SIDE RESEARCHERS PRACTICAL ADVICE ON HOW TO IMPROVE THE EXPERIENCE PEOPLE GET WHEN PARTICIPATING IN RESEARCH. * GRBN in partnership with ESOMAR issues joint guidelines for the research sector. You can access the latest joint guidelines from here. © 2017 GRBN Participant Engagement Handbook | 8 © 2017 GRBN Participant Engagement Handbook | 9

Once you have read the 10 Golden Tips, we encourage you to spend a few minutes reflecting on the projects you have Read me: I am important recently undertaken, evaluating them against the advice contained in the 10 Golden Tips. Where are you doing well? How can you improve? Copyright After that, feel free to browse the methodology-specific tips © 2017 GRBN put together by our experts or use the search function to find The copyright on this Handbook is owned by Global Research things you are specifically interested in. Business Network, GRBN. Parts or the whole of this We hope this Handbook provides you with food for thought Handbook may only be reproduced, distributed, or and enables you to take concrete action to improve the transmitted if GRBN is clearly acknowledged as the copyright experience you give the people who participate in your owner. In addition, when reproducing, distributing, or upcoming projects. transmitting tips where the source or sources are We would love to hear your feedback on the tips and your acknowledged in this Handbook, those sources must be testimonials on how you’ve incorporated them in your daily acknowledged in the reproduced, distributed, or transmitted work. Please get in touch and remember: material. Disclaimer The information provided within this Handbook is for general informational purposes only. There are no representations or NOBODY DESERVES A POOR warranties, expressed or implied, about the completeness, accuracy, reliability, or suitability with respect to the RESEARCH USER EXPERIENCE, information, products, services, or related graphics NOT EVEN YOUR MOTHER-IN-LAW. contained in this Handbook for any purpose. Any use of this information is at your own risk. Neither GRBN nor the contributors to this handbook assume and accept any liability to any party for any loss, damage, or disruption caused by applying the information shared in this Handbook. © 2017 GRBN Participant Engagement Handbook | 10 © 2017 GRBN Participant Engagement Handbook | 11

In the spring of 2016, we conducted research to explore three The birth of the Handbook drivers of trust: 1. Trust with the protection of data Since 2015, GRBN has been presenting data on the general a. Fundamental to the willingness to share data public’s lack of trust in the market research industry and with market researchers encouraging the industry to collectively do something about 2. The perceived value of market research it so we can maintain our ability to collect the data we need a. Fundamental to a willingness to say Ȋyesȋ to an interview request to meet clients’ needs. 3. The user experience a. Fundamental to participating again With respect to the user experience, our research showed ONLY 1-IN-10 PEOPLE HAVE A that participants found most surveys were too long, HIGH LEVEL OF TRUST IN unenjoyable, and lacked mobile-friendliness. MARKET RESEARCH COMPANIES. One key conclusion from the research was that: Source: GRBN Trust Survey 2016 THERE IS ABSOLUTELY NO POINT Early on our message largely fell on deaf ears and I vividly PROMOTING THE VALUE OF RESEARCH recall presenting to an almost empty room at IIeX AND ENCOURAGING PEOPLE TO Amsterdam. The lack of interest in the issue was demoralizing, to say the least. Rather than giving up, we PARTICIPATE IF WE GIVE THEM A BAD pushed forward and are Ȋfighting the good fightȋ as Roddy EXPERIENCE WHEN THEY DO. Knowles at Research Now would say. © 2017 GRBN Participant Engagement Handbook | 12 © 2017 GRBN Participant Engagement Handbook | 13

Not only are people likely never to participate again, but they Fortunately, the research came up with one also tell others about the bad experience. This makes it even unexpected nugget of insight, which was pivotal harder for the industry to recruit people to participate. in getting the industry to act. Therefore, we set out with our partners to conduct research In the US, the research told us that thinking badly about the that specifically looked at the research participant user companies or brands mentioned in the survey was the 1 experience . What we found was both disconcerting and number one thing survey-takers did when getting a bad encouraging. survey experience. What was encouraging was that the research gave us a very So the lightbulb went on: Survey-takers are savvy. They know clear understanding of what motivates people to participate there is an end-client behind most projects. They are in surveys and what causes a very good or very poor participating in research to try and help those clients. They experience. feel betrayed when in return they are given a bad experience. When we discussed these findings with industry leaders, One could argue that this doesn’t matter, especially since the more than one nodded and said with a sigh: general practice today tends to hide the end-client if possible, but we would argue otherwise. Firstly, even if you hide the actual client, this does not prevent the survey-takers from feeling negatively about the brands in the survey and ȃWe know, but these ȁbadȂ surveys are therefore the category. This does not help anyone trying to what the clients demand from us.Ȅ market products in that category, so in this case, research is creating negative brand equity and making marketers’ jobs harder than they need to be. 1 GRBN Online UX Survey 2016 © 2017 GRBN Participant Engagement Handbook | 14 © 2017 GRBN Participant Engagement Handbook | 15

Secondly, what if we are transparent about who the survey is can be convinced that creating great surveys is good for for and give participants a great experience? Will survey- brand equity, then the demand for Ȋbadȋ surveys will takers, in that case, think positively about the brand and, decrease, enabling the industry to deliver better insights back therefore, will research become a positive contributor to to their clients more efficiently…a virtuous circle. 2 long-term brand equity ? If that is the case: Fortunately, enough of the leaders we talked to had the courage and the foresight to buy into this vision and the GRBN Participant Engagement Initiative was born. WHY WOULD CLIENT-SIDE They also had the generosity to sponsor the initiative, which RESEARCHERS WANT TO CREATE enabled us to spend the time necessary to drive the project EXPERIENCES WHICH DAMAGED THEIR forward. Since that day in Utah, more and more companies CATEGORY OR WHY WOULD THEY NOT have come on-board the initiative, either contributing money, WANT TO CREATE EXPERIENCES WHICH their time, or other resources to the project. By the time the Handbook was launched, one year later, more than 30 CREATE POSITIVE BRAND EQUITY? companies are involved. Over the last 12 months, the partners have participated in Marketers across the globe are very much focused on workgroups, conducted research-on-research, collected delivering great customer experiences across all touchpoints. metrics data, and contributed to the Handbook. This Handbook is the fruit of all those efforts and we are From a customer’s perspective, a survey, or any other type of research, is just another touchpoint with the brand. From the extremely grateful for the support and active contributions of brand’s perspective, it is either an opportunity gained or lost all our partners. to create value for customers and build brand equity. Two years on from the start of the journey, I led a panel Armed with our data and these thoughts, we invited a group session on Participant Engagement at Insight Association’s of industry leaders attending the CASRO Annual Conference Next Conference to a standing-room only audience. I could in Park City, Utah, in October 2016, to a breakfast meeting. sense a clear shift in attitudes towards the issue and believe The intention was to get these industry leaders to take the sector is now ready to seize the moment and take the collective action. They immediately understood that if clients collective and individual actions needed to improve the user experience and participant engagement. 2 Our latest research shows that this is indeed the case. See Tip 62 for details. © 2017 GRBN Participant Engagement Handbook | 16 © 2017 GRBN Participant Engagement Handbook | 17

It has been a great journey and one I personally am extremely Acknowledgements proud to have been a part of. But the journey doesn’t stop here, it has just begun. For now, the hard work is getting this Handbook into the hands of as many client- and agency-side researchers as possible. More importantly, getting them to We would like to thank our sponsors listed on the next page, think about and act on the great tips we are offering. as well as all our partners on the project. Without their You will no doubt hear a lot from us and from our partners support and contributions, the creation of this Handbook over the months and years to come. You can actively follow would not have been possible. our progress on www.grbnnews.com or sign-up to our We would also like to thank and acknowledge the following newsletter (Global Insights) to get updates direct to your individuals for their contributions to the Handbook: inbox.  Big Sofa, Alicia Nicholls  Research For Good, Baillie We would also like more companies to become partners on  Doing Good Research, Buchanan the initiative. We want companies to deploy metrics to Allison Rak  Research Now, Roddy measure the user experience. As you can read in Tip 24, this  Echo Qualitative Project Knowles is a great way to drive change both within your organization Support, Kerry Hecht  RTi Research, Dave Rothstein as well as in cooperation with clients and data collectors. Labsuirs  RTi Research, Marilyn Wolpin Welcome, on-board!  Insight and Measurement,  Schwan's Shared Services, David Harris Kelsy Saulsbury Read more about the initiative here.  Jessica Broome Research,  SSI, Jackie Lorch Jessica Broome  SSI, Pete Cape Andrew Cannon,  KNow Research, Katrina  University of Utah, Cait Executive Director Noelle Wilson  Mesh Experience, Catherine  Verto Analytics, Connie Rickwood Hwong  MetLife, Neil Marcus  Vision Critical, Christy  Mindspark Research Ransom International, Nikki Lavoie  Vision Critical, Scott Miller  MRS, Debrah Harding  Wakoopa, Anna Hebbeln  Netquest, Ezequiel Paura  Wakoopa, Bernou Benne  Netquest, Oriol Llauradó  Odin Text, Tom Anderson  P2Sample, Janna de Bruijne © 2017 GRBN Participant Engagement Handbook | 18 © 2017 GRBN Participant Engagement Handbook | 19

© 2017 GRBN Participant Engagement Handbook | 20 © 2017 GRBN Participant Engagement Handbook | 21

10 Golden Tips 10 Golden Tips Table of contents Thoughts from the client side on the importance of engaging participants ................................................. 24 10 Golden 1. Show respect ........................................................... 27 2. Treat market research as a brand touchpoint ...... 28 Tips 3. Employ a participant-centric design process ........ 29 4. Take care of the basics ........................................... 30 5. Be transparent ........................................................ 31 6. Keep it relevant ....................................................... 32 10 tips everyone should follow 7. Keep it short ............................................................ 33 8. Share back ............................................................... 34 9. Thank your participants .......................................... 35 10. Use metrics to get better ........................................ 36 © 2017 GRBN Participant Engagement Handbook | 23 © 2017 GRBN Participant Engagement Handbook | 23

10 Golden Tips 10 Golden Tips Thoughts from the touchpoint which reflects on our image/reputation. In these cases, we need to make sure the survey experience leaves participants with a positive impression of MetLife. client side on the DR: Obviously, there are a lot of great engagement tips importance of engaging included in the handbook and you got to see them first- participants hand through your participation in the research on research. Which do you feel particularly excited about using in your research and why? Neil Marcus, AVP and research leader at MetLife, was NM: I believe there is an opportunity for us to implement all of the recommendations. In general, using more plain speak, instrumental in the GRBN’s Participant Engagement conversational language is one best practice that especially Initiative. His support as a corporate partner for the research on which some of these best practices are based resonates with me. On identified surveys, I’m also particularly was invaluable. Dave Rothstein, CEO at RTi Research, a Gold excited about improving participant engagement by using Sponsor on the Participant Engagement Initiative, talked video (or other means) to provide our participants with more with Neil about the initiative and its impact. detail on the purpose of the study and how the findings will be used. DR: You and MetLife have been tremendous supporters of the Participant Engagement Initiative. Why is the initiative so important to you and your organization? NM: The initiative is important in so many ways and to so many constituents. First, from an industry perspective, we are concerned about sustainability, specifically in the face of declining survey response rates, and thus risks to sample integrity. Next, we are concerned about the quality of the responses we receive when surveys are lengthy or focused on topics that are less engaging and of interest to people (e.g., insurance). And finally, from a brand perspective, for identified surveys, we recognize that a survey is yet another © 2017 GRBN Participant Engagement Handbook | 24 © 2017 GRBN Participant Engagement Handbook | 25

10 Golden Tips 10 Golden Tips DR: What obstacles do you see in putting these tips into 1 practice for improved participant engagement? How might they be overcome? Show respect NM: I expect some of our business partners may be somewhat uncomfortable with departing from the traditional survey Without respect for participants, the rest of the tips within practices. I’m glad we have this Ȋresearch on researchȋ this handbook are useless. through our partnership with GRBN to help assuage those concerns and really prove the criticality of employing We have to show the people, who voluntarily give us their techniques to improve participant engagement. Overall, given opinions or access to their data, the respect they deserve. our focus on insight-driven strategies and optimizing the We need to remember at every stage in the process that we customer experience, I am confident most will embrace these are dealing with people, not data points. new approaches – especially given the relationship between In this digital world, it is all too easy for researchers to survey experience and brand perception on identified surveys. dehumanize the process and ignore the fact that there is a real person behind the 0’s and 1’s. We encourage you to treat participants with the same respect you would treat a friend. Respect is the base upon which we can build and deliver great experiences. Dave Rothstein Neil Marcus CEO Assistant Vice President © 2017 GRBN Participant Engagement Handbook | 26 © 2017 GRBN Participant Engagement Handbook | 27

10 Golden Tips 10 Golden Tips 2 3 Employ a participant- Treat market research as centric design process a brand touchpoint Market research has become a brand touchpoint and a You no doubt have a well-grooved, client-centric design research experience that is positively aligned with the process in place to ensure that both your proposals and your overall customer journey is required. Companies need to research projects meet client needs. provide a great customer experience that spans across all Create a parallel participant-centric process. At every stage touchpoints, including market research. of your process, think about the impact of your decisions on If the market research experience of a consumer is positive, the participant, evaluate what could be done differently to it will not just influence their opinion on their entire have a positive impact on clients and participants, and consumer journey, but also their perception of the brand. design accordingly. Think about participants as well as clients at every stage in your process © 2017 GRBN Participant Engagement Handbook | 28 © 2017 GRBN Participant Engagement Handbook | 29

10 Golden Tips 10 Golden Tips 4 5 Take care of the basics Be transparent A great research experience requires great fundamentals. Transparency is key to successful customer-centricity in No amount of fancy design will save you if the basics business, and equally, it is key to successful participant- are wrong. centricity in research. Don’t take it for granted that all data collectors are equal in Is there really any reason why you cannot tell participants at this respect. Make sure yours is getting the basics right. the end of the interaction who the research was conducted on behalf of? Is there any reason why you can’t share what value their participation has? Again, don’t take it for granted that all data collectors are PARTICIPANT ENGAGEMENT BASICS equal with respect to transparency. Make sure yours is being transparent with the participants.  PAY A FAIR INCENTIVE COMMENSURATE WITH EFFORT  BE HONEST AND TRANSPARENT ABOUT THE PROJECT, ESPECIALLY THE TIME COMMITMENT REQUIRED  KEEP SCREENERS AS SHORT AS POSSIBLE THINGS TO BE TRANSPARENT ABOUT:  WHAT IS EXPECTED FROM THE PARTICIPANTS  WHO THE RESEARCH IS FOR  WHAT WILL BE DONE WITH THEIR INPUT  HOW THEIR PERSONAL DATA WILL BE PROTECTED AND APPROPRIATELY USED © 2017 GRBN Participant Engagement Handbook | 30 © 2017 GRBN Participant Engagement Handbook | 31

10 Golden Tips 10 Golden Tips 6 7 Keep it relevant Keep it short Participants need to feel that their participation in research All things being equal, short is better than long. There are is meaningful. One way to do this is to deliberately include many great tips in this handbook to help you shorten the sections in the research that are relevant to the participants amount of time you are taking from people in order to reach themselves. Try to find themes that are relevant to both the your objectives. Respect the value of people’s time. participant and the client, and build interesting questions and other engaging activities into your research. Source: Vision Critical © 2017 GRBN Participant Engagement Handbook | 32 © 2017 GRBN Participant Engagement Handbook | 33

10 Golden Tips 10 Golden Tips 8 9 Share back Thank your participants Many of the people who participate in research do so out of Many people participate in research because they want to curiosity and a desire to share their opinions. influence business decisions and help brands and One way to significantly improve the experience you give companies they like. Let them know that their opinion participants is to ask them questions, which they could find counts and is of value to you. interesting, and then share back data with them as to how The best way to do this is to let them know how their other people answered the same questions. participation will help you. Let them know what decisions This share-back technique is regularly used in online will be made because of the research. Be as specific as communities and passive metering research. We encourage you can. you also to make it standard practice in your online survey To give a best in class experience, make it personal and and qualitative research. create a short thank you video. To get you inspired here is Sharing back requires some advance planning, but the an example we used in the recent GRBN Online Survey positive impact you will have on the user experience and the Experience A/B Testing project. image of the company or brand sponsoring the research will make the effort well worth the work. - McMillan,1976 Source: Vision Critical © 2017 GRBN Participant Engagement Handbook | 34 © 2017 GRBN Participant Engagement Handbook | 35

10 Golden Tips 10 Golden Tips 10 We recommend you also ask, at least now and again, why participants found the experience to be either poor or very Use metrics to get better good. You can also ask them to rate the experience. Ask participants for feedback at the end of each engagement. As a bare minimum, ask: USER EXPERIENCE ATTRIBUTES TO CONSIDER INCLUDING WHEN COLLECTING FEEDBACK Boring I was able to express my opinions well Confusing Interesting topic(s) Easy to answer Repetitive We know that there is a very high correlation between a Easy to understand The reward (money/points) positive user experience and a positive impact on the image was fair for the effort of the brand or company behind the survey. Enjoyable Too long Fun Visually appealing Even so, ask participants now and again about the impact of I felt that my participation Well-designed the research experience on the sponsor so you have was valuable evidence to present to your clients that a positive user experience builds positive brand equity. This is something Don’t just ask the metrics questions, but use the feedback clients, especially CMOs and CEOs, should care about you get to improve your practices and your processes. Share deeply. the learnings from the feedback with your colleagues, your clients, and your data collectors. Share your insights with GRBN. We would love to share your success stories with the world and celebrate with you! © 2017 GRBN Participant Engagement Handbook | 36 © 2017 GRBN Participant Engagement Handbook | 37

Tips for Online Communities Tips for Online Communities Table of contents How to create high-engagement communities ........................ 40 1. Think mobile first when designing activities ................... 42 Online 2. Create personas when recruiting .................................... 43 3. Brainstorm recruitment sources ...................................... 44 4. Create a positive first impression .................................... 45 Communities 5. Design engaging messaging when recruiting .................. 46 6. Explain the community purpose when recruiting ........... 48 7. Set expectations from the beginning............................... 49 8. Encourage a sense of community membership .............. 50 Tips 9. Encourage a sense of community influence .................... 51 10. Create an engaging member portal ................................. 52 11. Stop repeating yourself .................................................... 53 12. Keep it interesting ............................................................ 54 13. Show gratitude ................................................................. 56 14. The shorter the better when designing activities ........... 57 15. Share back to members ................................................... 58 How to create high- 16. Keep the conversation going ........................................... 60 engagement communities 17. Encourage storytelling ..................................................... 61 18. Create the right farewell message ................................... 62 19. Share back results when sunsetting your community .... 64 20. Be careful when rebranding or changing a community experience ........................................................................ 65 21. Be respectful when members unsubscribe ..................... 66 22. Use member health metrics ............................................. 67 © 2017 GRBN Participant Engagement Handbook | 39 © 2017 GRBN Participant Engagement Handbook | 1

Tips for Online Communities Tips for Online Communities How to create high- engagement communities These tips have been put together by the experts at Vision Critical based on their considerable experience with the design, recruitment, and management of online communities. The tips give practical advice on how to provide a rewarding and fulfilling experience for community members from the initial stages of recruitment to building an ongoing relationship through activities and feedback. © 2017 GRBN Participant Engagement Handbook | 40 © 2017 GRBN Participant Engagement Handbook | 41

Tips for Online Communities Tips for Online Communities 1 2 Create personas Think mobile first when when recruiting designing activities Today’s customers choose whether or not to participate in People want and expect a personalized experience in online companies’ online communities. Their feedback is communities. Messaging needs to speak directly to invaluable, so it makes sense we solicit their input on customers and allow them to connect emotionally with the devices most convenient for them. Increasingly, customers community sponsor. However, it’s difficult to get to know are reaching for their mobile devices to participate in online each person individually. communities. Consider using personas to define your customer groups. Approximately 50% of activities are taken on a mobile device Personas are fictional, generalized representations that help or tablet, so it is extremely important to keep small screens you relate to your customer groups as humans, and can help in mind when designing and programming an activity. you tailor recruitment approaches to each group. FIVE BEST PRACTICES FOR MOBILE 1. BE CONCISE 2. ALWAYS TEST YOUR ACTIVITY ON A MOBILE DEVICE 3. AVOID GRIDS AND RANK ORDER 4. LIMIT THE NUMBER OF OPEN-ENDED QUESTIONS 5. SIZE IMAGES FOR SMALL SCREEN AND USE ONLY WHEN NECESSARY Source: Vision Critical Source: Vision Critical © 2017 GRBN Participant Engagement Handbook | 42 © 2017 GRBN Participant Engagement Handbook | 43

Tips for Online Communities Tips for Online Communities 3 4 Brainstorm recruitment Create a positive sources first impression Customer personas for key customer groups can help your The invitation survey is the first impression members have company discover unique ways to locate and approach of your online community. The invitation survey should members to join your online community. generate enthusiasm, educate and inform members, and For example, the ȊMillennial Melanieȋ persona shows you collect their qualification information. At the same time, the she prefers social media, spends time online, and is a full- survey should remain short and engaging. time student. Therefore, some ways to approach ȊMelanieȋ Remember, you can build on the relationship with the would be via communications she has subscribed to with members in your online community over time so only your company (email), online intercepts/banners, and essential questions are necessary in the invitation survey. through university message boards. As you learn more from your members, update your personas to keep them robust and available for ongoing recruitment. INVITATION SURVEY CHECKLIST  MOBILE-ENABLED  AVOID DISQUALIFYING PEOPLE UNLESS NECESSARY  SET EXPECTATIONS OF WHAT COMMUNITY MEMBERSHIP WILL BE LIKE  INCLUDE 1-2 RELEVANT INDUSTRY OR TOPIC-SPECIFIC QUESTIONS TO BE MORE ENGAGING Source: Vision Critical Source: Vision Critical © 2017 GRBN Participant Engagement Handbook | 44 © 2017 GRBN Participant Engagement Handbook | 45

Tips for Online Communities Tips for Online Communities 5 Design engaging messaging when recruiting Customers want to feel like you are talking directly to them, so companies can no longer rely on a one-size-fits-all approach to recruitment. You can leverage the work you have done on creating personas to help. Once you have Ȋhumanizedȋ your personas (brainstormed characteristics, assigned a photo and name to each, and discovered unique ways to recruit), you can start to design your messaging to fit each group. On the next page is an example of a social media recruitment message. The tone is conversational and short. In addition, the call to action is easy to find. Source: Vision Critical Source: Vision Critical © 2017 GRBN Participant Engagement Handbook | 46 © 2017 GRBN Participant Engagement Handbook | 47

Tips for Online Communities Tips for Online Communities 6 7 Set expectations from Explain the community the beginning purpose when recruiting Tell people what the online community is for so they can get Set expectations right away on what being a community excited and invested in the purpose. Think of a purpose that member means to your customers. Give them an idea of would appeal to your customers and communicate it from what you will be asking them to do, with actual example their viewpoint. Some examples include: topics and hook questions.  Help us create products and services personalized for Members will appreciate the experience more if they know you what they’re getting into and base their judgement on the  We want to make it easier for you to get our products first activity. Be straightforward, keep it short, and easy to and services complete.  Help inform the future direction of our company SET EXPECTATIONS ON:  CADENCE OF ACTIVITIES  FREQUENCY OF SHARING BACK  RELEVANT TOPICS FOR ACTIVITIES  ACCESSIBILITY OF ACTIVITIES (SHORT AND MOBILE-FRIENDLY) Source: Vision Critical Source: Vision Critical © 2017 GRBN Participant Engagement Handbook | 48 © 2017 GRBN Participant Engagement Handbook | 49

Tips for Online Communities Tips for Online Communities 8 9 Encourage a sense of Encourage a sense of community membership community influence Being in an online community is similar to any other Your customers want to feel like their time and opinions community: your neighborhood, circle of friends, or social matter. Create a community experience where members media groups. Your customers want to feel like they belong feel like they have a say. Members need to know that in your online community and are a part of something someone is listening. special with a purpose. You can do this by sharing back what you’re doing with their You can do this by creating a unique look and feel to your feedback, what decisions are being made, and what will be community, so members are proud to be a part of it. For changing as a result. example, create a name and creative that represents your brand but is unique to the community. Additionally, you can send members sneak peeks or insider knowledge that is exclusive to them. ȃPeople who acknowledge that others' needs, values, and opinions matter to them are often the most influential group members, while those ȃSense of Community is a feeling that members who always push to influence, try to dominate have of belonging, a feeling that members others, and ignore the wishes and opinions of matter to one another and to the community, others are often the least powerful member.Ȅ - Sense of Community and a shared faith that members’ needs will be met through their commitment together.Ȅ (McMillan & Chavis, 1986) - McMillan, 1976 Source: Vision Critical Source: Vision Critical © 2017 GRBN Participant Engagement Handbook | 50 © 2017 GRBN Participant Engagement Handbook | 51

Tips for Online Communities Tips for Online Communities 10 11 Create an engaging Stop repeating yourself member portal No one wants to be asked the same thing over and over; Many online communities have a portal where members can log repetitive questions can frustrate your customers and it in and access activities. Portals can promote a sense of implies you’re not listening to them. More importantly, you community; people have a place to engage with other are not using the power of your online community to profile members, share ideas, and connect. Remember that your members and gain insights that drive important community portals are reflective of your company so the business decisions for your company. The longitudinal look and feel should be consistent and match your brand. nature of online communities provides more opportunities for a variety of insights. Some examples of content to include in your member portal: Source: Vision Critical Source: Vision Critical © 2017 GRBN Participant Engagement Handbook | 52 © 2017 GRBN Participant Engagement Handbook | 53

Tips for Online Communities Tips for Online Communities 12 Keep it interesting PERSONA BUILDER ATTRIBUTES Even though you may do similar surveys, community 1. KEEP IT SHORT AND EASY FOR MEMBERS TO COMPLETE members crave variety and different experiences! Consider (4 CLOSED-ENDED QUESTIONS + 1 OPEN-ENDED QUESTION). putting together a calendar of activities to ensure an array 2. COLLECT RELEVANT DEMOGRAPHIC AND LIFESTYLE of business questions are being addressed at least once INFORMATION. every six months, or by activity type (e.g., online journaling or 3. ENRICH THE PERSONAS BY DESIGNING ACTIVITIES AROUND discussions). THEMES THAT ARE RELEVANT TO THE COMMUNITY MEMBERS Consider adding in Ȋpersona buildingȋ activities to mix it up. AND YOUR BUSINESS. Include short and interactive activities that go beyond 4. SHARE BACK WHAT YOU LEARN WITH YOUR MEMBERS AND business objectives to shed light on who your members are, YOUR STAKEHOLDERS. what they think, and what motivates them. Persona building allows the community manager to learn more about customers and provide more depth to the member relationship. Source: Vision Critical Source: Vision Critical © 2017 GRBN Participant Engagement Handbook | 54 © 2017 GRBN Participant Engagement Handbook | 55

Tips for Online Communities Tips for Online Communities 13 14 Show gratitude The shorter the better Your customers want to feel like their participation is when designing activities appreciated. Remember to show gratitude and thank members for sharing their insights and opinions. One of the goals of your community is to develop an ongoing relationship with your customers. Members don’t want to do You can also offer Ȋsurprise & delightȋ rewards for long surveys anymore, which is why survey response rates participation, but avoid incentive programs aimed to are low and plummet each year. Most customers prefer motivate individuals. Incentive programs aren’t sustainable; surveys that are less than 10 minutes long, according to the people who expect money for their time lose their innate GreenBook Research Industry Trends Report. desire and motivation to participate. A sense of community If you have a longer activity, consider splitting it in two. One can be lost when the focus shifts from genuine participation of the benefits of an online community is the iterative in research to extrinsic rewards. process; it’s better to have several shorter activities than a long one that leads to fatigue. Engaging members more frequently with shorter activities can help cultivate an ongoing relationship with members. ȃPeople generally lose concentration after eight seconds, highlighting the effects of an increasingly digitalized lifestyle on the brain.Ȅ - Source: Microsoft survey of Source: The puzzle of motivation, Dan Pink via TED Canadian media consumption Source: Vision Critical © 2017 GRBN Participant Engagement Handbook | 56 © 2017 GRBN Participant Engagement Handbook | 57

Tips for Online Communities Tips for Online Communities 15 Share back to members Share-backs are important as members get to see firsthand that their feedback was valued and used for positive change. Members have a more positive brand experience and are more engaged in their community when research results are shared. The table below shows the key reasons why individuals join and remain active in communities. REASONS FOR JOINING AND Source: Vision Critical REMAINING IN A COMMUNITY: 1. FIND OUT ABOUT NEW PRODUCTS 2. HAVE A VIP EXPERIENCE, OR ACCESS INSIDER INFORMATION 3. BE HEARD 4. LEARN SOMETHING NEW 5. RECEIVE ACTIVITY RESULTS 6. LEARN HOW THEIR FEEDBACK INFLUENCES DECISIONS Source: Vision Critical © 2017 GRBN Participant Engagement Handbook | 58 © 2017 GRBN Participant Engagement Handbook | 59

Tips for Online Communities Tips for Online Communities 16 17 Keep the Encourage storytelling conversation going Encourage members to their stories—about themselves, about Online communities encourage members to share stories their experiences, and about your brand. This helps members about themselves and their experiences. Companies can to get to know one another and feel an emotional connection to use probing questions to get people thinking and start a others in the community. It can also help you uncover hidden dialogue with community members. For example, ask a insight and inspire new ideas across your organization. ȊThink Aloud Questionȋ where members recount a recent You can share members’ stories through regular share- experience in as much detail as possible. The goal of this backs, your community portal, and in activities. activity is to capture evaluations based on the Ȋvoice of the customer.ȋ It allows the consumer to assess a situation or a product in their own words, rather than reacting to structured questions. ȃA lotto and gaming company was collecting stories about how customers choose their lotto numbers. They received fascinating stories! One was so impressive they followed up with the member and ultimately produced a TV commercial that features her and her story.Ȅ - Vision Critical customer story based on actual online community use case Source: Vision Critical Source: Vision Critical © 2017 GRBN Participant Engagement Handbook | 60 © 2017 GRBN Participant Engagement Handbook | 61

Tips for Online Communities Tips for Online Communities 18 Create the right farewell message RECOMMENDED CONTENT IN SUNSET MESSAGING If a decision is made to shut down an online community, it  THANK MEMBERS FOR THEIR TIME AND PARTICIPATION. is important to notify the members in an authentic and  INFORM THAT THE PORTAL WILL BE TAKEN DOWN, AND THERE transparent way. Members view the community as an WILL BE NO FURTHER INVITATIONS/SHARE-BACKS. extension of your brand and will express their opinions  RECAP MAIN DECISIONS MADE FROM COMMUNITY INPUT. about the experience via a variety of channels (e.g., social  ASSURE THAT INFORMATION IS PROTECTED BY THE PRIVACY media). POLICY. Make sure the sunset messaging is written in the right tone to match your brand and provides a way for members to leave comments. Consider keeping an activity open for the remaining weeks of the community so members can submit Source: Vision Critical feedback. Source: Vision Critical © 2017 GRBN Participant Engagement Handbook | 62 © 2017 GRBN Participant Engagement Handbook | 63

Tips for Online Communities Tips for Online Communities 19 20 Share back results when Be careful when rebranding sunsetting your community or changing a community Members contribute to various experience activities over the life of a If your company is going through changes that require you community, so it is recommended to rebrand or restart your community under a new name, it’s to send a summary of results if important to notify members that you are doing so and allow you are shutting the community them to decide whether they stay or go. down. When a community shuts down, You should inform members of the change, thank them for the aim is to leave members with their participation to date, and encourage them to continue a good impression of your membership in the Ȋnewȋ community. Transparency with company. It’s important to first members is key in building authentic relationships. thank community members for their time and effort. Then consider including an infographic of the results or a summary WHEN THE NEW COMMUNITY LAUNCHES… newsletter of all the insights you  HIGHLIGHT WHAT HAS CHANGED (NAME, BRANDING, ETC.). have collected over time that  ASK MEMBERS TO BOOKMARK THE NEW COMMUNITY members have contributed to. DOMAIN.  INVITE MEMBERS TO A FEEDBACK ACTIVITY.  PROVIDE A REMINDER OF INFORMATION PROTECTED BY PRIVACY POLICY. Source: Vision Critical Source: Vision Critical © 2017 GRBN Participant Engagement Handbook | 64 © 2017 GRBN Participant Engagement Handbook | 65

Tips for Online Communities Tips for Online Communities 21 22 Be respectful when Use member health metrics members unsubscribe A healthy community is one with engaged members that are At any time, your members can decide they no longer want motivated to participate. An involved community leads to to be a part of the community. People may leave the higher response rates, better quality data, and lower churn. community for reasons that are out of your control; they are Key community health metrics that should be tracked too busy, are no longer a customer, or have lost interest in include size, participation and utilization, community the topic. Other reasons for leaving may be in your control; profile, activity cadence, and response rates. Non- they feel like they aren’t being listened to, are not receiving responders can be removed on a regular basis to free up any share-backs, or don’t feel a sense of community. space and improve data quality. New members should be If possible, ask individuals why they are leaving when they recruited regularly to provide a fresh perspective and boost unsubscribe so you can be aware of any improvements you the size of the community. can make. ‘ Source: Vision Critical Source: Vision Critical © 2017 GRBN Participant Engagement Handbook | 66 © 2017 GRBN Participant Engagement Handbook | 67

Tips for Online Surveys Tips for Online Surveys Table of contents How to motivate people to take your surveys and to do so again ...................................................................... 72 What makes a good survey – in the words of Online survey-takers ................................................................... 76 What makes a poor survey – in the words of survey-takers ................................................................... 77 23. Understand what motivates people to take part ..... 78 Survey 24. Build a user experience best practice knowledge hub .......................................................... 79 25. Hire a professional… ................................................. 80 …because faulty mechanics are a big turn-off… ...... 82 …you should engage your data collector early on… 83 Tips …and humanize your survey ..................................... 84 26. Eat your own dog food .............................................. 85 27. Do cognitive interviewing before launch.................. 86 28. Do qualitative research to inform how to write the questionnaire ...................................................... 87 How to motivate people to take your surveys and 29. Ensure mobile friendliness ....................................... 88 to do so again 30. Pilot your questionnaire properly ............................ 91 31. WIIFM – sell them the experience ............................. 92 32. Promise and pay a fair incentive .............................. 93 33. Money isn’t enough… ................................................ 94 …and charitable donations can increase motivation .................................................................. 95 34. Honesty is the best policy ......................................... 96 © 2017 GRBN Participant Engagement Handbook © 2017 GRBN Participant Engagement Handbook | 69

Tips for Online Surveys Tips for Online Surveys Table of contents (cont’d) Table of contents (cont’d) 35. Give examples of how similar research has helped 49. Write like a human… ................................................ 123 the client .................................................................... 98 …by being concise and conversational ................... 124 36. Promise to let them know the sponsor of the 50. Make questions easy… ............................................ 126 research at the end of the survey .......................... 100 …and use simple wording to encourage intrinsic 37. Spell out in the invite what you expect of motivation ................................................................ 128 survey-takers ........................................................... 101 51. Don’t make people have to tell you Ȋliesȋ ............... 130 38. Put your own skin in the game ............................... 102 52. Go off the grid ......................................................... 132 39. Avoid email reminders ............................................ 103 53. Create surveys that people actually enjoy… .......... 133 40. Never ask people the same questions ................... 104 …so you don’t bore them to death ......................... 134 41. Ask all screening questions at the beginning ........ 106 54. Gamify your surveys with words for more engagement and better results .............................. 136 42. Write shorter questionnaires… ............................... 108 55. Break it up – have some fun ................................... 138 …by focusing on the key objectives… ..................... 110 56. Use motivational language ..................................... 139 …and avoiding long lists .......................................... 111 57. Look and feel matter ............................................... 140 43. Short is good, relevant is better ............................. 112 58. Convenience is key .................................................. 141 44. Show interest in the participants as PEOPLE ......... 113 59. Respect people’s privacy ......................................... 142 45. Use native language… ............................................. 114 60. Avoid assumptions .................................................. 143 …and take care with translations ........................... 116 61. Share back ............................................................... 144 46. Use video to make research easy and engaging .... 118 62. Say Ȋthank you!ȋ ....................................................... 147 47. Text comments enable customers to tell you …by telling them who sponsored the survey ......... 148 exactly what they feel you need to hear ................ 120 …and inspire them with a video for maximum effect ........................................................................ 149 48. Break it up – get personal ....................................... 122 63. Ask for and act on feedback ................................... 152 64. Analyze Ȋdrop outsȋ ................................................. 154 65. Control the quality................................................... 156 © 2017 GRBN Participant Engagement Handbook | 70 © 2017 GRBN Participant Engagement Handbook | 71

Tips for Online Surveys Tips for Online Surveys How to motivate people to Three of the key hypotheses we were able to support were: take your surveys and to do so again 1. GIVING SURVEY-TAKERS DETAILED The tips in this section come not only from the collective INFORMATION ON WHY THEIR experience of experts in the field of online surveys but also PARTICIPATION WILL BE OF VALUE TO THE from numerous research-on-research studies which have CLIENT AND THEIR CUSTOMERS IMPROVES been conducted on the online survey user experience. THE BRAND EQUITY OF THE SPONSOR. As part of the GRBN Participant Experience Initiative, we wanted to build on the great research already conducted 2. SHOWING A VIDEO OF THE CLIENT and do something different. We decided to undertake a PERSONALLY THANKING SURVEY-TAKERS large-scale A/B testing project in order to test how changing AND EXPLAINING HOW THEIR a survey impacts both the user experience and the impact PARTICIPATION WILL BE OF VALUE TO THE on the brand sponsoring the survey. CLIENT AND THEIR CUSTOMERS IMPROVES GRBN Online Survey Experience A/B Testing BOTH THE USER EXPERIENCE AND THE In the test, we tested several hypotheses and ended up with BRAND EQUITY OF THE SPONSOR. 20 different versions of the questionnaire. 3. SHARING BACK DATA WITH SURVEY- Interestingly, one of the key hypotheses that we failed to TAKERS IMPROVES BOTH THE USER provide support for was that shorter screeners provide a EXPERIENCE AND THE BRAND EQUITY OF better experience than longer screeners. Based on the THE SPONSOR. feedback we received from the survey-takers, the reason for Source: GRBN Online Survey Experience this was that the questions we added to the survey were of A/B Testing interest to them, with the moral of the story being: ȊSHORT IS BETTER THAN LONG, BUT INTERESTING IS EVEN BETTER.ȋ © 2017 GRBN Participant Engagement Handbook | 72 © 2017 GRBN Participant Engagement Handbook | 73

Tips for Online Surveys Tips for Online Surveys We would like to thank all our partners on the project, and in particular Critical Mix, who stepped in last minute to program, host, and manage the fieldwork: ƈA good user experience is the foundation of a trust- based relationship with participants.Ɖ We would like to thank Cait Wilson, from the University of Utah, for creating the motivational/conversational survey design we tested as part of the research. You can read in Tip 49 how survey-takers reacted to that approach. We would also like to thank Cait for assisting with the quantitative and qualitative data analyses. We would also like to extend a special thanks to MetLife for allowing us to use their questionnaire for testing purposes. In particular, we would like to thank Neil Marcus for creating the video message we used in the test. You can read about the impact the video had on the perception of the survey sponsor here. © 2017 GRBN Participant Engagement Handbook | 74 © 2017 GRBN Participant Engagement Handbook | 75

Tips for Online Surveys Tips for Online Surveys What makes a poor What makes a good survey - in the words survey - in the words of survey-takers of survey-takers Source: P2Sample feedback data Source: P2Sample feedback data © 2017 GRBN Participant Engagement Handbook | 76 © 2017 GRBN Participant Engagement Handbook | 77

Tips for Online Surveys Tips for Online Surveys 23 24 Understand what motivates Build a user people to take part experience best practice Giving a fair incentive is a must, but you need to understand knowledge hub what else motivates potential participants and align the Delivering a great user experience requires practice. Using experience you give them to those motivations. this handbook will help, but you will ultimately need to For example: Making the survey fun to answer, sharing back decide what works best for your participants and your data, and telling people how their participation will help the brand. client are all great ways to motivate people. Delivering great experiences shouldn’t be left to chance or dependent on which researcher in your team is planning the research and writing the questionnaire. Collect examples of what works and what doesn’t; share that knowledge amongst your researchers, as well as with your clients and suppliers. ȃCollecting feedback from survey- takers on what engages, and what doesnȂt, enables our researchers to design better surveys by learning from each other and developing best practices.Ȅ - David Rothstein, RTi Research Source: GRBN Online UX Survey 2016 © 2017 GRBN Participant Engagement Handbook | 78 © 2017 GRBN Participant Engagement Handbook | 79

Tips for Online Surveys Tips for Online Surveys 25 Hire a professional… ȃSome are vague or force you into an answer Everyone knows how to ask a question, right? Maybe, but that doesn't fit for you.Ȅ that doesn’t make everyone an expert at questionnaire ȃErrors, bad spelling, hard to understand design. questions, bad scripting, bad formatting.Ȅ Participants dislike poor survey design; poorly written ȃInability to express my opinion. No I DO questions, illogical response options, and a poor layout are NOT KNOW or NOT APPLICABLE option.Ȅ just some of the things they complain about. ȃVery unclear confusing survey questions.Ȅ This shows a lack of respect for the participant and makes ȃHUGE grids to rate LONG LISTS of their experience unnecessarily poor. It may give participants criteria.Ȅ a bad impression of your company and potentially your client and can deliver poor quality data. ȃBoring repetitive survey layouts page after Designing a great questionnaire is both a science and an art, page.Ȅ so do yourself a favor and hire a professional to take care of ȃMany or most are written by people who don't this critical task. properly understand English, or the attitudes that people will have towards products or services.Ȅ - Quotes from survey-takers about unprofessional surveys © 2017 GRBN Participant Engagement Handbook | 80 © 2017 GRBN Participant Engagement Handbook | 81

Tips for Online Surveys Tips for Online Surveys 7 …because faulty mechanics …you should engage your are a big turn-off… data collector early on… People have very high expectations when it comes to digital Even if you are an expert questionnaire designer, your data experiences. With a plethora of amazing experiences collector partner should be involved early in the design available online, especially on a smartphone, people are not process. Your data collector makes a living from engaging going to tolerate a technically poor online survey people to participate in research and should know what experience. drives a good and bad user experience. Broken links, faulty skip logic, getting thrown out and not Send your questionnaire outline to your data collector and being allowed back in, having to start over are all things ask their advice on design issues such as responsiveness, guaranteed to create angry participants. And you don’t want routing, radio buttons, drag and drops, scales, sliders, and angry participants. autofills. ȃWhen I click to the finish survey and it Email correspondence example from a Research For gives me an error sign and won't let me Good client: back into the survey.Ȅ ȃMy bad experiences with surveys have been ȃIȂm perfectly fine with starting sending sample the ones that are not well executed, freeze towards hard-to-reach targets and your plan up constantly, or do not play back video or below seems perfect; letȂs do as you suggested. audio correctly.Ȅ Regarding the education, you are correct with your assumptions, thanks for looking into this. - Quotes from unhappy survey-takers IȂll send news as soon as I have them. Cheers!" © 2017 GRBN Participant Engagement Handbook | 82 © 2017 GRBN Participant Engagement Handbook | 83

Tips for Online Surveys Tips for Online Surveys 7 26 …and humanize your survey Eat your own dog food Here are some of the ways your data collector might be able Test your own survey, then test it again. Have co-workers to help you humanize your survey if they are involved in the test it. Have your client test it. Have your parents, your son, process early enough: and your best friend test it. Have your barista test it. If they can't get through it (or don't want to), it’s likely those taking your survey will feel similarly. Yes, some will power through for the incentive, but what FIVE WAYS YOUR DATA COLLECTOR does this mean for the quality of their answers and what CAN HELP HUMANIZE YOUR SURVEY taste does this leave in their mouth for the next time? 1. BY COPYWRITING IT: AVOIDING JARGON, SIMPLIFYING INSTRUCTIONS, CONSIDERING THE ONLINE ENVIRONMENT, MAKING SURVEY ANSWERS EXHAUSTIVE AND MUTUALLY ȃWhen testing your survey, donȂt EXCLUSIVE. only test the mechanics – links, 2. BY LETTING YOU AVOID SOME QUESTIONS THAT ARE routings, etc., also test the experience CURRENTLY IN THE PANEL PROFILING DATABASE. you are going to give people.Ȅ 3. BY ENSURING ITS USABILITY AND MOBILE COMPLIANCE. - Baillie Buchanan, Research For Good 4. BY NEUTRALIZING AND AVOIDING SOME BIASES. ȃAlways test on a smart phone or use 5. BY DESIGNING AN APPROPRIATE INCENTIVE SCHEME. an emulator, like the one in Chrome.Ȅ Source: Netquest - Roddy Knowles, Research Now © 2017 GRBN Participant Engagement Handbook | 84 © 2017 GRBN Participant Engagement Handbook | 85

Tips for Online Surveys Tips for Online Surveys 27 28 Do cognitive interviewing Do qualitative research before launch to inform how to write We never know if our surveys will make sense to people the questionnaire unless we talk to them. For decades, trained survey Surveys need to make sense to participants and match their professionals have used cognitive interviewing to find and ame of reference.ȋ Too many times, we ask people fix flaws in questionnaires. We all need to practice this Ȋfr technique, especially when we have not done qualitative questions that have jargon, technical terms, and unclear research or are researching something new. terms and phrases. Cognitive interviewing involves conducting interviews with Do qualitative research first to know what to ask and how to people from each segment that take the questionnaire. We ask it. In qualitative research, you will discover the words ask them to think aloud as they read and answer questions. and phrases people use to describe concepts. You will find We use standard cognitive probes, such as: out what really matters to them, and this will inform what questions to ask and how to ask them. Further, when people  Please repeat the question, in your own words. skip qualitative research, they often expect the quantitative  What does the term Ȋoutpatientȋ mean to you? exercise to gather insights that only qualitative research can  How are you sure that your insurance covers mental address. health treatment? There is no substitute for qualitative work. Skipping this step The only way you know your questions make sense and is one of the hallmarks of poorly written questionnaires. people can answer them is to do cognitive interviewing. David Harris, author of The Complete Guide to Writing David Harris, author of The Complete Guide to Writing Questionnaires: How to Get Better Information for Questionnaires: How to Get Better Information for Better Decisions Better Decisions © 2017 GRBN Participant Engagement Handbook | 86 © 2017 GRBN Participant Engagement Handbook | 87

Tips for Online Surveys Tips for Online Surveys 29 Ensure mobile friendliness WE SEE THAT IN NORTH AMERICA AT LEAST 30% OF OUR POTENTIAL PARTICIPANTS COME Hitting demographic quotas does not mean the right people TO US LOOKING TO PARTICIPATE IN A SURVEY are participating in your study. To achieve as close to a VIA THEIR MOBILE DEVICE. IN OTHER representative sample as possible, make sure people can COUNTRIES, THAT NUMBER CAN BE MUCH participate on the device of their choosing. HIGHER, FOR EXAMPLE, AS MUCH AS 80% IN Smartphone-first participants differ in notable ways from BRAZIL. EXCLUDING THESE HIGH % OF PC-first participants with respect to key psychographic and PARTICIPANTS THAT MEET YOUR OTHER behavioral traits that are important for researchers and DEMOGRAPHIC OR SCREENING CRITERIA marketers to know. ENSURES YOUR DATA IS NOT REPRESENTATIVE OF THE TARGET AUDIENCE. Source: Research For Good How Are Mobile-first Participants Different? Personality Differences Trend/fashion follower 48% 59% 37% 37% FURTHER READING SUGGESTED BY P2SAMPLE: 63% 69% Smartphone-first Likes excitement 54% 51%  Quirks: When it Comes to Mobile Respondent Experience and Data Quality, Likes variety 87% 91% Survey Design Matters 82% 80%  KinesisPanel: Survey Data Quality in an 80% 77% PC-first Evolving Market Research Ecosystem Likes new/different things 71% 70%  DataSpring: How Mobile Respondents Can ©2016 Research Now Group, Inc. 13 Affect Data Quality © 2017 GRBN Participant Engagement Handbook | 88 © 2017 GRBN Participant Engagement Handbook | 89

Tips for Online Surveys Tips for Online Surveys 30 One example of how to design a questionnaire for mobile is to provide people with options for answering open-ended questions, such as voice to text. Responses to open-ended questions are usually shorter on Pilot your questionnaire properly mobile devices, though that doesn’t mean data quality is lower. Keep mobile in mind and ask questions that can be answered concisely. Remind participants they can use voice Most surveys are soft-launched to test survey length and the to text to respond, but don’t force them. Those who use mechanics, but how many are properly piloted? While this is voice to text usually provide more depth. important, it is just as important to collect feedback from participants on what works and what doesn’t, and then make appropriate changes before launch. Time for piloting needs Voice-to-Text Quality of Response – Words Used to be built into the process. 20 16 12 8 4 FURTHER READING SUGGESTED BY P2SAMPLE: 0 Mobile Phone Tablet Typed Voice-to-Text  Explorable: Pilot Survey Q: In as much detail as possible, please share the one thing that [RESTAURANT NAME] could do to gain more of your business in the future. This  study.com: What is a Pilot Study?- could include additional menu items (please specify items if this is the case), pricing, hours of operation, service, interior design, cleanliness, etc. Definition & Example 23  Nielsen Norman Group: Pilot Testing: Getting It Right (Before) the First Time  smartamarketing.com: Preliminary or Exploratory Marketing Research – Don’t Just Launch Yourself Into Market Research © 2017 GRBN Participant Engagement Handbook | 90 © 2017 GRBN Participant Engagement Handbook | 91

Tips for Online Surveys Tips for Online Surveys 31 32 WIIFM – sell them Promise and pay a the experience fair incentive Germán Lowe, Netquest CEO, reminded everyone at the The incentive is one of the key motivators for panel Insights Association’s Next Conference that Ȋthere was a members to take a survey. Whilst it is not the only motivator time when taking a survey was one of the most fun things (see Tip 23), it is an important part of the package you are you could do on the internetȋ. This is hard to believe today, positioning to potential participants. The incentive needs to with YouTube, Instagram, and many other platforms that be in line with the effort made and communicate you value offer people much more fun ways to spend their time. the time participants are giving you. A fair incentive has a If you want people to take your survey, especially on their positive effect on both the experience and data quality. mobile, you need to offer a clear value proposition and In particular, consider the time and effort people need to stress the What's In It For Me? take the screener. A 10-minute questionnaire with a 5-minute screener is a 15-minute questionnaire. If But be careful with design, especially images, as you don’t you have a long screener, it is only fair to pay those who want your invite to be qualified as "spam" by the main mail providers. screen out, especially if you use the data. ȃTime to take the survey and the amount of money paid were not equal to what I thought I should have received.Ȅ - Quote from an unhappy survey-taker © 2017 GRBN Participant Engagement Handbook | 92 © 2017 GRBN Participant Engagement Handbook | 93

Tips for Online Surveys Tips for Online Surveys 33 35 Money isn’t enough… …and charitable donations It’s not a simple exchange of time for money; you will get can increase motivation more valuable information from your participants if they are A small charitable donation on top of a personal incentive engaged in what they are doing and really feel that their can increase engagement. input is appreciated. For example, let participants know their feedback will make People’s motivations to participate vary, from wanting to a difference. Always test your surveys and ask yourself if you shape products and services to just in it for the incentive. would want to take part. There is evidence the feel-good factor of a charitable donation increases the likelihood of a participant sticking with a survey and/or returning for the next invite. That said, a donation-only model may have the opposite effect and/or bias the data. We only recommend a donation A SURVEY WITH THE HIGHEST in addition to a personal incentive, not instead of. INCENTIVE DOESN’T GET HIGHER SATISFACTION SCORES. ȃI think it is awesome that you are helping feeding hungry children! It makes me want to Source: MESH Experience metrics data complete more surveys on this page.Ȅ - Quote from a Research For Good survey-taker Source: Mesh Experience Source: Research For Good © 2017 GRBN Participant Engagement Handbook | 94 © 2017 GRBN Participant Engagement Handbook | 95

Tips for Online Surveys Tips for Online Surveys 34 Honesty is the best policy 18% OF SURVEYS HOSTED IN THE PLATFORM ARE LONGER IN One thing survey-takers complain about regularly is being PRACTICE THAN INDICATED TO misled in the invite about the length of surveys. At best, this PARTICIPANTS IN THE INVITE. can make you seem incompetent. At worst, people may think you are dishonest and can reflect on the reputation of your client’s brand. This is another good reason to pilot your ON AVERAGE, THESE SURVEYS ARE survey properly before launch (see Tip 30), and when doing so, remember to test the survey on a smartphone. 45% LONGER THAN PROMISED. DISHONESTY ABOUT THE LENGTH ȃI dislike the fact that most of these surveys are CREATES MORE COMPLAINTS ABOUT double and even triple the length of time you're THE SURVEY BEING LONG AND told that they are at the beginning.Ȅ NEGATIVELY IMPACTS SATISFACTION - Quote from an unhappy survey-taker WITH THE SURVEY. ȃSmartphone surveys take 20-40% longer than on PC, depending on quality of design.Ȅ Source: P2Sample metrics data - Roddy Knowles, Research Now © 2017 GRBN Participant Engagement Handbook | 96 © 2017 GRBN Participant Engagement Handbook | 97

Tips for Online Surveys Tips for Online Surveys 35 This type of additional information was particularly appreciated by those with an interest in the insurance category, regular survey-takers, and those motivated out of Give examples of how a desire to influence things. Sharing this type of insight with similar research has participants had a very positive impact on the perception of helped the client the survey sponsor. Participants want to know that spending their time on a survey means something. For example, if you are conducting a segmentation survey, tell them how similar surveys have helped the client develop a new product or service to meet specific customer needs. If you are conducting a customer experience survey, tell them how similar surveys have helped the client improve a specific aspect of their service. Talk in general terms in the invite to the survey so as not to ȃI feel like they are bias the research, but promise to be specific at the end of actually interested the survey and follow through on that promise. In our A/B in finding out the testing research, we informed participants that: results and will use ȊPrevious studies like this provided valuable input that was used to the information create new insurance and savings products for consumers. For example, similar research conducted last year guided the to provide development of their new MetLife Rapid Term life insurance better service.Ȅ product that was launched earlier this year. This product has been designed to deliver on key needs of affordability, simplicity, and customization as identified by the responses given by the survey participants.ȋ Source: GRBN Online Survey Experience A/B Testing © 2017 GRBN Participant Engagement Handbook | 98 © 2017 GRBN Participant Engagement Handbook | 99

Tips for Online Surveys Tips for Online Surveys 36 37 Promise to let them know Spell out in the invite what the sponsor of the research you expect of survey-takers at the end of the survey It might be obvious to you, but don’t assume survey-takers Whilst you don’t want to risk biasing the survey, in most know automatically what is expected of them. Spell it out. cases, there is no harm in revealing the name of the survey For example, you might not want the participants to take the sponsor at the end of the survey. survey on their mobile device due to the nature of the stimuli or the complexity of the task you are asking them to In fact, assuming the experience you give people is positive, do. Or you might want them to be at home when taking the this will leave a positive impression on the company or survey and not on the move. We need to ensure participants brand behind the survey. It will also increase people’s are able and safe to participate. motivation to take surveys in the future. Be clear on the expectations in the invite, otherwise, you risk losing perfectly capable participants and giving them a very negative experience. Source: GRBN Online UX Survey 2016 Source: Kelsy Saulsbury, Schwan's Shared Services, IIeX NA 2017 © 2017 GRBN Participant Engagement Handbook | 100 © 2017 GRBN Participant Engagement Handbook | 101

Tips for Online Surveys Tips for Online Surveys 38 39 Put your own skin Avoid email reminders in the game You might be upset that not everyone is answering your Just because you have the opportunity to hide your identity survey, but experience tells us that you have more to lose when designing surveys doesn’t mean that you should. Sign than gain by sending out email reminders. your survey invites with your own name and at the end of Accept that people might have better things to do than the survey give them your email address so people can answer your survey and move on. You’ll be more likely to get contact you with their feedback. them next time and won’t risk losing them forever. We guarantee that people will appreciate your transparency and you’ll learn a lot from the direct feedback you’ll get. Source: Kelsy Saulsbury, Schwan's Shared Services, IIeX NA 2017 Source: Kelsy Saulsbury, Schwan's Shared Services, IIeX NA 2017 © 2017 GRBN Participant Engagement Handbook | 102 © 2017 GRBN Participant Engagement Handbook | 103

Tips for Online Surveys Tips for Online Surveys 40 Never ask people the Survey Drop Rate Sharing vs. Not Sharing same questions Redundant Screener Data 30% One of the things survey-takers complain about most is 25% being asked the same questions, especially during 20% screening. 15% This will cause people to drop out before they ever get the 10% chance to answer your survey. This may also put them in a 5% bad frame of mind when they do take the survey and make 0% them think badly about your company or brand. Jan-17 Feb-17 Mar-17 Apr-17 May-17 Jun-17 Jul-17 Aug-17 Sharing Not Sharing So, do yourself a favor and insist that your data collectors accept pre-existing data (such as demographic information) Source: Research For Good via the survey URL. This shortens the survey experience and lowers the chance of a frustrated participant supplying poor quality responses because they’re annoyed they had to tell you their household income, yet again. Source: Research For Good © 2017 GRBN Participant Engagement Handbook | 104 © 2017 GRBN Participant Engagement Handbook | 105

Tips for Online Surveys Tips for Online Surveys 41 Quotes from survey-takers: Ask all screening questions ȃI dislike answering 20 minutes or more of at the beginning specific questions only to be told I did not qualify. I think my time has value and I wish more surveys took this into consideration.Ȅ Participants give us their valuable time and opinions; treat those gifts with respect. If a participant does not qualify, ȃAnswering questions for 10 minutes, then thank them and end their participation as early in the survey being told I don't qualify. In that time, they as possible. Late terminates are to be avoided entirely and ask for enough information that my answers filled quotas should be quickly communicated to sample could be included. Sad that a company would suppliers so the sample can be managed accordingly. Pay scam people for $.75. Lol.ȃ those who do manage to complete the survey even if the ȃI would get accepted into, complete it, get to quota is full. the end, and become disqualified because the Engage your sample partner in the survey creation/testing survey reached its quota while I was taking process; they can provide feedback to help avoid participant the survey.Ȅ frustration. - Source: GRBN Online UX Survey 2016 ȃI don't enjoy being disqualified after taking FURTHER READING SUGGESTED BY P2SAMPLE: my time to give considerate answers. Makes  Cloudcherry: Why a Targeted Audience is me want to never consider taking any more a Critical Component of Effective Survey surveys ever again on this site.Ȅ Questionnaire Design - Quote from a Research For Good survey-taker  CheckMarket: The Importance of Socio- Demographics in Online Surveys  Inc: How to Define Your Target Market © 2017 GRBN Participant Engagement Handbook | 106 © 2017 GRBN Participant Engagement Handbook | 107

Tips for Online Surveys Tips for Online Surveys 42 ȃKeep questionnaires under 20 minutes.Ȅ - Jackie Lorch, SSI Write shorter questionnaires… The keys to writing shorter questionnaires Shorter questionnaires are essential for participant involve consulting with clients on the decisions engagement and for data quality. the survey will help inform and then designing the survey to inform those decisions. State decisions as HE declarative statements. For example: 1. We will decide which of three concepts to progress. ALL ELSE BEING EQUAL, EVEN A 2. We will decide which group to target and whether to target more than one group. ONE MINUTE SHORTER QUESTIONNAIRE Most decisions are made based on no more than three to 10 PRODUCES HIGHER SATISFACTION pieces of information. RATINGS AMONGST THE SURVEY-TAKERS. This process helps the team focus on the information WE SEE A MASSIVE SATISFACTION needed to inform the decisions. It helps avoid all the Ȋnice- DIFFERENCE BETWEEN A 15-MINUTE AND to-knowȋ questions. Consider developing professional A 30-MINUTE SURVEY IN OUR DATA. questionnaire design skills. As a result, the team will want you to design the survey rather than ask you to email the survey around for everyone else to edit. Source: MESH Experience metrics data David Harris, author of The Complete Guide to Writing Questionnaires: How to Get Better Information for Better Decisions © 2017 GRBN Participant Engagement Handbook | 108 © 2017 GRBN Participant Engagement Handbook | 109

Tips for Online Surveys Tips for Online Surveys 45 …by focusing on the …and avoiding long lists key objectives… Long lists are especially problematic on smartphones Develop a briefing document which enables clients to because the scrolling required to view them is frustrating specify the key business objectives and separate them out and can negatively impact data quality. from Ȋnice-to-knowȋ ones. Primacy effects (selecting what is first visible at the top of a Align the research objectives in the same way and focus the list) and recency effects (selecting what is last seen at the survey on collecting data that will inform the business bottom of a list) are more pronounced on smartphones, and objectives. items in the middle of long lists are less often selected. ȃShorten questions. Reduce the number of Response response options/attributes by conducting/ choices leveraging exploratory analyses. Determine get buried in long which attributes/questions are core and only lists include those in trackers and other relevant studies.Ȅ - Client, Financial Services Source: Research Now © 2017 GRBN Participant Engagement Handbook | 110 © 2017 GRBN Participant Engagement Handbook | 111

Tips for Online Surveys Tips for Online Surveys 43 44 Short is good, Show interest in the relevant is better participants as PEOPLE If your starting point is focused purely on what you want to A good conversation starts with asking get out of the survey, then the advice is to keep the questions about the person you are talking to. experience as short as possible. If you are also interested in What are they up to? How are they feeling? what participants can get out of the survey, then you are not What are they interested in? How do they feel about various so constrained. issues? Our A/B testing supports that you can create a 25-minute In longer questionnaires, we love the use of these kinds of survey, which the vast majority of survey-takers found to be questions. They enable you to add filters later based on a very good experience and would willingly take similar these questions and to personalize the experience by adding types of surveys again. The key: relevance. survey-break questions. When you ask a Ȋwhat are you interested inȋ question, embed the category you are researching into the list. You will most likely find huge differences when analyzing your data by Ȋthose very interested in the categoryȋ vs. Ȋthose not interested in the category, which could lead to some interesting insights for your marketing team. Once you’ve shown survey-takers you are interested in them as individual people, they are much more primed to answer the duller questions you might need to ask to get to your - insights. - Source: GRBN Online Survey Experience A/B Testing © 2017 GRBN Participant Engagement Handbook | 112 © 2017 GRBN Participant Engagement Handbook | 113

Tips for Online Surveys Tips for Online Surveys 45 Use native language… When was the last time you paid a bill online? 40% 41% Even for populations like B2B where you might expect 18% 19% 18% people to speak English, use native language. 17% 14% 12% It can be tempting to save time and expense by fielding 7% 6% 3% 2% 3% 2% surveys in English, especially for groups we assume readily speak English, such as higher income or better educated Past day Past week Past Past 6 Past year Over a I have month months year ago never paid demographics, or business-to-business populations. This Germany (English) Germany (German) a bill practice risks hidden misunderstandings which could online change your data and make results less valid. Translating surveys using a professional agency is highly recommended. Based on this commercial, please complete the following sentence as you remember SSI tested the impact of native vs. non-native it from the video. language in China and Germany. Participants were 33%33% 36% asked if they spoke English, then shown questions in English and native language. Results showed more 20% 16% response consistency when a questionnaire was 13% 13%10% designed in the native language. 9% 8% 6% 5% Learn more at this blog Option 1 Option 2 Option 3 Option 4 Option 5 Don’t knoǁ China (English) China (Chinese) Source: SSI Source: SSI © 2017 GRBN Participant Engagement Handbook | 114 © 2017 GRBN Participant Engagement Handbook | 115

Tips for Online Surveys Tips for Online Surveys …and take care with translations Take extra care for translations involving technical terms, especially in B2B, healthcare studies, and multi-country studies where translation of technical terms might not be a standard translation for all markets. Good quality translations drive user participation and engagement in surveys. Non-interpretation of terms, questions, and instructions can drive users away. FURTHER READING SUGGESTED BY P2SAMPLE:  GEOFF NORBY: Translation for Market Research  TRANSLATING QUESTIONNAIRES AND OTHER RESEARCH INSTRUMENTS, ISSUE 133 (BY ORLANDO BEHLING, KENNETH S. LAW) © 2017 GRBN Participant Engagement Handbook | 116 © 2017 GRBN Participant Engagement Handbook | 117

Tips for Online Surveys Tips for Online Surveys 46 P PS TO GET THE GET THE MOST OUT OF VIDEO! Use video to make research easy and engaging 1. RECRUIT QUALITATIVE WHERE BUDGET Interesting questions, a fair incentive, and exciting subject ALLOWS matter are your starting points, but how do you keep 2. CLEVER SURVEY DESIGN research fresh and snappy? The answer could lie in including 3. ENGAGING VIDEO QUESTIONS a video question at the very start of your survey. 4. QUALITY CONTROL 5. INCENTIVIZE A two-minute video can give you as much data as a 15- 6. SHOW AN EXAMPLE minute radio button survey and even more. You’ll capture 7. CELEBRATE THE RESULTS context, behavior, emotion, engagement, as well as in-depth reviews of products/concepts/routines. For example, if you’re surveying about a product, you’ll see all other products Source: Big Sofa they use if you request a bathroom tour. This saves them having to recall and type out all the things they think they remember (but very often don’t!). Blog: Big Sofa Recruiting for video tasks in market research © 2017 GRBN Participant Engagement Handbook | 118 © 2017 GRBN Participant Engagement Handbook | 119

Tips for Online Surveys Tips for Online Surveys 47 Text comments enable customers to tell you exactly what they feel you need to hear Many companies are almost exclusively listening to happy customers. We think this is a BIG problem. The distribution of satisfied to dissatisfied customers in the results of the average customer satisfaction survey typically follows a very different distribution from a normal distribution. We’ve seen that there is very little variance in structured customer experience data and variance is one of the key things companies should be looking for. Free-form text responses to the right open-ended question can be effective in predicting actual customer behavior because text comments enable customers to tell you exactly what they feel you need to hear. Pro-tip: Don’t just ask the typical NPS follow up: ȊWhy did you give that rating?ȋ, but also follow up with: ȊIs there anything else we can do better?ȋ to get more areas to improve on, even from satisfied customers. Source: Odintext © 2017 GRBN Participant Engagement Handbook | 120 © 2017 GRBN Participant Engagement Handbook | 121

Tips for Online Surveys Tips for Online Surveys 48 49 Break it up – get personal Write like a human… In Tip 44, we encouraged you to start your It’s easy to fall into the habit of writing in Ȋresearch speak,ȋ survey with questions you think the person but doing so makes questions awkward to read and takes up answering the survey might find interesting. In precious screen space on mobile devices. Better phrased longer surveys, use the same technique to maintain their questions can even yield better data! interest and motivation to complete the survey. Challenge yourself to write concisely and as if you were During the survey, ask some personal attitude questions, speaking to someone (because you are). Read your but keep them centered on the topic at hand. For example, questions aloud and ideally to another human. in a banking-related survey, you could ask, ȊWhat worries you most about the future?ȋ or ȊIf you inherited $10,000 what would you do with it?ȋ Pro-tip: Combine with Tip 61 and share back the data on Consider Simpler Wording and Shorter Questions these questions at the end of the survey. • Agree Strongly • Love • Agree Somewhat • Like • Neutral • Neutral • Disagree Somewhat • Dislike ȃShorten questions. Reduce the number of • Disagree Strongly • Hate response options/attributes by For each of the following descriptions How descriptive are conduct/leveraging exploratory analyses. of shopping behaviors, please indicate whether the description is these characteristics Determine which attributes/questions are core highly characteristic, somewhat when you visit a characteristic, slightly characteristic, or not characteristic at all of you membership-only and only include those in trackers and other when you visit a membership-only warehouse club store? warehouse club store. relevant studiesȄ 54 © 2016 Research Now Group, Inc. 48 Pippa Source: Research Now © 2017 GRBN Participant Engagement Handbook | 122 © 2017 GRBN Participant Engagement Handbook | 123

Tips for Online Surveys Tips for Online Surveys ȊConcise and conversationalȋ is something people, especially …by being concise and those under 40 years old, are very keen to see in future conversational surveys. So, go ahead and strike up a conversation next time you write a survey and don’t talk down to your survey-takers. Conversational language is likely to give a better user experience than official-sounding Ȋresearcher-speak.ȋ As people’s attention spans decrease, especially on mobile devices, writing concisely also wins you points with survey- takers. Survey-taker comments: Worried about the impact on your data? In our A/B test, the ȃEasy to complete, nice tone of the survey.Ȅ data suggests there was no difference, even on a highly detailed subject like insurance. ȃEasy to work through and friendly wording.Ȅ ȃI liked the statements at the top of the screen. It was like someone was talking to me through the survey.Ȅ 9 IN 10 SURVEY-TAKERS WOULD LIKE - Source: GRBN Online Survey Experience A/B Testing SURVEYS TO BE MORE CONCISE AND CONVERSATIONAL. Source: GRBN Online Survey Experience A/B Testing © 2017 GRBN Participant Engagement Handbook | 124 © 2017 GRBN Participant Engagement Handbook | 125

Tips for Online Surveys Tips for Online Surveys 50 ƈWe need to treat Make questions easy… participants with the same respect we Too often surveys ask questions that are much treat our friends.Ɖ harder than they need to be. Why ask participants to rate 15 items on a 10-point scale when you only need information about half of those items? Why use a 10-point scale? Do you really think participants have 10 levels of attitude towards your brand of coffee or your banner ad? Don’t ask for more detail than you really need. Also, make the questions conversational, short, and simple. For example, instead of asking this: ȊThink about your role in purchasing food for the family. What percent of the time are you involved in at least 50% of the shopping for groceries?ȋ Why not ask this: ȋWhat percent of grocery shopping do you do for your family?ȋ David Harris, author of The Complete Guide to Writing Questionnaires: How to Get Better Information for Better Decisions © 2017 GRBN Participant Engagement Handbook | 126 © 2017 GRBN Participant Engagement Handbook | 127

Tips for Online Surveys Tips for Online Surveys 56 …and use simple Try using these words and phrases in your next survey: wording to encourage intrinsic motivation Autonomy: Thank you very much for choosing to do this important survey. Please press the > button According to Deci and Ryan’s Self-Determination Theory, when you are ready to continue with the survey. when faced with a task, people’s motivation lies on a continuum from amotivation (not motivated at all), to extrinsic motivation (needing an external reward), to Competence: As one of our most expert survey- intrinsic motivation (wanting to complete the task for its own takers we have selected you to help with this sake). survey. We can move survey participants along the continuum by encouraging feelings of autonomy (you are free to do this or not), competence (you are good at this), relatedness (people Relatedness: People like you all over the country like you are doing this), and value (the task has meaning). are taking part in this survey. Value: Your answers and those of everyone else will help improve the services our client provides to customers like you. Source: SSI Source: SSI © 2017 GRBN Participant Engagement Handbook | 128 © 2017 GRBN Participant Engagement Handbook | 129

Tips for Online Surveys Tips for Online Surveys 51 Don’t make people have to tell you ƈliesƉ Too often we ask people questions they can’t answer. For example, a survey asked physicians the following question: ȊHow many of your patients are following the American Heart Association exercise program?ȋ These physicians don’t know what their patients are doing. Source: Kelsy Saulsbury, Schwan's Shared Services, IIeX NA 2017 Instead, ask physicians a question they can answer: Make sure people can give their opinion or choose not to give an opinion if your question is not relevant to them. ȊOf the last 10 patients you have seen, for how many did you People want to give you their honest opinion, so don't make recommend the American Heart Association exercise program?ȋ them Ȋlieȋ to you. Here is one more example: ȊDo you plan to travel by air in the next 30 days?ȋ o Yes o No FURTHER READING SUGGESTED BY P2SAMPLE: What if people don’t know if they will travel by air in the next 30 days? Their answer choice, ȊDon’t knowȋ is not on the list.  SNAP SURVEYS: Include All Valid Response Options in Survey Questions  SSI: Nine Ways to Avoid Bad Questionnaire Design David Harris, author of The Complete Guide to Writing Questionnaires: How to Get Better Information for Better Decisions © 2017 GRBN Participant Engagement Handbook | 130 © 2017 GRBN Participant Engagement Handbook | 131

Tips for Online Surveys Tips for Online Surveys 52 53 Go off the grid Create surveys that people Avoid grid questions whenever possible. While some mobile actually enjoy… alternatives exist, grids are often designed with PCs in mind. We should aspire to write questionnaires that people can These traditional ȊPC-styleȋ grids (picture radio buttons) add take easily and, dare I say, would enjoy responding to. Even significant time and headaches for participants. They encourage speeding and lack of attentiveness, especially if you are in the regrettable minority who don’t care about when you subject participants to multiple grids. Research our participants’ experiences, you should still care about has shown that data quality drops after the first bad grid. data quality. Better, more mobile friendly surveys mean better data quality. Easy & Enjoyable Surveys = Better Data Quality coorelation with "survey was easy" Correlation ǁith ͞Surǀey ǁas easy͟ coorelation with "survey was enjoyable" Correlation ǁith ͞Surǀey ǁas enjoyable͟ When people enjoyed the 0.585 survey, and found it easier, their on 0.31 i t a answers show l e r more variability, or C on and they are less s -0.002 ar likely to use e -0.081 P midpoint. -0.273 -0.361 Variability Extreme Response Midpoint Response ©2015 Research Now Group, Inc. Source: Research Now Source: Research Now © 2017 GRBN Participant Engagement Handbook | 132 © 2017 GRBN Participant Engagement Handbook | 133

Tips for Online Surveys Tips for Online Surveys 60 …so you don’t bore Stepwise Regression - shows level of boredom with survey is strongest driver of survey experience rating. them to death Model Summary Adjusted Std. Error of Model R R Square R Square the Estimate 1 .616a 0.379 0.354 6.20 Ensure that the research survey is fun and engaging. a. Predictors: (Constant), Total Boredom Always remember that your survey is competing for a Coefficients attention with social media such as Facebook and Instagram, Unstandardized Standardized Coefficients Coefficients Correlations and with games like Candy Crush, among other things. Model B Std. Error Beta t Sig. Zero-order 1 (Constant) 99.404 3.257 30.522 0.000 Boredom with an online survey is the strongest driver of a Total Boredom -5.326 1.364 -0.616 -3.906 0.001 -0.616 negative survey experience. A bored participant is less likely a. Dependent Variable: Weighted Survey Experience to give thoughtful survey answers and, eventually, might opt out of your survey or out of taking research surveys Source: RTi Research metrics data altogether. Consider these elements to create a successful survey: survey length, participant engagement via gamification, and survey logic/relevance of questions to the audience. Cut out as much as possible to streamline! If you must be THE HIGHEST DIARY SURVEY RATING repetitive, warn your participants so it will be easier for WE HAD WAS FOR THE QUESTIONNAIRE them to repeat themselves! THAT WAS THE LEAST REPETITIVE. Source: MESH Experience metrics data © 2017 GRBN Participant Engagement Handbook | 134 © 2017 GRBN Participant Engagement Handbook | 135

Tips for Online Surveys Tips for Online Surveys 54 Gamify your surveys with RESEARCH SHOWS THAT: words for more engagement 4. GAMIFYING SPONTANEOUS AWARENESS BY ADDING and better results A TIME ELEMENT AND A MEANS TO ƈWINƉ INCREASED AVERAGE BRAND MENTIONS BY 50%. THIS RESULT WAS SEEN ACROSS COUNTRIES AND Researchers sometimes fear that gamifying surveys is too CULTURES. complex and time-consuming, or will not improve results. 5. GAMIFYING OPEN-ENDED QUESTIONS BY ADDING A Simple gamification elements, with clear Ȋrules,ȋ and a stated Ȋway to winȋ result in richer, deeper answers across a SCENARIO AND ASKING PARTICIPANTS TO TELL US broad range of question types and research topics. ƈTHREE THINGSƉ INCREASED THE NUMBER OF CODES PER PERSON FROM 1.5 TO 2.6 AND THE NUMBER OF PEOPLE GIVING THREE ANSWERS ROSE FROM 5% TO 47%. 6. ADDING A GAME SCENARIO TO AN OPEN-ENDED QUESTION THAT ALREADY ASKED FOR ƈTHREE THINGSƉ INCREASED THE NUMBER OF PARTICIPANTS OFFERING THREE FROM 71% TO 81%. Source: SSI To find out more, see the work of SSI’s Global Knowledge Director, Pete Cape here. Source: SSI © 2017 GRBN Participant Engagement Handbook | 136 © 2017 GRBN Participant Engagement Handbook | 137

Tips for Online Surveys Tips for Online Surveys 55 56 Break it up – have some fun Use motivational language Many people love a quiz! In longer surveys, we In longer surveys or surveys where you are asking them to recommend breaking the survey up by asking do difficult or repetitive tasks, show empathy and give them quiz-style questions on topics that interest the encouragement to continue with the survey. participant (asked at the beginning of the survey using Our A/B testing data shows that survey-takers really Tip 44). appreciate this kind of thoughtfulness and respect for the In our A/B test, we gave people the opportunity to answer effort they are putting into the task. questions on cars, films and celebrities, travel and sports, and shared the data back with them. The people presented with these types of questions to break up the survey spent on average an extra four minutes answering the survey and Survey-taker comments: were just as satisfied with the experience for the same incentive as the other participants were paid. ȃYour encouraging words in the questions were Other types of questions you could ask to break up a long, entertaining.Ȅ boring survey are: ȊIf you won $100,000 in a lottery what ȃThe survey was user friendly, encouraging the would you do with it?ȋ Or ȊWhat new skill would you most participant throughout the survey.Ȅ like to learn?ȋ Have fun coming up with some good quiz questions and your survey-takers will have fun answering ȃDidn't make me choose from too many them. answers. Like the funny comments added in Pro-tip: Combine with Tip 61 and share back the data on too.Ȅ these questions either right after they answer or at the end - Source: GRBN Online Survey Experience A/B Testing of the survey. © 2017 GRBN Participant Engagement Handbook | 138 © 2017 GRBN Participant Engagement Handbook | 139

Tips for Online Surveys Tips for Online Surveys 57 58 Look and feel matter Convenience is key Which of these experiences would you rather look at for 10- Time is precious and participants don’t want to spend time 15 minutes? Which one is easier on your eyes? Which one accessing links or figuring out how to follow instructions. potentially fits better with the rest of your brand’s marketing Making everything as easy and intuitive as possible will keep and customer engagement efforts? Which one may hold everyone as happy as possible. your attention long enough to stop playing a video game, For example: can participants actively do anything to help binge-watching a show, or scrolling social media? the process go smoothly? Especially if they are part of an Use survey tools which allow for a beautiful UX and your ongoing study, shortcuts or quick access to instructions can participants will thank you. really make a difference. Remember when testing that familiar processes for you won’t be second nature to your participants, so spell everything out clearly! THE ONLY SURVEYS THAT WENT UP IN RATING ACROSS THE THREE STAGES HAD CLEARER INSTRUCTIONS FOR HOW TO SET UP SHORTCUTS TO PARTICIPATE IN THE STAGES VIA MOBILE. Source: MESH Experience metrics data Source: Research For Good Source: Mesh Experience © 2017 GRBN Participant Engagement Handbook | 140 © 2017 GRBN Participant Engagement Handbook | 141

Tips for Online Surveys Tips for Online Surveys 59 60 Respect people’s privacy Avoid assumptions Many people are very concerned about the privacy of their You might be a master questionnaire designer, but don’t data. Their first instinct is to not trust you, so you need to assume you know it all or that you have all the bases covered earn that trust. Understand what people consider to be in your questionnaire. Always give people the possibility to sensitive personal data and respect their right to privacy. give their spontaneous comments at the end of the survey. Only ask people for demographic and other profiling Not only does this show that you truly care about what they questions if they are relevant to your study, not Ȋjust in caseȋ, and explain why you need the data. Transparency is think, you might also be surprised by the additional insights key. Whilst participants have legal protection, transparency you get. is about going beyond this to secure trust. FEWER THAN 1 IN 10 PEOPLE HAVE A HIGH LEVEL OF TRUST IN MARKET RESEARCH COMPANIES TO PROTECT AND APPROPRIATELY USE THEIR PERSONAL DATA. LESS THAN HALF OF SURVEY-TAKERS FEEL WELL- INFORMED ABOUT HOW RESEARCH COMPANIES COLLECT, STORE, AND USE PERSONAL DATA. Source: Kelsy Saulsbury, Schwan's Shared Services, IIeX NA 2017 Source: GRBN Trust Survey 2016 © 2017 GRBN Participant Engagement Handbook | 142 © 2017 GRBN Participant Engagement Handbook | 143

Tips for Online Surveys Tips for Online Surveys 61 “People who live in your part of the country think…or Share back 18-24-year olds like you say…“ Your survey-takers will appreciate the effort to personalize the experience for them. If you’ve followed Tip 44 and Tip 55 you will have included While you are waiting to get started you can use the share- questions in the survey you think participants find back data we have collected on these general interest interesting. At the end of the survey, share back how other questions. people have answered these questions (in aggregate form). Our A/B testing shows that you can add an extra two to three minutes to your survey in this way without any additional incentive and increase satisfaction with the survey experience. Answers to attitudinal questions are good for sharing since these are seldom confidential and many survey-takers are inquisitive by nature. Participants want to share their own opinions and hear what others think about certain topics. We recommend you create a battery of such questions related to your sector/category and add them to your next survey. Once you have the data, create some share back charts. You can then sprinkle these questions and the associated share-back charts into your future surveys, and don’t have to worry about dealing with live data. Expert tip: Create charts for different demographic groups and filter the data-shares accordingly, saying for example: © 2017 GRBN Participant Engagement Handbook | 144 © 2017 GRBN Participant Engagement Handbook | 145

Tips for Online Surveys Tips for Online Surveys 62 Say ƈthank you!Ɖ Survey-taker comments: ȃIt was interesting and As simple as that. I liked being able to see the comparison of others in my age bracket.Ȅ ȃOffers information of what others are concerned about too, helps you feel not alone or overwhelmed.Ȅ - Source: GRBN Online Survey Experience A/B Testing © 2017 GRBN Participant Engagement Handbook | 146 © 2017 GRBN Participant Engagement Handbook | 147

Tips for Online Surveys Tips for Online Surveys 62 71 …by telling them who …and inspire them with a sponsored the survey… video for maximum effect In the survey invite, promise to tell participants who the Get your client to make a short video explaining how sponsor is at the end of the survey (see Tip 36). people’s participation in the research will not only help the At the end, also thank them for their participation on behalf client but will help other people (citizens or consumers), but of the client. This will not only motivate them to participate make sure that the message is not promotional or marketing in future surveys but will also create a positive impression in nature. on the brand or company sponsoring the survey. The video doesn’t have to be professionally made. A Ȋhomemadeȋ video can even increase authenticity and empathy, which are useful when building a relationship. Include the video at the end of the survey to avoid bias, but promise it upfront as a key selling point in the invite. - Source: GRBN Online Survey Experience A/B Testing - Source: GRBN Online Survey Experience A/B Testing © 2017 GRBN Participant Engagement Handbook | 148 © 2017 GRBN Participant Engagement Handbook | 149

Tips for Online Surveys Tips for Online Surveys Survey-taker comments: ȃI truly appreciate being thanked for participating. That is a huge incentive to complete more for MetLife in the future. I now have a greater appreciation for this company as well.Ȅ ȃI thought it was just like all other surveys until I watched the video and found out exactly what they are trying to accomplish.Ȅ Source: GRBN Online Survey Experience A/B Testing ȃI also like the personal touch with the video at the end. I like knowing how my participation may help a company.Ȅ This chart shows that 49% of participants who saw the ȃI like that there was a video of an actual Ȋthank youȋ video said that the survey experience improved the image they have of the company sponsoring the survey. person telling the survey-takers that the surveys This compares with 29% giving the same answer in the have actually been useful to a real company and control cell, demonstrating that the Ȋthank youȋ video from my participation may help a company.Ȅ the client has a significant impact on the experience. - Source: GRBN Online Survey Experience A/B Testing © 2017 GRBN Participant Engagement Handbook | 150 © 2017 GRBN Participant Engagement Handbook | 151

Tips for Online Surveys Tips for Online Surveys 63 Ask for and act on feedback ȃ… in online communications, it's important to communicate what will happen (ȁby sharing your opinion you Don’t bury your head in the sand. Ask those taking your will help us to...Ȃ) and also what won't happen surveys for their honest feedback on your survey. How good (ȁby doing this your likelihood to take part in was it? How could it be better? new projects will remain the same...Ȃ). Nothing Make sure you act on the feedback too. By simply asking for can be taken for granted in self-administered feedback you will raise people’s expectations, so follow- tools.Ȅ through and improve your surveys based on the feedback you get. - Oriol Llauradó, Panel Care Director Don’t stop there; share what you learned with your and CPO, Netquest colleagues, your data collectors, and your clients (Tip 24). Blog Baillie Buchanan, Research For Good © 2017 GRBN Participant Engagement Handbook | 152 © 2017 GRBN Participant Engagement Handbook | 153

Tips for Online Surveys Tips for Online Surveys 64 Analyze ƈdrop outsƉ Keep close attention to the drop out percent; these are a direct indication of the quality of the survey and the responses. Request drop out information from panel companies/suppliers. These data can provide valuable information that can be used to improve future surveys. Analyze drop out percent per question post completion of a project to identify if the drop out was driven by one or several questions. Learn from this for future projects while taking survey satisfaction ratings and open-ended feedback into account. Incorrect survey setup can introduce bias that can require exclusion of certain questions in the final data. FURTHER READING SUGGESTED BY P2SAMPLE:  FLUID SURVEYS: Response Rate Statistics for Online Surveys - What Numbers Should You be Aiming For?  NEOACADEMIC: Predicting Dropout Rates for Students Completing Online Surveys  SURVEYMETHODS: Why is Dropout Rate a Problem? Source: P2Sample © 2017 GRBN Participant Engagement Handbook | 154 © 2017 GRBN Participant Engagement Handbook | 155

Tips for Online Surveys Tips for Online Surveys 65 L Control the quality The danger in quality control checks that are too strict is that they exclude not only poor performing participants but many good performers as well. This not only impacts ESS THA feasibility but could result in a sample of people who are unusually attentive and not representative of the normal range of human attentiveness. Source: SSI The problem with so many attention-checking questions is that they do not measure inattention at all! In addition, even a single attention slip (a normal human error) might not be evidence of a general lack of attention. THE RECOMMENDATION IS TO USE FIVE OR MORE OF THESE CHECKS: SUGGESTION: - MULTIPLE LOW-INCIDENCE ITEMS Use at least five simple quality-control checks and - OPEN-ENDED ANSWER QUALITY (IF AVAILABLE AND TIME ALLOWS) only remove people if they fail more than one. - CONFLICTING ANSWERS IN A SHORT GRID - SPEEDER CHECK SSI tested 15 commonly used quality control checks and - DIRECT INSTRUCTION (E.G., CHOOSE ANSWER 5) IN A SHORT recommends only removing people if they fail more than GRID one. Some checks fail no one, others fail over 60%, so - FACT CHECK (VALIDATE A FACT ASKED IN THE BEGINNING OF they cannot be measuring the same thing. THE QUESTIONNAIRE VS. THE END) See the article AND REMOVE ONLY THOSE PEOPLE WHO FAIL TWO OR MORE. Source: SSI Source: SSI © 2017 GRBN Participant Engagement Handbook | 156 © 2017 GRBN Participant Engagement Handbook | 157

Tips for Passive Metering Tips for Passive Metering Table of contents Customers’ trust means better data ............................... 160 Passive 66. Don’t lose touch with consumers ........................... 162 67. To maximize conversions, use ȊVIPȋ messaging and set expectations ............................................... 164 Metering 68. Integrate data sources to increase satisfaction rates ......................................................................... 165 69. For cross-device metering, focus on registering all devices on day 1 ................................................. 166 Tips 70. Easy installation flow ............................................... 167 71. Periodic surveys improve retention for passive measurement panels ............................................... 168 72. For projectable audience measurement, active panel management is key ....................................... 170 Passive participation 73. Passive should be passive ....................................... 171 requires great UX too 74. Keep confidential information confidential ........... 172 75. Gain trust with transparency .................................. 175 76. Give back control to the participant ....................... 176 77. For large panels, automation and re-engagement campaigns are critical ............................................. 177 78. Easy deinstallation flow .......................................... 178 © 2017 GRBN Participant Engagement Handbook | 159 © 2017 GRBN Participant Engagement Handbook | 159

Tips for Passive Metering Tips for Passive Metering Customers’ trust gain. People we serve as customers need to be informed, means better data empowered, and capable of making choices. These principles especially apply to the collection of behavioral data. Certainly, behavioral data is a wonderful and amazing addition to our lives. Nevertheless, it is obvious Citizens are the best source of rich and meaningful data. But that when people generate thousands of data points every questionable data practices and zero-sum business models day they become more vulnerable to exposure in have been eroding consumers' trust. To protect themselves unimaginable ways. from such practices, citizens are likely to not provide data or We firmly believe that our responsibility as market research provide fake data. professionals is to always act according to the highest In much of the world, it’s become a widely held view that ethical standards. For decades, this has been the promise of models rely on Ȋpeople as an assetȋ—it cannot continue. our industry to society and it must be maintained in the future. In doing this, we will be building a more valuable, Regulation won’t allow for it. People won’t stand for it. In trustable, and sustainable data industry. 2017, privacy and transparency are not only compliance requirements, they are pre-requisites for every organization, and in 2018, these requirements will become even tougher as the General Data Protection Regulation is introduced in the EU. We urge our industry to turn from a data-centered to a human-centered approach. If we think about it, it is just a matter of getting back to basics. A more balanced data environment with more visibility and openness about data flows can contribute to the increase of participants' trust. In this chapter, we will describe a set of practices that put the user in the center of their own data instead of the Anna Hebbeln Oriol Llauradó current internet business model, where companies are in full control and make the most out of the data for their own © 2017 GRBN Participant Engagement Handbook | 160 © 2017 GRBN Participant Engagement Handbook | 161

Tips for Passive Metering Tips for Passive Metering 66 Don’t lose touch with consumers ȃI think when you answer a question in relation to certain topics/companies, the follow-up questions should only be about the While trying to understand today’s consumer behaviors, companies you say you have used. For example, data collection has mostly continued with traditional communication questionnaires. You say you use methods. Participants typically complete long, generic [one company] and then you are asked to rate surveys, resulting in a bad research experience, Ȋbad dataȋ, another company or more that you have never and a decline in response rates. used...very frustrating!Ȅ We need to remember that participants are actually people, ȃSometimes an ȁotherȂ option is necessary but who are essential to market research. We must respect and appreciate the efforts of those people whose opinions we not available. Makes me feel like I'm not being claim to value. completely honest.Ȅ Improving their experience in terms of usability, privacy, and ȃThe rewards should be based on percentage of incentives will lead to more valuable data. completion. It is frustrating to spend 15 minutes [on a survey] and just because the very last question the answer was not the expected one I get just 20 cents.Ȅ - Participant comments from a study conducted by Wakoopa & TEG: Rewards on the participant experience, April 2017 Blog by Bernou Benne, Wakoopa How to stay relevant & NOT alienate people: The benefits and challenges of passive metering and surveying – Part I Source: Wakoopa © 2017 GRBN Participant Engagement Handbook | 162 © 2017 GRBN Participant Engagement Handbook | 163

Tips for Passive Metering Tips for Passive Metering 67 68 To maximize conversions, Integrate data sources to use ƈVIPƉ messaging and increase satisfaction rates set expectations The only way to extract all possible value from each data Operating passive measurement panels is an art and a source and to provide the best research experience is to science. Prospective panelists need to be motivated to combine data sources. To do this, we must use the right register, become compliant, and remain active and valid. tools for the right job. Beyond incentives, people are increasingly interested in For example, passive metering allows us to ask Ȋwhatȋ taking part in a bigger initiative. Ensure panelists people do online, whereas surveys only ask Ȋwhyȋ people are understand they are an elite community that helps some of doing it. the world’s biggest companies shape apps, sites, and Passive metering lets us ask only relevant questions and devices. eliminates the use of redundant surveys, which leads to a higher research satisfaction rate. Blog: How behavioral data helps to better recruit research participants ȊTake for instance the example of people interested in buying cars: with behavioral data research participants who have visited an automotive website can be selected directly. Recent research has indicated better targeting increases participation rates, reduces dropout rates, improves survey experience, and yields the same or better survey data quality.ȋ Source: Verto Analytics Source: Netquest & Wakoopa © 2017 GRBN Participant Engagement Handbook | 164 © 2017 GRBN Participant Engagement Handbook | 165

Tips for Passive Metering Tips for Passive Metering 69 70 For cross-device metering, Easy installation flow focus on registering all Participation in research should be made as easy devices on day 1 as possible. One of the goals of passive metering technology is to achieve the easiest possible If your passive measurement panel is Ȋcross-deviceȋ or installation process for the user. Ȋcross-platformȋ, use the initial attention of the panelist to Eventually, there should be a one- your benefit. Help panelists get all meters, for all devices, click installation on all of a installed on the day that the panelist signs up. In these initial participant’s devices. moments, panelists are more likely to recall the benefits and The installation should be purpose of the panel and have the motivation to complete tailored toward the participant’s the installation process. ease of use by leaving out unnecessary technical details. There should also always be a singular installation process. When activated, it should run continuously until it is ended by the participant. Source: Wakoopa Source: Verto Analytics © 2017 GRBN Participant Engagement Handbook | 166 © 2017 GRBN Participant Engagement Handbook | 167

Tips for Passive Metering Tips for Passive Metering 71 Periodic surveys improve retention for passive measurement panels Passive measurement panels need not be passive in all respects. In fact, panelists need incentives and triggers to stay compliant and active. For example, if they choose to replace their smartphone or PC, they may forget to reinstall the metering app. Periodic surveys to check in with panelists increases retention and the likelihood that panelists will keep their devices and meters active. Source: Verto Analytics © 2017 GRBN Participant Engagement Handbook | 168 © 2017 GRBN Participant Engagement Handbook | 169

Tips for Passive Metering Tips for Passive Metering 72 73 For projectable audience Passive should measurement, active panel be passive management is key When executing research using passive metering, it is Building a projectable panel that is representative and important to keep in mind that Ȋpassiveȋ actually means no statistically controlled is tough. It is easier to build active participation from the research participant. convenience samples that any panelist can join versus To provide users with a good research experience during all statistical samples where the acceptance criteria is clearly phases of the research, try to request as little active defined. For a panel to be projectable, apply weighting and involvement from the participants as possible other than the quotas to each incoming panelist, and segment so your initial installation of the tracker. For example, avoid panel is representative of the consumers you wish to irrelevant popups and ensure the tracker has no impact on measure. the device performance or browsing experience. ȃBy using quotas and calculating for each THE LESS PEOPLE ARE REMINDED THEY ARE incoming panelist, for each segment, the PASSIVELY MEASURED, THE MORE LIKELY THEY ARE likelihood for him/her to still be active in a TO FORGET THEY ARE BEING TRACKED. monthȂs time (and be part of the valid pool of 25% OF PARTICIPANTS INDICATED THEY FORGOT panelists) helps to ensure your panel represents THAT THEY HAD INSTALLED THE TRACKER. the cross-device online universe.Ȅ - Dr. Hannu Verkasalo, Verto Analytics Source: Study conducted by Wakoopa & TEG: Rewards on the participant experience, April 2017 Read more here Source: Wakoopa Source: Verto Analytics © 2017 GRBN Participant Engagement Handbook | 170 © 2017 GRBN Participant Engagement Handbook | 171

Tips for Passive Metering Tips for Passive Metering 74 Keep confidential information confidential Consumers create millions of moments on their devices. This data can contain direct (and indirect) identifiers and personal information. Thus, data collected via passive Blog by Ezequiel Paura, Netquest: metering should be considered as personally identifiable Keeping Confidential Data Confidential But Useful information (PII). Once you have an individual's confidential data, it is important to keep it that way. We recommend that the data be cleaned of any PII to prevent identification. It’s a trade-off between the risk of ȃA simple eye inspection of a clickstream of identification and the amount of information preserved. It’s about finding a good balance between data protection and behavioral data makes it possible to identify data utility. When in doubt, always favor the user’s privacy. 85% of the users in the dataset.Ȅ – Study conducted by Wakoopa, March 2016 "Data can be either useful or perfectly anonymous but never both." – Ohm 2010 Source: Wakoopa © 2017 GRBN Participant Engagement Handbook | 172 © 2017 GRBN Participant Engagement Handbook | 173

Tips for Passive Metering Tips for Passive Metering 75 ƈOne day you’ll just be a Gain trust memory for some people. with transparency Do your best to make it a good one.Ɖ Passive metering experiences need to be trustworthy and transparent regarding the use of participants’ data. It is important to have participants’ free, informed, and explicit consent to handle their personal information. Moreover, be clear on your purpose and be specific on how you are planning to use their data and for what. Additionally, your privacy policy should be comprehensive, but clearly understandable as well as easily available for the user. Although these rules will apply for any research study, the importance increases when the perceived risk increases. Blog by Oriol Llauradó, Netquest Collecting Behavioral Data With Ethics ȊThe more trust people have, the more information they are willing to share.ȋ ȊDo not do to participants what you would not want done to yourself.ȋ Source: Wakoopa © 2017 GRBN Participant Engagement Handbook | 174 © 2017 GRBN Participant Engagement Handbook | 175

Tips for Passive Metering Tips for Passive Metering 76 77 Give back control For large panels, to the participant automation and Another part of building a privacy program that ensures the re-engagement campaigns best user experience is to empower research participants by are critical giving them effective mechanisms to exercise their right to data protection. For bigger panels, it’s often cheaper to retain existing They are the owners, so they should determine how their panelists than to recruit new ones. Because it’s not cost- data will be used. The following measures can help to effective to maintain a large support team, instead build enhance the control that participants have over automated alerts that remind panelists to take action, flag their data: meter issues, and provide incentive notifications. Track incoming requests to identify query patterns that can be tackled pro-actively with regular updates. Blog by Ezequiel Paura, Netquest Striving for Quality: A Privacy-Compliant Model - White list: Provide the user with a tool to list specific sites they want to browse privately. These sites won’t be recorded by the meter. - Pause function: Give the user the possibility to pause and resume the meter whenever she/he wants. - Ex-post removal function: Provide the user with an opportunity to browse through the data and remove selected data before sending it. Source: Wakoopa Source: Wakoopa © 2017 GRBN Participant Engagement Handbook | 176 © 2017 GRBN Participant Engagement Handbook | 177

Tips for Passive Metering Tips for Passive Metering 78 Easy deinstallation flow Refer back to the installation requirements mentioned earlier in this chapter, the same applies for the deinstallation process of the tracker. An easy uninstall process means that the user should be able, in no more than a few clicks, to fully uninstall the meter. They should also be confident in the fact that the meter is fully removed and no longer tracking their behavior. ȃOne day, youȂll just be a memory for some people. Do your best to make it a good one.Ȅ Source: Wakoopa © 2017 GRBN Participant Engagement Handbook | 178 © 2017 GRBN Participant Engagement Handbook | 179

Tips for Qualitative Research Tips for Qualitative Research Table of contents How to motivate people to give you their time ..................... 182 79. Be advisors to your clients/look at your suppliers as experts in their roles ................................................. 186 Qualitative 80. Use targeted blasts/pre-qualifications .......................... 188 81. ȊSellȋ the engaging components of participation .......... 190 82. Refresh terminology ....................................................... 192 83. Be transparent ............................................................... 194 Research 84. Be clear on what to expect ............................................ 196 85. Explain the impact .......................................................... 198 86. Reduce screener length ................................................. 200 87. Be flexible when you rescreen ....................................... 202 88. End the games ................................................................ 204 Tips 89. Update contact methods ................................................ 206 90. Continue the transparency ............................................ 207 91. Use a friendly moderator ............................................... 208 92. Allow time for icebreakers ............................................. 210 93. Enforce the impact ......................................................... 212 94. Use group tasks/collaborative activities whenever How to motivate people possible .......................................................................... 214 95. Consider fewer Ȋblindedȋ studies .................................. 216 to give you their time 96. Consider in-situ .............................................................. 217 97. Respect the time commitment ....................................... 218 98. Thank them ..................................................................... 219 99. Continue the feedback loop ........................................... 220 100. Share the impact ............................................................ 221 101. Surprise and delight with bonuses ............................................. 223 © 2017 GRBN Participant Engagement Handbook | 181 © 2017 GRBN Participant Engagement Handbook | 181

Tips for Qualitative Research Tips for Qualitative Research How to motivate people With the help of the below partners, we undertook an online community in the US and offline group discussions in both to give you their time the US and the UK. Whilst a lot of research-on-research has been conducted on the online survey user experience, there has been very little research done on the experience participants have with qualitative research. As part of the Participant Engagement Initiative, we decided that we needed to do more research into this issue. The research was led by Kerry Hecht, Katrina Noelle, and Nikki Lavoie, who designed a project to meet the following research objectives: 1. To understand intrinsic motivations people have for Overall takeaways from the research-on-research participating in qualitative research 2. To understand how a good or bad user experience can The research uncovered that money is the lowest common affect the company/brand behind the survey, and how denominator and easiest motivator. However, giving it can be improved before, during, and after the feedback to a brand that interests participants and process participating collaboratively with others is a key secondary appeal factor (e.g., the intrinsic value of giving them time, 3. To understand what drives a good or bad user learning, and collaborating). experience, and how the experience can be improved We found out that professional/repeat participants before, during, and after the process participate even more frequently than we knew. This calls for 4. To test how changing the user experience/end-client an increase in using behavioral verifications at the point of interaction can increase motivation screening to ensure quality regardless of past participation. © 2017 GRBN Participant Engagement Handbook | 182 © 2017 GRBN Participant Engagement Handbook | 183

Tips for Qualitative Research Tips for Qualitative Research While repeat participants are engaged, one goal for the team Overall, we hope to provide clients and agencies the chance was to recommend practices to engage all participants, not to look in the mirror and acknowledge our practices that feed just repeat ones (this includes those who sign up for panels this engagement problem. The tips communicate how to and are then not called/invited to studies). It is clear improve engagement given these overall takeaways on the retention and engagement initiatives should start at the state of the industry. moment a potential participant joins a panel. To that end, we have built a set of recommendations to both clients and agencies to improve systemic practices across the industry to keep the most valuable asset to the industry, our participants engaged. Kerry Hecht Labsuirs Katrina Noelle FOUR KEY THEMES FROM THE RESEARCH 1. ORGANIZATION AND TRANSPARENCY MATTERS TO THE PARTICIPANT; THEY FEEL WHEN IT’S LACKING. 2. SETTING EXPECTATIONS ARE CRUCIAL (AND LIVING UP TO THEM!). 3. RESPECT IS KEY; RESPECT PARTICIPANTS’ ROLE IN Nikki Lavoie THE PROCESS, THE WORTH OF THEIR TIME, ETC. 4. PARTICIPANTS ARE SAVVY TO THE RULES OF OUR GAME, ESPECIALLY EXPERIENCED ONES, SO MANY OF OUR TIPS REFLECT ADVICE TO (1) REDUCE ISSUES, AND (2) MANAGE EXPECTATIONS. © 2017 GRBN Participant Engagement Handbook | 184 © 2017 GRBN Participant Engagement Handbook | 185

Tips for Qualitative Research Tips for Qualitative Research 79 Be advisors to your clients/ ȃHow does participant/sample quality figure into what look at your suppliers as results we trust?Ȅ experts in their roles ȃCan we really claim that quality sample is important if we donȂt see the participant experience as important?Ȅ Client-side researchers look to MR agencies for expertise, but MR agencies often do not look to their suppliers ȃItȂs concerning that supplier side researchers arenȂt (recruiting houses, panel companies, etc.) for their interested in what their own suppliers see as best expertise. This must change. practices.Ȅ The database owners are the ones closest to the ȃHow can we expect database owners to maintain participants. They have insight, such as the longer the healthy databases if we arenȂt concerned with the screener the more likely you’ll get professional participants participant experience?Ȅ or participants who lie. A good supplier you treat like a partner can tell you, in advance, where you will have - 2017 Q1-2 GRIT Report problems and help you find solutions. They should be consulted with as early as possible when the specs and screener are being developed. Source: Echo Qualitative Project Support, KNow Research, MindSpark Research International © 2017 GRBN Participant Engagement Handbook | 186 © 2017 GRBN Participant Engagement Handbook | 187

Tips for Qualitative Research Tips for Qualitative Research 80 ȃIt would be easier to get interviewed through email Use targeted blasts/ or text.Ȅ pre-qualifications ȃThe screening process can sometimes be tiresome...Of course, for bigger projects, it is important to get the Consider Ȋblastingȋ invitations to fewer people during the right person...I think sometimes the questions that are recruitment process to stop overexposure to studies people asked in the screening process are too black and white may or may not qualify for. Suppliers have data that allows and rigid.Ȅ for more appropriate targeting. Consider sending a Ȋblastȋ with a short pre-qualification - 2017 GRBN Qualitative Workgroup ROR screener that touches on the high points without taking them through the entire screener. This respects participants’ time, and the more information they see about studies they don’t qualify for might motivate them to lie in order to qualify. On the other hand, if they do qualify for a lot and are constantly not invited or screened out, participants start to feel disengaged or worse. In this process, try to screen online as much as possible. While a phone call is often very important, it should be brief and focused on a conversation to make sure the participant is right for the project and engaged. Source: Echo Qualitative Project Support, KNow Research, MindSpark Research International © 2017 GRBN Participant Engagement Handbook | 188 © 2017 GRBN Participant Engagement Handbook | 189

Tips for Qualitative Research Tips for Qualitative Research 81 ƈSellƉ the engaging ȃTurned out that focus groups give me a chance to components of participation think creatively and voice my opinions and feel like an insider (kind of).Ȅ While money is a big motivating factor, we do very little to ȃI like being on a team - fun to work together to promote the environment or the opportunities for accomplish something.Ȅ activity/collaboration as a result of participating in a study. Often screener language and invitation language is very dry, - Participant Experience Online Qualitative Community, if not almost non-existent. That needs to change. We can't Jessica Broome & Kerry Hecht, May 2016 expect to attract great people to participate in MR if we aren't promoting the benefits to them and making sure their ȃInform potential participants of the fun and ease of the experience is excellent. Highlight some of the Ȋfunȋ aspects proc ess, the comfortable and welcoming settings, and of the research people can expect. the incentives…Along the lines of: ȂWould you like to Caveat: Please note the more we promote things like help the client improve their customer service by sharing collaboration, the more we might risk isolating people who your experiences and opinions, debating with other aren’t extroverts. This is worth keeping in mind and/or customers, and creating new ideas? Be our VIP for the mentioning for the sake of showing we understand the day, and leave the hard work to us!ȂȄ implications on the data as a whole. - 2017 GRBN Qualitative Workgroup ROR Source: Echo Qualitative Project Support, KNow Research, MindSpark Research International © 2017 GRBN Participant Engagement Handbook | 190 © 2017 GRBN Participant Engagement Handbook | 191

Tips for Qualitative Research Tips for Qualitative Research 82 Refresh terminology ȃItȂs like a little workshop. Brainstorming, type of thing.Ȅ Consider using alternative descriptions for research ȃA structured brainstorming session. A collaborative activities (e.g., brainstorming sessions), as this will increase discussion.Ȅ transparency by better reflecting what you are asking people to participate in. Many people mentioned they have other experiences in their work/life/community that involve giving - 2017 GRBN Qualitative Workgroup ROR feedback, coming up with solutions to problems, and generating ideas. They would be just as comfortable participating in this type of session compared to a Ȋfocus groupȋ so consider using different terminology. Side benefit: You may get fewer repeat participants using this approach since they are primed to look for Ȋfocus groupȋ offers. Source: Echo Qualitative Project Support, KNow Research, Source: Echo Qualitative Project Support, KNow Research, MindSpark Research International MindSpark Research International © 2017 GRBN Participant Engagement Handbook | 192 © 2017 GRBN Participant Engagement Handbook | 193

Tips for Qualitative Research Tips for Qualitative Research 83 Be transparent ȃI dislike the process of the screening. You never know what a research company is looking for demographic- Participants are very articulate about feeling as though we, wise. I like to participate in them and it is unfair when on the research side, are not honest or transparent about you can get disqualified for being a different race or what we're asking of them. This is the foundation of our age. It should be able to accept a wide range of people relationship with them and goes a long way to setting the instead of only looking for certain demographics.Ȅ stage for a dysfunctional one. Be upfront about how long the screener will be, what the ȃThey left out a portion of what the study entailed methodology is, how long the project will take, what will be over the phone and told me about once I got there. Left, expected of them, and what they will get out of it. They wasn't worth my time.Ȅ appreciate this and it's nothing that can't be built into the - Participant Experience Online Qualitative Community, screener. Jessic a Broome & Kerry Hecht, May 2016 Clients must give direction for suppliers to be as proactive and clear to the participant as possible about the logistics and what to expect. Be sure to share key details multiple times! Source: Echo Qualitative Project Support, KNow Research, MindSpark Research International © 2017 GRBN Participant Engagement Handbook | 194 © 2017 GRBN Participant Engagement Handbook | 195

Tips for Qualitative Research Tips for Qualitative Research 84 ƈNobody deserves a poor Be clear on what to expect research user experience, not even your Be very proactive and clear to the participant about the mother-in-law.Ɖ logistics and what to expect. Share key details multiple times. Ensure that participants feel they are prepared and capable of speaking on the subject you want to learn about. In some cases, it can be beneficial to get them to study up on their own behavior via homework. This can be easier to do when we are clearer about the subject of the research and our objectives. In this way, we are helping them help us! ȃBesides the actual study, I thought the scheduling, reminders, and location info was great and accurate. Thanks for being on top of that!Ȅ ȃI wasnȂt aware that we were going to a house really - it was mentioned once in correspondence, but it made me a bit nervous at first.Ȅ - Quotes from participants in research conducted by Doing Good Research Source: Echo Qualitative Project Support, KNow Research, MindSpark Research International © 2017 GRBN Participant Engagement Handbook | 196 © 2017 GRBN Participant Engagement Handbook | 197

Tips for Qualitative Research Tips for Qualitative Research 85 Explain the impact ȃI wish there were feedback when I am not selected. I can speculate why they don't give feedback, but that Teach participants the value of market research and explain wouldn't accomplish anything.Ȅ the impact it has when they are not honest. In a perfect world, we wouldn't have to do this, but we have created a ȃNo one knows what people are looking for (especially very imperfect world. Because we currently aren't as the level of knowledge of a product) so they likely transparent as we could be, participants start to guess about exaggerate in the hopes of being picked.Ȅ how the research is being used and who is behind it. Because they are so distanced from the impact, they ȃSometimes people lie and they don't think about how fundamentally don't see it as a problem to lie in the it may affect the researchers.Ȅ screening process. We need to tell people why their participation matters and - Participant Experience Online Qualitative Community, help them to understand the impact it has on the client Jessica Broome & Kerry Hecht, May 2016 commissioning the research. This feeds into some of the intrinsic motivations they have for participating. What they ȃIt would be really nice to see where this research goes! I are doing is important and we should remind them of this would love to hear back on what you used this for and throughout the research process. Once we help them to where I can expect it to make an impact.Ȅ understand why their role is so important, we can ask them to give us that much more of themselves. - 2017 GRBN Qualitative Workgroup ROR Source: Echo Qualitative Project Support, KNow Research, MindSpark Research International © 2017 GRBN Participant Engagement Handbook | 198 © 2017 GRBN Participant Engagement Handbook | 199

Tips for Qualitative Research Tips for Qualitative Research 86 If a long screener cannot be avoided, alert participants to that fact before they dive in and explain why it's so long. Reduce screener length Gathering actual evidence of behavior, instead of self- reporting can also help us reduce the length of screeners Often, screeners are too long. It is not respectful of and weed out people who aren't being honest. This is a much participants to have them spend 20-30 minutes of their time more direct way of getting at what we need. to see if they qualify. Long screeners will not help you get good and qualified participants, as they have the opposite to the intended effect, because professional participants are ȃI hate that sometimes you spend ř0 minutes on the the only people who will tolerate the length and make it phone going through a screener and then don't qualify.Ȅ through. One way to reduce length is to identify any holes or gaps as ȃI find it annoying when they ask questions like do you well as to adjust language. Remember your screener is not agree disagree and to what range you disagree or agree. just there to help you find the right people, but also to help I also find it annoying when they ask the same the right people find you. Is the language engaging and questions over and over in different wording. It's like inviting? Is your screener bloated? Are you trying to slip a you just asked me that question but now it's worded quantitative survey in there and think no one notices? Are differently, but my answer is still the same.Ȅ you asking repetitive or redundant questions? Do you mean what you're asking? Have you added in behavioral ȃSometimes it annoys me when they ask the same que verification that isn't self-reporting? What can you take out? stions 4 or 5 different times (and don't even try to change the question to make sure I'm not trying to lie my way through itǼ.Ȅ - Participant Experience Online Qualitative Community, Jessica Broome & Kerry Hecht, May 2016 Source: Echo Qualitative Project Support, KNow Research, MindSpark Research International © 2017 GRBN Participant Engagement Handbook | 200 © 2017 GRBN Participant Engagement Handbook | 201

Tips for Qualitative Research Tips for Qualitative Research 87 Be flexible when ȃWhat I donȂt like are the repeated screening questions. Having to answer them online, then on a phone call, you rescreen then again when arriving for the event. X times asking the same questions is a time waster.Ȅ When a person is screened using long batteries of attitudinal statements, either with or without algorithms, they will often ȃThey spent about Ř0 minutes on the phone with me not rescreen exactly the same way. Attitudes are just that, basically repeating every single question I already did attitudes, and they fluctuate depending on any number of online. Then they put me on hold, only to come back and things. Rescreening a participant at the point of research is say my quota was filled. Not sure why they couldn't guaranteeing that some people will screen out. It's also figure that out already before they wasted my time guaranteeing a bad experience for them, in the moment. If calling me.Ȅ participants are screened tightly the first time and you have ȃI absolutely hate the recruiting process...As for being them participate in topical homework assignments, annoying some companies have a required three-step rescreening won't be necessary. verification process…Each contact is by a different person asking the same screening questions to ensure you provide the same information.Ȅ - Participant Experience Online Qualitative Community, Jessica Broome & Kerry Hecht, May 2016 Source: Echo Qualitative Project Support, KNow Research, MindSpark Research International © 2017 GRBN Participant Engagement Handbook | 202 © 2017 GRBN Participant Engagement Handbook | 203

Tips for Qualitative Research Tips for Qualitative Research 88 We should be more straightforward about what we are looking for upfront and what we hope to achieve, instead of engaging in this chess game of Ȋtricks,ȋ ultimately End the games streamlining the screener. It’s not necessary to leave it at that, though. We can easily, through conversation, video and pictures, verify the behavior we’re screening for. When Acknowledge the Ȋchess gameȋ and end it. writing a screener, ask yourself: Do we need this? Does this We recruit participants from databases and we know they question matter? Have we asked this already? What am I very likely belong to more than one; we consider them really looking for? professional panelists. When we try to catch participants in a lie via a screener, they know that and feel compelled to be dishonest and the chess game begins. Let’s reinvent how we screen—our parameters for past participation—and ȃThen, when they ask me if I did a recent study with encourage more honest answers in the process. them and I say yes, they disqualify me. Why did they even call me then?! So much time could have been Lengthy, complicated screeners should not be our method saved if someone just cross referenced my name in for identifying and inviting the right people to a study. The their database before they called me.Ȅ longer and more complicated the screener, the less likely we are to end up with the people we need. ȃFirstly, I don't like having to answer so many redundant questions. I understand it is to see that your answers are consistent, but it doesn't appear that they value my time. And it is such a long, slow process to get qualified.Ȅ - Participant Experience Online Qualitative Community, Jessica Broome & Kerry Hecht, May 2016 Source: Echo Qualitative Project Support, KNow Research, MindSpark Research International © 2017 GRBN Participant Engagement Handbook | 204 © 2017 GRBN Participant Engagement Handbook | 205

Tips for Qualitative Research Tips for Qualitative Research 89 90 Update contact methods Continue the transparency Use multiple or preferred methods of communication. Not Participants told us it is important for the research everyone reads email, many don’t check voicemail, and experience to line up with what was explained in the texting is preferred more and more. Consider your sample invitation. Be respectful of their time and other and try out different methods of communication to entice specifications you introduced during the screening process. them to participate and then screen them. Consider offering multiple methods/options to your recruiting partner so they can be more flexible based on the responses. ȃTo just, like, give us a brief intro, like this is going to be recorded or weȂre not going to be on YouTube or anything. Stuff like that makes us feel more ȃI really appreciated the text reminder ǻeven though the group was on my calendarǼ.Ȅ comfortable.Ȅ - Quote from participant in research conducted by ȃIf there are contingencies, state them clearly…IȂve Doing Good Research done those ones…where they had, like, 70 people, and they pick like six for the second part. Granted, you do ȃTo get the right people for a study group you should have a chance, but thatȂs almost lottery-like. So, if itȂs reach out to social media of all kinds, posting ads that a contingent, like, say 10% of people will be picked people could get paid for giving their feedback. Post that their voice will be heard for doing so and they or whatever.Ȅ could help make these things better.Ȅ - 2017 GRBN Qualitative Workgroup ROR - 2017 GRBN Qualitative Workgroup ROR Source: Echo Qualitative Project Support, KNow Research, Source: Echo Qualitative Project Support, KNow Research, MindSpark Research International MindSpark Research International © 2017 GRBN Participant Engagement Handbook | 206 © 2017 GRBN Participant Engagement Handbook | 207

Tips for Qualitative Research Tips for Qualitative Research 91 Use a friendly moderator ȃI think it is great to get feedback from whoever is conducting the research to allow you to know if you are It may sound obvious, but friendly and personable on the right track or if you need to try harder or researchers and client team goes a long way towards something different. It makes me more likely to give creating a great participant experience. The moderator and solid contribution because I know it is being used and their style should match that of the participants in the checked and is valid.Ȅ group/interview. We heard many comments about how relieved participants ȃI think itȂs the moderator that sets the tone, though, more than anything. I just want friendly, just being were to see that I was a Ȋreal personȋ who obviously has training in what we do, and not just a face with a clipboard. welcomed and not really judged, and just making sure A more serious or stern approach used by researchers to everyone gets a turn, and no oneȂs speaking more than protect their status as the leader of the conversation can risk the other person.Ȅ setting a bad tone for people who are nervous about - 2017 GRBN Qualitative Workgroup ROR participation in the first place. Source: Echo Qualitative Project Support, KNow Research, MindSpark Research International © 2017 GRBN Participant Engagement Handbook | 208 © 2017 GRBN Participant Engagement Handbook | 209

Tips for Qualitative Research Tips for Qualitative Research 92 Allow time for icebreakers ƈUnderstand what motivates ple to participate peo Don’t underestimate the power of an icebreaker! and deliver on Participants felt relieved there was something Ȋrightȋ or Ȋeasyȋ at the beginning that introduced them to everyone those motivations.Ɖ else and involved a little humor. When done properly, the icebreaker allows participants to relax, and they also feel like they had permission to be themselves and share their opinions in an honest way. ȃThere is this one moderator that I had once who said to break the ice he would give extra money to the first few people who would get up and sing a song. He said he would pay up to an extra $100, but the group would have to decide. That was quite fun. I don't know if they impact me, because I'm going to participate no matter what, but it makes the experience more fun!Ȅ - 2017 GRBN Qualitative Workgroup ROR Further Reading Source: Echo Qualitative Project Support, KNow Research, MindSpark Research International © 2017 GRBN Participant Engagement Handbook | 210 © 2017 GRBN Participant Engagement Handbook | 211

Tips for Qualitative Research Tips for Qualitative Research 93 Enforce the impact ȃI am very interested in the product type and liked sharing my perspectives.Ȅ At the end of the day, people remember getting to share their opinions/insights, being listened to, and having an ȃIt was fun to see the different packaging options and interesting experience. These aspects should be ensured for to be exposed to the creative process.Ȅ participants in any project and should be communicated during the recruiting process. People like to contribute to ȃIt was fun learning about what you are working on something larger than themselves so ensuring that and being able to help.Ȅ participants feel this is important. ȃThat they took my feedback and used it.Ȅ If possible, share with participants the results or action ȃI liked knowing I could help shape the outcome and in taken as a result of the research. When this is not possible, a small way, hopefully contribute to the company's talk with them about the purpose of the research and what success.Ȅ might be done with the results. - Quotes from participants in research conducted by Doing Good Research Source: Echo Qualitative Project Support, KNow Research, MindSpark Research International © 2017 GRBN Participant Engagement Handbook | 212 © 2017 GRBN Participant Engagement Handbook | 213

Tips for Qualitative Research Tips for Qualitative Research 94 ȃIn the last research I attended, we had to come up with Use group tasks/ an idea for a new product, and then present it to the rest of the group. I enjoy activities like this, as it requires me collaborative activities to be creative, and to also work with others to create whenever possible the best end result we can. I liked this as it gets me thinking, and we all have to pitch in and come up with When possible, include a group task/collaborative activity in something. Plus, we had to present it to the rest of the group discussions. Participants told us that they enjoy the group, which brought a bit of a competitive edge into it, opportunity to work together because it was different from which is something I enjoyed.Ȅ always answering questions from the moderator. ȃOne of the most enjoyable studies that I have done This allows them was a study for different types of baby food. I really to engage with enjoy studies that I get to add my children into the fun. others and roll up The best part of this study was watching my daughter their sleeves to get try all the different types and flavors. Some of the faces involved. It makes she would make were absolutely perfect!Ȅ time move faster and the client - 2017 GRBN Qualitative Workgroup ROR team benefits from observing non-moderated collaboration when some of the most honest comments surface. Further Reading Source: Echo Qualitative Project Support, KNow Research, MindSpark Research International © 2017 GRBN Participant Engagement Handbook | 214 © 2017 GRBN Participant Engagement Handbook | 215

Tips for Qualitative Research Tips for Qualitative Research 95 96 Consider fewer Consider in-situ ƈblindedƉ studies Offer more opportunities to do in-situ interviews. Participants enjoy giving honest feedback to brands Participants say they like to do in-store and in-home regardless of if they know the sponsoring brand or not. A interviews since it's often easier for them to get to and more positive experience with the research study reflects exciting to do interviews on site. positively on the brand. Almost everyone who had a positive Side benefit: You see participants engaging in the real world experience with the study felt more positive toward the environment and give them a more authentic setting to talk brand after. The few who didn’t said the study had no impact about the category. on their perception of the brand (which could be because the study was blind). ȃI really enjoyed being able to test out a product that I have never heard of before. Being able to try something new and discovering that it could be something that you ȃFor me, if I know that IȂm making a difference and buy or replace what you already use for this new product change, itȂs almost like a family feeling. But if IȂm doing is awesome.Ȅ something ambiguous and itȂs like I donȂt know whatȂs going on, itȂs like, okay, for the money, I guess.Ȅ - 2017 GRBN Qualitative Workgroup ROR - 2017 GRBN Qualitative Workgroup ROR Source: Echo Qualitative Project Support, KNow Research, Source: Echo Qualitative Project Support, KNow Research, MindSpark Research International MindSpark Research International © 2017 GRBN Participant Engagement Handbook | 216 © 2017 GRBN Participant Engagement Handbook | 217

Tips for Qualitative Research Tips for Qualitative Research 97 98 Respect the time Thank them commitment Even if you can't provide them with any additional information about what the research was about, you Keep to the schedule. Don’t take advantage of people’s time. certainly can send a thank you note to the participants. Also, Start on time, tell them you will end on time, and follow let them know they might not hear from you in a while, but through. that you will reach out as soon as you have another opportunity that would benefit from their opinions. ȃIt takes some planning…like, getting your ȃYou sort of want to feel valued, like your opinion confirmation sooner. I mean, in some IȂve got the confirmation that day, you know. I was going back and is valued.Ȅ forth, sending them emails, and, I mean, whatȂs going - 2017 GRBN Qualitative Workgroup ROR on? So, I can plan my day.Ȅ ȃGood hospitality – water and snacks and things like that. Caffeinated beverages. We appreciate the comfortable chair. Starting on time. Leaving on time.Ȅ - 2017 GRBN Qualitative Workgroup ROR Source: Echo Qualitative Project Support, KNow Research, Source: Echo Qualitative Project Support, KNow Research, MindSpark Research International MindSpark Research International © 2017 GRBN Participant Engagement Handbook | 218 © 2017 GRBN Participant Engagement Handbook | 219

Tips for Qualitative Research Tips for Qualitative Research 99 100 Continue the feedback loop Share the impact Consider that participants should be thanked for their time, If possible, follow up with people after the study to let them but also be sure to get feedback from the participants on know how their feedback was used. Participants appreciate what the process was like for them from start to finish. This being given any relevant information that can be shared could be deployed by the supplier and moved back up the (e.g., ȊWe were able to redesign our label thanks to you! food chain. Include in this feedback a record of what types Keep an eye out for the new version in your local store.ȋ). of projects they enjoyed (self-reported) and what they are This helps shift the emphasis slightly away from financial best at (from the moderator/client perspective). This helps incentives and more to sharing the work we are doing researchers invite participants to the projects that most collaboratively with participants. benefit the client and that they will be the most likely to When it is possible to share, participants love it and it makes attend. them eager to participate and provide quality insights again. ȃI particularly enjoyed looking at the X Brand advertisements and saying what I liked/disliked about them. And now when I see them on TV I think 'I helped to decide how these adverts look' and feel quite proud.Ȅ - 2017 GRBN Qualitative Workgroup ROR Source: Echo Qualitative Project Support, KNow Research, Source: Echo Qualitative Project Support, KNow Research, MindSpark Research International MindSpark Research International © 2017 GRBN Participant Engagement Handbook | 220 © 2017 GRBN Participant Engagement Handbook | 221

Tips for Qualitative Research Tips for Qualitative Research 101 ȃI think they compile them into a statistical survey to Surprise and delight see what the general responses are. They report back to the companies with recommendations based on those with bonuses responses. The companies then use that to target their audience. I have actually seen commercials that I think After the interview, surprise and delight people in a visible took some of our responses and put them into action.Ȅ way when they do a good job. Reinforce engagement by giving additional thanks. This is the one point at which - Participant Experience Online Qualitative Community, transparency is less important than delight! Jessica Broome & Kerry Hecht, May 2016 ȃI would like that we get the results, like, from the taste test, and they actually know. Like we probably go for a ȃSo, I did one…where they said, ȁAnyone who came one taste test, or discussion, or we talk about this, we talk minute or more early, five minutes or more early, they about that. We can actually know where theyȂre at in get $25 cash.ȂȄ their production. Maybe I want to buy the product.Ȅ - 2017 GRBN Qualitative Workgroup ROR ȃI kind of like having a check pop up in my mailbox.Ȅ - 2017 GRBN Qualitative Workgroup ROR Source: Echo Qualitative Project Support, KNow Research, MindSpark Research International © 2017 GRBN Participant Engagement Handbook | 222 © 2017 GRBN Participant Engagement Handbook | 223

Have skin in the game? Get involved Even though we have more than 30 companies currently partnering with us on the Participant Engagement Initiative, that is only a tiny fraction of companies who have skin in the game. The entire industry will benefit from increased participation rates and participant engagement, or Afterwords conversely, will suffer if rates and engagement continue to decline. If the company you work for is currently not a partner, we strongly encourage you to get involved. Become a partner on the initiative and we will help you: The journey doesn’t end here 1. COLLECT METRICS DATA ON THE USER EXPERIENCE 2. ENGAGE YOUR TEAMS, YOUR CLIENTS, AND YOUR DATA COLLECTORS ON IMPROVING THE USER EXPERIENCE 3. SHARE YOUR LEARNINGS AND BENEFIT FROM THE LEARNINGS OTHERS SHARE Contact us to partner on this initiative. © 2017 GRBN Participant Engagement Handbook | 224 © 2017 GRBN Participant Engagement Handbook | 225

Building The Building Public Trust Charter Public Trust Charter The foundation for the future of our industry © 2017 GRBN Participant Engagement Handbook | 226 © 2017 GRBN Participant Engagement Handbook | 227

Measuring the teams implement a ROI measurement program and ROI from Insights demonstrate the impact insights have on the business. You can read more about the ROI from Insights Initiative here. This Handbook is all about increasing participant engagement by delivering better user experiences. GRBN is concurrently working on producing a second handbook; one to help client-side insights teams measure the business impact the insights function is delivering. We see ROI as critical to growing insights budgets over the years to come. Boston Consulting Group (BCG), Cambiar, and Yale conducted research on the Future of Insights in 2015. One of the key findings was that relatively few insights teams are measuring their ROI. You can read more about these research insights here. This year, GRBN has co-operated with BCG on conducting further research, specifically looking at the ROI from insights measurement issue. In Q4 2017, we will be issuing, in collaboration with BCG, a report on the Please get in touch if you would like to find out more about research. In Q1 2018, an ROI from insights measurement this initiative and to receive the report and Handbook. handbook will be issued to help client-side insights © 2017 GRBN Participant Engagement Handbook | 228 © 2017 GRBN Participant Engagement Handbook | 229

About GRBN About The Global Research Business Network, GRBN, is a not-for-profit organization, bringing together four regional GRBN federations and over 40 market, social and opinion research associations. GRBN connects over 3500 research businesses on six continents. GRBN’s mission is to promote, protect and educate the global research sector. Promoting, protecting, and educating the global research sector © 2017 GRBN Participant Engagement Handbook | 230 © 2017 GRBN Participant Engagement Handbook | 231

Many thanks to our partners ENGAGE 101 tips to improve the research participant user experience Premier Sponsor Gold Sponsor © 2017 GRBN Participant Engagement Handbook | 232