By: Kristin Cavallaro, Knowledge and Data Analysis Specialist, SSI
The “Death of Panels” has become a common phrase that we have been hearing in the Market Research industry for a few years now. But what does that really mean? Why are panels “dying?” And if they are, how does that affect online market research? These are all questions that plague market researchers and public opinion pollsters.
Many sample suppliers have put multi-sourcing in place as a solution to the “dying panels” issue. In multi-sourcing, suppliers and researchers combine several panels in one study and in many cases, include non-panel sources as well. This is an effective solution as long as proper blending practices are in place. It does not, however, answer the question of what is happening to our online access panels.
Part of this shift away from panels may be explained by the migration away from traditional email to social media channels, which are becoming more popular as vehicles for. The way in which people communicate, gather and share information has evolved over the past few years, while, unfortunately, the method of contact we use for online research panels has remained essentially the same.
But is that the only reason many panels are struggling to find and keep members? After much hypothesizing about why only a select few would ever join an online research panel, SSI decided to just ask the people who would know for sure—respondents themselves.
How Did We Gather the Information?
After targeting SSI’s non-panel sample, we asked questions targeting those who are either no long on a panel or those who have never joined a panel. Interestingly, only 22% of our “non-proprietary panel” stream qualified for this study. This suggests that there are more people across several research panels than we tend to think there are. There were intentionally no demographic quotas set in the effort to identify differences in the demographic profiles of panel vs. non-panel participants.
How Are Panel Resisters Different from Panel Members?
Since there are not any demographic quotas set in this test, we are able to look at any demographic differences among participants who currently belong to an online research panel vs. those that do not. The results revealed that there are no significant or perceived age differences between the two groups.
We do, however, find differences in gender and employment status. We know that men are difficult to recruit no matter what their age. Therefore, we would expect to see a higher percentage of males in the panel resisters group. The results do show a slightly higher percentage of males in the resisters group than in the panel group— but there is not a significant difference.
Surprisingly, the employment status of panel resisters vs. panel members does show a meaningful difference in distribution. We see a 7% increase in respondents employed full time in the resisters’ groups over the panel members’ group. One can speculate that this finding relates to respondents’ perception that they do not have time to take a survey. They then find, however, that they do have time between meetings, during a lunch break or just when they want to break up their routines.
So Tell me Already! Why Don’t people Join Panels?
Surprisingly, the most common reason people give for not joining a panel is simple. No one ever asked them. With all the recruitment campaigns running from all the panel companies, it is difficult to grasp the fact that there are so many people who have never ever heard of an online research panel, let alone had the opportunity to join one. This finding does reinforce the idea that we need to reach people in the right place and under the right conditions for them to notice us.
Believe it or not, fear of commitment is another reason people don’t want to join panels. It is shocking to see that 26% of participants are afraid of committing to panels, even though they are willing and able to take surveys. We hypothesize that these participants fear that there will be an expectation for them to take lots of surveys if they join a panel and therefore shy away from the membership.
Upon looking at their participation data over a three month span, however, we see those afraid to join panels actually coming back several times to participate. They must feel less pressure to take a “one-off” that flashes across their PCs than they would from actually joining a panel.
Many people also say that they do not have time to take surveys. But they took the survey for this study didn’t they? Without psychoanalyzing these respondents, we can’t really understand their reasons, but we can speculate. It could have something to do with the perceptions of their available time.
A typical respondent has several things going on in their lives that researchers are competing against for time and attention. School, work, family, friends, exercise, sleep, TV, Face book and video games are all in direct competition for our respondents’ time. If we ask them to join a panel, they look at their list of daily activities and surveys don’t seem to fit in their schedules. If researchers can catch them during down time, however, they will take a survey. This may occur between meetings at work, when the kids take a nap or when they are waiting to catch a plane.
Just out of curiosity, we looked into how many surveys panel resisters attempted in the past 90 days to take surveys compared to the known panelists. To our astonishment, resisters have come back to take a survey several times over the past three months. The resisters don’t take as many as the panel members, but they still represent a respectable third of the total starts of panel members.
We took this a step further and looked only at the participants who stated they don’t have time to take surveys and found the same results. They do have time to take surveys. Researchers just need to be in the right place at the right time.
Why Do People Who Join Panels Drop Out of Them?
We know why people do not join online research panels in the first place. But why is it that some who do join decide to leave? Our research examines that issue, as well. Our showed that respondents are upset with the fact that they never qualified for a survey. Participants are tired of feeling like they are not wanted when it comes to their opinion. As an industry, our qualification rates are getting lower, as we try to pinpoint our exact targets. As a result, our respondents’ survey taking experiences suffer.
Sample providers understand that low qualification rates put a strain on respondents’ experience. Respondents get frustrated when they are turned away from multiple surveys before finally qualifying for one. To combat this, many sample providers, SSI included, have been working to develop and implement methodologically sound methods of prescreening. This approach allows providers to match participants to appropriate surveys that they might actually qualify for rather than rejecting them. For those of us that have implemented this method, we already are seeing a positive effect on respondents’ participation and experiences.
What Does This Mean for Researchers and Their studies?
It is important to remember that panel resisters are people first and foremost. They are consumers and they are part of our target markets. They have opinions that need to be heard. By restricting our research to online research panels only, we exclude all of the “panel resistant” consumers from our sample. Knowing how to blend these respondents properly into research is key to collecting representative and reliable data.
It is also important to realize that panels are not going anywhere. They will continue to be part of the online research world for years to come. We need to understand our respondents now more than ever so that we can slow the turnover of current panelists, as well as continue to recruit new panelists. Respondent experience is the largest driving force behind the survival of our panels. To keep our panels vital and healthy, it is important to do all we can to ensure those experiences are positive, nurturing and engaging.