Quality of access panels


A recent article in Research World April 2007 highlighted concerns in the market research industry caused by the increasing use of online access panels. The concerns raised were about the variation in the quality of the access panels, how they are managed and the consequences this has on the quality of data coming from them.

The article by Jo Bowman (“Hands up, who’s on an access panel?”) highlights the importance of distinguishing those who are doing research properly from those doing cheap opinion polls on the web with no basis of quality behind the work. But Bowman points out that even amongst access panel providers there is no agreement on what is a quality panel.

The article points to a few key factors for good panel management i.e. careful sampling, multi-sourcing of panellists, watching responsiveness and speed of survey completion etc, but it says that the end clients of the research are not asking enough questions of the access panel suppliers to distinguish good from bad in terms of panel quality. It implies a certain naivety in the purchase decisions being made, assuming that all panels are run the same way and achieve similar results.

In reality there are many factors that can affect the quality of results from an access panel.
Subtle differences in the way a panel is managed can affect the types of people who actively participate in surveys.

As providers of software to the online panel market Centurion sees varying attitudes to panel management. Take the argument about frequency of contact, for example: some providers contact panellists as often as possible on the basis that they will loose them anyway so they must use them as much as possible while they have them. Others will nurture their respondents into a community providing carefully designed incentive schemes, information on the outcomes of research, blog space etc and control contact to a specified number of times per month.

ESOMAR have listed 25 key questions to ask when buying such services that will flush out the good from the bad providers.

Of the questions suggested by ESOMAR many can be given the best answers by using a proper panel management and sampling tool like MARSC. Below are many of the questions and the contribution made by MARSC to give the best quality reply.

 

	ESOMAR Question	How MARSC can help
1.	Is it an actively managed panel (nurtured community) or just a database?	MARSC helps to manage a panel actively e.g. controlling number of surveys sent, careful targeting based on demographic information etc.
2.	What is the percentage of active members and how are they defined?	MARSC keeps a record of panel activity and so allows the user to establish a measurement of active panellists.
3.	Where are the respondents sourced from and how are they recruited?	You can store information on source of a panellist in MARSC. Then you can report on e.g. active panellists from different sources to find what your successful sources of recruits are for particular demographic groups.
4.	How many members have clearly opted in and if so was this a double opt-in?	The MARSC panel web portal has a recruitment element with a double opt in as standard. You can plug it straight into a web page, looking like the rest of your site.
5.	What do panel members get in return for participating?	In MARSC it is easy to operate a transparent rewards system with points and prizes automatically available to those who have earned them.
6.	Is panel compliant with laws in the territories it operates in?	A typical area of regulation is the operation of an opt-out on request of the panellist.  MARSC operates a “Do not contact” flag.  When this is marked the name will never be put forward for interview.  The user does not have to remember to set it in a query each time.
7.	What basic profile info is kept on its members?	You can store whatever profile information you need in MARSC and add new profile information collected in surveys. There is no limit to the number of records MARSC can store.
8.	How often is it updated?	You can bring new information into MARSC on a daily basis if required. The process is made easy and quick to either update current information or to add new.
9.	In what other ways can the panellists be profiled?	It is possible to hold as much profile data as you want in MARSC. There are no limits and it is made easy to add new.
10.	What are the likely response rates and how is it calculated?	MARSC offers a response rate calculator. You put in how many interviews you need and MARSC will calculate from historic information how many names are needed for a survey.
11.	Can panel members who have recently taken part in a similar survey be excluded from a sample?	In MARSC you can set up exclusions based on previous surveys or groups of surveys, so it is easy to exclude people who have worked on similar projects recently.
12.	Is a response rate guaranteed?	If you don’t get the expected response rate in MARSC it is easy to issue a “top-up” sample to complete the necessary surveys.
13.	How often are panellists contacted?	In MARSC you can set the rules about how often people should be contacted both an upper and a lower limit.
14.	How is the sample selection done?	MARSC provides a clear selection process, saving the target matrix for each sample job so correct definitions can be checked at any time.
15.	Can samples be deployed as batches?	You can issue sample in batches from MARSC to suit the research being done.

For further information on MARSC please contact Russell Lambert on +44 (0)1306 621062 or email sales@marsc.com or see our web site www.marsc.com. For the full list of ESOMAR recommended questions click here for their guidlines.