I have had a number of issues with quotas over the past few surveys. This has been based on the system excluding people based on “number of people started survey + existing completes”, rather than solely when the completes have met the target.
This has brought me into conflict with panel companies (as I am rejecting vast quantities of sample) and extending fieldwork times (creating problems with workloads/clients). The only fix I can see for this is to open up the quotas – but that leaves me open to cost implications (and begs the question what are the quotas for!) and for some surveys those are sizable risks I won’t take and so is losing business.
Does anybody have a solution or tips?
I am assuming you are referring to the scenario described here (as it sounds like you are running into a situation where there are folks in progress and folks that have finished and the quota system is making sure that it doesn’t let in too many respondents):
Without knowing what the size of your survey audience is, I would assume you have a lot of respondents jumping into the survey at once and taking up the available slots – perhaps one option to consider is to launch the survey in waves?
This might help avoid a flood of initial responses that take up available slots, and as respondents are disqualified or don’t finish, their slots are open again once the second wave of respondents comes in, and so on…
Correct me if I’m wrong but It sounds like the quota system is doing what it’s meant to do (making sure that you don’t go over quota), but you might be getting too many respondents into the survey all at once..
Hope this helps!
Sorry Dominic but I have tried waves of respondents and it does not solve the issue. Even with that the only way to complete the surveys is to open up the quotas, – hence he system is actually not fit for purpose, in terms of being a usable practical tool.
To give you a specific where the main problem come at the end of a survey. Here with an audience with a penetration of 1% trying to get the last 2 interviews. What happens is that even with doing waves of respondents you have no option but to throw sample at it at the end but the odds of an eligible person doing the survey when there aren’t 2 people in the survey become astronomic. This results in unhappy panel companies, additional time in manageing projects for me, extened survey times and unhappy clients.
Unless you base quotas solely on completes, before ejecting respondents, this is fatal flaw in your system.