We are all sick to death of surveys.
Survey fatigue is a huge problem, especially in education. Our students are constantly surveyed to the point where they just toss in any old answer, just to get it done. And yet, the drive for accountability pushes us evermore towards asking for more and more data.
So how can we address this burgeoning problem? Why not take a lesson from Yelp and use stratified sampling?
Yelp is very dependent on its customers to add to its extensive database about restaurants, services etc. They could not do this if they were to depend on staff to enter all this data – they need user-driven data collection i.e. surveys. But they are smart about it. If you wanted to enter a review for a restaurant that they know little about, imagine how much of a pain it would be if they were to ask you a whole bunch of questions (the typical survey) – after your first painful review, you wouldn’t do it anymore.
Instead of asking all the key pieces of information about that restaurant, they will only ask you for 2-3 things e.g. do they have parking?; do they take credit cards?; do they serve children? (boiled or fried?). Most of us will put up with a really short survey, taking only a minute.
The next person who reviews that same restaurant will be asked a slightly different set of questions: do they do breakfast?; are they expensive?; do they take credit cards?. But by gathering a slightly different set of questions from each user, they can get a broader dataset, without having to hassle each one for everything. They also get more depth in their dataset, not being dependent on one person’s impression (often half remembered) about whether they have parking etc. They can aggregate the answers towards a likely result.
This is basis of stratified sampling. Asking different strata or layered samples of your population slightly different questions. Fewer hassles –> happier respondents –> more quality data.
Now, Yelp is a bit more clever than this. They don’t just ask completely random questions. The stratification is weighted: more important questions are asked more often; users are tracked and are asked questions that they are more likely to respond to; answers that have more variability are asked of a larger (and more varied) pool of respondents. The algorithms underlying this are pretty sophisticated.
So, how does this help us in medical education? Well, we don’t have the multimillion dollar resources of Yelp.com — but we do have tools that can help us with stratified sampling, such as OLab4. Unlike most survey platforms, OLab4 can apply some smarts to which users are asked which questions.
Lessen the survey fatigue in your students and teachers. Use stratified sampling to get more out of your surveys.