It is often said that “less is more”, but when it comes to Online Marketing & Advertising Research, “the more the better”.
That said, doing both quantitative and qualitative should be only done with a clear understanding of the strengths and benefits of each.
For example, when doing qualitative exploratory research, the number of respondents beyond a number between 5-10 per study, is not the main factor; far more important is to enable a setting with a minimum of bias (within a participant’s home), and minimum barriers to participation (to ensure you are receiving feedback from your target demographic, rather than limiting yourself to stay-at-homes, students or the unemployed).
If you allow a minimum invasive testing environment, within the participant’s home, with tasks that do not require a large amount of time (e.g.: 10-15 minutes) and allow the participant to schedule the test at his or her convenience (when their favorite TV show has finished, dinner is over, the kids in bed, etc.), then you are making it easy for almost anyone to participate, and, you can leverage the targeting capability of social media and other online techniques to recruit your exact target demographic.
For quantitative testing, the number of respondents in each study IS very important, because, rather than exploratory research, you are attempting to deliver conclusive research, and you need statistically valid sample numbers, drawn from appropriate target populations.
One type of quantitative research (although not generally labeled as such) that unites quantitative research (discovering what works best with a target population) with feature design and development, is A/B testing and Multivariate testing.
By using large numbers of respondents, sent to alternative versions of a display ad, landing page, etc., we can see which version “converts” best, and thus, conduct continual optimization (which unites, under one practice, research, and design & development).
This type of quantitative research has seen an explosive growth in the last 10 years, and yet, many of the organizations using it are not achieving its full potential because they are not using it in conjunction with qualitative analysis.
Because qualitative research is rarely used in conjunction with A/B testing, the potential to conduct a “best of the worst” optimization is very high, and, perhaps more worryingly, may be invisible; if we only conduct quantitative (what, how many, where, when, etc.) research and do not complement this with qualitative (why) research, we may never achieve the “step change” in improvement that may be possible.
By conducting interspersed 5-10 person qualitative studies in between each bout of A/B testing to discover the why of the results, or to open up the creative discovery process and enable a “best of the best” quantitative optimization, we can bring A/B testing and Multivariate testing to its full potential.
The ubiquity of web cams and broadband means that it no longer takes 2-8 weeks to conduct in-home qualitative research; now you can receive qualitative research studies showing full audiovisual results of participants in their homes (what they say, do, see and how they interact) within days or even hours of setting up a test.
In other words, now you can unite both quantitative and qualitative online research in a set of interspersed iterative testing, where one (exploration) feeds the other (conclusive).
Or, in layman’s terms, now you can have your cake and eat it too.
