Under the title "The Market Research File," Der Spiegel has been continuously reporting for several months on questionable practices in the market research industry.
Since then, the topic of quality in market research has been discussed in relevant media.
Compared to quantitative market research, there are very few standardized criteria in qualitative market research to measure the quality of qualitative research. We would like to present ideas for quality assurance in individual process steps:
(1) in briefing, (2) in conception, (3) in execution, and (4) in analysis and preparation.
Quality in Briefing
The client has a specific reason for market research, but it is often difficult to clearly describe the decision problem. Therefore, a shared understanding of the client’s business situation is important to us. Through critical questioning, we strive to comprehensively understand the decision problem and, based on that, advise on information needs. The fundamental idea of qualitative market research is to “understand” a specific phenomenon (e.g., loyalty) or a target group (customers, employees, etc.), which usually involves relatively small sample sizes. For this reason, we do not make promises regarding the general validity and representativeness of the results in qualitative market research projects. We clarify with the client which questions need to be answered, which goals need to be achieved, and which decisions should be made based on the information. Depending on the objective and research question, we select suitable qualitative methods and justify our choice. In some cases, it might also be useful to combine different methods (triangulation) to expand the insights gained. If certain methods are proposed due to the provider’s specialization or the availability of specific infrastructure (e.g., test studio or call center), clients should critically question the approach.
Quality in Conception
The formulation of questions (e.g., interview guide) is a central aspect of qualitative market research. The goal is to cover the defined information needs in the guide and to ask only questions that help to better understand the decision problem. We follow the PAKKO principle, meaning the questions must be personal, activating, concrete, concise, and open-ended. Pre-tests help us to avoid inconsistencies in the questions and their order and to ensure that all relevant topics are adequately addressed.
Quality in Execution
When selecting the “right” interview partners, we work with the client to ensure that all relevant information is included and that the heterogeneity of the target group is covered. Theoretically, the number of informants is determined by the “point of saturation” (i.e., an additional interview yields no new information). In practice, we rely on experience (20-25 informants). However, this number varies depending on the type of analysis or perspective (e.g., subgroups).
Interviewers play an important role in qualitative market research. Not everyone can conduct qualitative interviews; it requires fundamental qualifications and intensive training. Some clients are firmly convinced that their field staff can conduct interviews due to direct customer contact. We believe this is not the case: (1) the intensity of the interviewer’s personal interaction has a major influence, (2) people often only hear what they want to hear, and (3) customers are not as honest and open as they would be with a neutral third party. To conduct high-quality conversations despite the necessary distance from the respondents, intensive interviewer training is required before the field phase begins. The goal is to provide interviewers with general information about the client and their products, as well as project details (including objectives, information needs, and important topics for the client) and industry insights. The interviewer is thus sufficiently informed to be taken seriously in conversations with the target group and to ask relevant follow-up questions.
Through regular exchange among interviewers during the field phase, regarding findings and deviations from what has been heard so far, etc., commonalities and differences can already be identified at this stage, and insights can be mentally integrated into subsequent interviews. Subsequently, the transcription of the data helps us in the structured evaluation and interpretation process.
Quality in Analysis and Preparation
To avoid a one-sided interpretation of the data, we conduct the evaluation of the collected data in analysis groups with at least two people (researcher triangulation) and apply different forms of analysis (e.g., classical content analysis, hermeneutic analysis, semiotics, metaphor analysis). This allows for various perspectives to be taken.
Our aim is to present the results clearly and comprehensibly to the client. The results must always be viewed in the context of the subjects interviewed. Every behavior and every statement can only be correctly interpreted within its context. To illustrate the quality of our interpretations, we include original quotes from respondents in presentations. We emphasize a detailed description of the specific situation and the interview partners, so that our client is able to understand and interpret the data themselves. However, this always happens while maintaining the anonymity of the interview partners or with their consent for named attribution.
When preparing the data, we disclose contradictions and unresolved questions and make it transparent what are facts and what are initial interpretations. In the next step, we interpret the data together with the client, discuss consequences, and derive measures. We not only deliver the data but also highlight new insights and present key statements so that the focus on the essentials is not lost.
CONCLUSION: OUR UNDERSTANDING OF QUALITY
The quality of qualitative market research is only as good as the quality, in terms of competence and understanding, of the actors involved. Qualitative research only depicts a segment of subjective reality, and the proof of an objective, universally valid truth is not sought.
The collection and preparation of data alone do not guarantee high-quality market research. Ultimately, quality only becomes apparent when the results provide concrete courses of action. For us, the clear separation of collected facts and interpretations is important. The “facts” are interpreted together with the company; only then can the true value for the company be crystallized in terms of “What does this specifically mean for the company?”
Services