An Interdisciplinary Network on the Methodology and 
Applications of Integrative Research Methods
The second workshop meeting of the network „Mixed Methods and Multimethod Research” took place on July 5-7, 2018 at the University of Cologne.
Videos of the keynote presentations are available here.
15 network members and international guest researchers participated in the network’s second meeting on the topic of “mixed methods and multimethod research for causal analysis” on July 5-7, 2018 at the University of Cologne.
Along with different methods and disciplinary perspectives, this time notions of causality and causal research design were at the core of the workshop proceedings. Two distinguished keynote speakers were invited to add to the discussion: Gary Goertz (University of Notre Dame, Indiana), one of the most prominent experts in the field of case-based and set-theoretic methods in comparative political science, and Joseph Maxwell (George Mason University, Virginia), who highlighted the potential of an interpretivist notion of qualitative research for causal analysis from the perspective of educational science and anthropology. Thomas Wencker and Gerald Leppert of the German Institute for Development Evaluation complemented the debate with an applied perspective on mixed methods and causal analysis in evaluation research, whereas Nina Baur (TU Berlin) presented a methodological critique of current models of explanation in quantitative research.
At the center of Gary Goertz‘ keynote stood the notion of a causal “research triad” (Goertz 2017), which models multimethod research as a combination of multiple causal inference methods. Based on a concept of causal mechanisms that assumes the primacy of singular causation, i.e. causal processes in particular events, over causal regularities, the research triad combines case studies for analyzing local causation, causal mechanisms for generating explanatory models for the observed causal processes, and cross-case comparisons for assessing scope and generalizability. Concerning the practical application of this approach, Goertz focused on guidelines for case selection. Here, the selection of cases “on the dependent variable” – a practice widely perceived as ill-advised (King/Keohane/Verba 1994) – is of central importance, since the analysis of causal mechanisms hinges on the observation of cases that show both the proposed cause and outcome.
One especially interesting aspect of Goertz‘ model of method integration is the absence of a traditional qualitative-quantitative distinction, which is replaced by the within- vs. cross-case analysis distinction. Mixed methods and multimethod research, terms used as synonymous by Goertz, hence becomes „multicausal inference“.
In contrast, Joseph Maxwell in his keynote insisted on a notion of qualitative research more closely connected to its origins in symbolic interactionism and anthropology. His focus was on the concept of interpretation and the contrast of interpretivist social research to both statistical large-N research and formalized understandings of qualitative methods (e.g. QCA, process tracing). Maxwell argued that the specific strength of qualitative methods is a subject-oriented analysis of meaning, which also constitutes its characteristic contribution to causal analysis: It facilitates the observation of actors’ perceptions, meanings and intentions, which due to their situated and historically flexible character defy law-like generalizations. It is especially in the area of intervention research and program evaluations, where standardized and statistical approaches aiming at average-treatment-effects (e.g. RCTs) miss this important aspect of social phenomena, and are thus prone to producing invalid and insufficiently applicable results, Maxwell claimed.
However, despite his emphasis on interpretive methods, he also directed his criticism towards qualitative researchers who equate their rejection of “positivist” methods with the rejection of causal analysis per se – exemplified most prominently by Yvonna Lincoln and Egon Guba’s work (Lincoln/Guba 1985). The strengths of qualitative methods for causal analysis, Maxwell argued, also need to be defended against such approaches to qualitative research.
The lively discussion following each of the keynote presentations dealt with questions regarding the relation of the two conceptualizations of causality proposed by Goertz (causal mechanisms) and Maxwell (causal processes). A possible convergence of both concepts was discussed, despite the considerable methodical differences in both speakers’ approaches. Additional questions addressed, among other topics, the issue of distinguishing case-based vs. cross-case-methods in Goertz’ account, e.g. the positioning of experimental designs within the “research triad”.
During the afternoon of the first workshop day participants discussed the formulation of a common definition of mixed methods and multimethod research within the network. A proposal for a network “mission statement” was presented, which focused on an inclusive understanding of MMMR, highlighting the potential of method integration without claiming to form a unified research approach or “third paradigm”. Instead, MMMR was defined as a site for dialogue and constructive criticism between different methodological traditions. The fragmentation of the MMMR field along disciplinary, methodological, and regional divides was discussed as one of the most pressing issues within MMMR.
Future methodological research should thus focus more strongly on systematic reviews and meta-analyses of current mixed research practice, as one research proposal suggested. Such contributions to a “methodological ethnography” (Goertz) of MMMR may not only yield a better overview over current research practice, but also serve important functions for methodological dialogue across disciplinary and methodological boundaries.
A more systematic approach towards MMMR teaching and training was discussed as another important step towards a more inclusive and visible mixed methods community. The development of MMMR-specific data analysis software, which not only extends already existing QDA or statistics applications, but fundamentally incorporates both qualitative and quantitative techniques was debated as another important premise of the advancement of mixed research.
The following day continued with a methodological presentation by Nina Baur (TU Berlin) and an applied perspective from Gerald Leppert und Thomas Wencker (DEval). Nina Baur presented her ongoing research on the critique of explanatory models in quantitative social science. Tackling multiple methodological core issues, such as the processual character of social phenomena, the modelling of different “time layers” (Kosseleck 2003) in time series analysis, or the linking of micro and macro perspectives, she pinpointed persisting difficulties in integrating an interpretivist perspective into the methodology and philosophy of science underpinning social research.
Gerald Leppert und Thomas Wencker contributed a thematic shift towards the application of mixed methods designs in empirical research. They gave insights into the extensive program evaluation projects conducted at the German Institute for Development Evaluation. Gerald Leppert presented the intricacies of an impact assessment study for a comprehensive land use planning project in the Philippines, integrating document analyses, geo-information-data, and qualitative interviewing into a field-experiment. Thomas Wencker outlined current research in the area of DEval’s methodological framework, highlighting the distinction of constitutive vs. causal factors, as well as an extended concept of invariance for causal research design.
Letzte Änderung: 20. August 2018