Katri Huutoniemi, "Evaluating interdisciplinary research"

There is little knowledge or consensus on how to evaluate interdisciplinary research, which does not seem to fit in well with the current system for producing scientific knowledge. The chapter by Katri Huutoniemi analyzes the key characteristics and challenges of interdisciplinary assessment by drawing insights from the conceptual and pragmatic discussions of interdisciplinary research, empirical analyses of evaluation activities, and initiatives and experiences of participating organizations. It introduces the central challenges involved in evaluating interdisciplinary research and focuses attention not on the criteria used to conduct interdisciplinary research, but on the perspectives used to evaluate it, and highlights the consequential role of both concepts and practices in defining merit: First, it shows how different conceptualizations of interdisciplinarity shape assumptions about quality; and second, it discusses how values are actively constructed by the people and practices involved. 

At the beginning of the chapter she defined a interdisciplinary as "a genus of integrative research activities that combine more than one discipline, field, or body of knowledge" and differentiate from the term transdiciplinary which refers to "trans-sector problem solving where various stakeholders in society are actively involved in knowledge production." 

Although interdisciplinary research is not easily amenable to evaluation, she valued the key functions of interdisciplinary research evaluation, which can be summarized as follows: 

• needed for organizational learning and improvement of performance and quality of research activities
• to bolster the credibility of research; it helps to legitimate research and its results
• accountability and transparency in the use of public funds
• not only for separating the qualified from the unqualified, but also for distinguishing between competing types of high-quality research 

Next, the author distinguished between three evaluative approaches to interdisciplinary research that "highlights some key features of interdisciplinary as a special type of challenge for research evaluation": (1) mastering multiple disciplines, (2) emphasizing integration and synergy, and (3) critiquing disciplinarity. 

(1) mastering multiple disciplines 

• baseline for assessment: disciplinary originality or excellence
• good interdisciplinary research must fulfill existing methodological requirements and theoretical standards
• greater challenge for interdisciplinary scholars, because it's hard to be met expert and generalist criteria at the same time
• reception problems, incompatibility of different disciplinary standards

(2) emphasizing integration and synergy

• baseline for assessment: create a new model of excellence
• need to combine knowledge resources in order to develop an integrated product
• proponents of the view: within boundary-crossing organization, consultants, or practitioners

(3) critiquing disciplinarily

• views disciplinarily and interdisciplinary as strongly opposed: interdisciplinary as the force that diverts the discipline-driven direction of knowledge production
• undermine the prevailing status of disciplinary standards in the pursuit of a non-disciplinary, integrated knowledge system
• difficulties of evaluating interdisciplinary research will not be overcome by creating new quality standards for that type of research, but by transforming the prevailing ethnocentrism and mutual ignorance between disciplines
• quality judgment should be made by using external criteria but no recipe for how to do it

This framework helps to identify the relevant epistemic stakeholders, the functions and benefits of proposed research, as well as the methodological procedures for accomplishing the stated goals, which constitute the prerequisite for any evaluative act. It argues that these competing positions on interdisciplinarity shape assumptions about quality and how it should be evaluated, while the actual process of evaluation with various social, cognitive, and pragmatic aspects also plays an important role in quality judgments.