This is the TIAS Newsletter
View this email in your browser
TIAS Quarterly

No. 02/2017 (June)
The Newsletter of
The Integrated Assessment Society (TIAS)

Visit TIAS
Keit Trysh

In this Issue

The Society

The Integrated Assessment Society is a not-for-profit entity created to promote the community of inter-disciplinary and disciplinary scientists, analysts and practitioners who develop and use Integrated Assessment (IA). The goals of the society are to nurture this community, to promote the development of IA and to encourage its wise application.

Integrated Assessment can be defined as the interdisciplinary process of integrating knowledge from various disciplines and stakeholder groups in order to evaluate a problem situation from a variety of perspectives and provide support for its solution. IA supports learning and decision processes and helps to identify desirable and possible options for addressing the problem. It therefore builds on two major methodological pillars: approaches to integrating knowledge about a problem domain, and understanding policy and decision making processes. IA has been developed to address issues of acid rain, climate change, land degradation, water and air quality management, forest and fisheries management and public health.



Integrated Assessment ‘in between logics’: practical concerns and learning by doing

Willemijn Tuinstra, Independent advisor on  knowledge  development  for  environmental  policy
Eva Kunseler, Department of Information, Data and Methodology, PBL Netherlands Environmental Assessment Agency


How can participatory forms of knowledge production be connected to traditional assessment repertoires?  Looking back at the feature articles of previous issues of the TIAS newsletters it is noteworthy that they all have one ‘feature’ in common. They all stand for change, for approaching practice differently, whether improving the use of the Sustainable Development Goals (SDG) -indicators, supporting sustainable production, experimenting with trans-disciplinary approaches, setting up water-related curricula in higher education or building capacity for water management etc. Very often there is also an idealistic and prescriptive tone in these articles. We see that things should be organised better, integrated better or done in more deliberative settings. The articles show that we believe that this can be done in particular ‘new’ ways and we are convinced that people (scientists, politicians, the general public) should change their behaviour and their ways of working to achieve the things we believe are so much needed to achieve real change that promotes sustainable development. Of course, in the tone and the presentation of our projects we then also provide successful examples and we report on the things which went well and are promising. This of course works very well in inspiring others to try the same, or even more.

What is ‘new’?

What we (the authors of the current feature article) wondered about was why we (the TIAS community) still refer to our approaches within integrated assessment as ‘new’? While TIAS and its predecessors  have worked with these ‘new approaches’ for more than 20 years, why are they still seen as ‘new’ and challenging and different? Because these approaches are still challenging and different: integrated assessment in a participatory mode or when using transdisciplinary approaches is not mainstream. The theory sounds good and is inviting, yet in practice is often not easily implemented.  As this article argues, one of the reasons for this is that IA practitioners have to operate ‘in between logics’, using logic in the plural. They intend to work in a reflexive way following the logic of ‘Mode-2’-science, where a more traditional pattern following the logic of ‘Mode 1’-science is still expected. In ‘Mode-1’ science produces objective facts which it passes on to policy making. Mode 2-science involves a non-linear iterative process of co-creation whereby all actors involved together produce knowledge in order to take action.

The stories we do not hear

Indeed, what we read about less (though admittedly TIAS articles are relatively open in this respect) are the ‘challenging’ parts, the failures and what did not work. We always try to tell a coherent story and to make our actions sound logical. If there are failures they should be somehow streamlined to fit into the picture. Or we simply leave them out. How often did you try to start a participatory project, which didn’t work out as well as in the results you read in other people’s papers? Participants didn’t show up, the scenarios they produced weren’t as inventive as you had hoped for, or they looked far too much like each other, so there was little to write and to compare. Or at the end you did your exercise mostly with students or colleagues, because you didn’t have the resources to invite enough externals. Or, if this didn’t sound familiar, what about the projects that never started up, because the organisational constraints were just too overwhelming? No money, no time, no expertise, no interest within the management team. Or your own worries: participatory integrated assessment will not be taken seriously, the results will not be credible, it is not my place to take a participatory role, the ministry doesn’t want us to do this research because we trespass on its territory. It is particularly these practical concerns that are at the heart of a recently completed PhD thesis by Eva Kunseler, an Integrated Assessment practitioner at the Netherlands Environmental Assessment Agency (PBL).

In between logics: ‘reflexive’ ways of working bounded by ‘modernist’ structures

The thesis examined how scientific policy advisers in a government-expert agency attempted to connect participatory forms of knowledge production to their traditional assessment repertoires. The thesis illustrates how practitioners at the PBL Netherlands Environmental Assessment Agency found themselves ‘in between logics’: they are inclined to adopt reflexive (Mode-2) ways of working, while they are bounded by the limits imposed by modernist (Mode-1) structures, norms and ways of working. PBL traditionally works for a group of professional representatives and government departments. Its role is traditionally seen as providing objective and neutral assessment of policy goals and options. Under the influence of changes in and the demands of society in the last decade, PBL has been shifting towards more reflexive ways of working. This has led PBL to formulate guidance documents for uncertainty assessment and stakeholder participation. Their practical implementation is a huge challenge: PBL researchers need to develop capacity and skills to bring it in practice and are confronted with traditional expectations of their role.

The two logics of scientific advice are essentially incommensurable: their representations of the quality of scientific advice and the roles of experts in society are in conflict (see table 1). Fundamental differences between them are related to different societal and political understandings of the role of expertise, as well as to different ontological and epistemological foundations. While modernist logic understands reality as singular and objectively knowable, reflexive logic understands reality as pluralistic and socially constructed.

  Table 1: Attributes of a modernist and a reflexive logic of expert advising

  Purpose of expert advising Role of expert organisations at the science-policy interface Quality principles
for expert advising
Technical methods are employed to produce empirically confirmed and logically consistent statements to inform evidence-based decision making
 Experts mediate domains of  science and policy and work to  keep them apart
Neutrality; detachment from politics; scientific autonomy
Reflexive logic
Participatory and interpretive methods are employed to integrate knowledges and perspectives to inform societal problem-solving
Experts bring different perspectives and rationales in line and organise productive interactions between them
Inclusiveness of perspective plurality; uncertainty assessment; extended peer review; organisational learning; reflective practice


Practical concerns

The thesis was written against the backdrop of challenges to credibility and intensive debates about the legitimacy and authority of government expert agencies at environmental science–policy interfaces, such as the IPCC in the field of climate change. Public trust seemed lost after errors had been found in the regional assessment part of the IPCC Fourth Assessment Report. PBL was tasked by the Minister of Infrastructure and the Environment with reviewing the IPCC Fourth Assessment Report. For this purpose, PBL organised an extended peer review process in which critical peers and public parties were invited to contribute to the review of potential mistakes. A large number of employees did not think this was a legitimate role for PBL. However, a small group saw it as a challenge and important experiment for exploring a new form of reflexive science.

The starting point for the thesis was the observation that previous studies have normally not paid attention to the practical concerns which govern and affect practitioners employed in government expert agencies. How do researchers in practice cope with the challenges of working with two different logics?


Worries about quality

One of the findings was that PBL practitioners are concerned about the quality of knowledge produced in participatory assessment: is it improving the relevance and impact of the study? Will participation generate reliable knowledge? On the other hand, they are concerned about their role performance. Can we safeguard our independence when involved in participatory processes? How does participation affect our identity as an objective expert?
When PBL practitioners attempt to connect participatory forms of knowledge production to their traditional assessment repertoires, they essentially wonder how participatory activities may contribute to the production of independent, scientifically sound and policy-relevant knowledge.  Practitioners and their peers adhered to both modernist and reflexive logic of scientific advice simultaneously, and mobilised one or the other (not always consciously) according to the situation. This resulted in quite a messy situation, which caused confusion but at the same time appeared to be productive.

The findings in the thesis illustrate how the incommensurability of different logics produces inconsistencies in practice. Among others, these inconsistencies become explicit in positive versus negative perceptions of the impact of stakeholder participation (in ‘the Sustainable City’ and the ‘Nature Outlook’ projects), and in contradictory evaluation styles and approaches (in policy evaluations) and various interpretations of what it means to be objective.  For the sake of brevity we focus on the quality perceptions of stakeholder participation in the remainder of this article.

Credibility of stakeholder participation

The three core values of policy relevance, independence and scientific soundness guide the daily operations of PBL researchers. Interpretations and perceptions attributed to the credibility, relevance and legitimacy of a study differed considerably across and within policy assessment settings. The research identified the different positive and negative perceptions attributed to, for example, the credibility of a participatory foresight study by practitioners and their peers. In this study practitioners and peers mobilised reflexive logic when pointing out positive credibility implications of stakeholder participation, such as improvement in the quality of knowledge, while they were inclined to mobilise modernist logic when highlighting impacts on credibility. They expressed, for example, practical concerns about the rigour of impact calculations, or the lack of a systematic approach to integrating the various analytical and deliberative activities. This finding indicates how practitioners struggle to fruitfully implement their participatory assessments in a reflexive manner. They aimed for the production of integrative and socially-robust knowledge, but encountered concerns regarding the scientific validity of the outcomes, which is a principal quality criterion imposed by institutionalised modernist logic. Moreover, modernist and reflexive logic were mobilised interchangeably during assessment processes under the influence of, among others, path-dependent choices, dynamics in the project setting and disciplinary understandings of knowledge production. We’ll elaborate on the latter.

Disciplinary understandings

Stakeholder participation triggers interdisciplinary confusion within the PBL organisation about the standards for quality control. Different quality perceptions, e.g. of how to assess the validity of stakeholder knowledge, originate from different disciplinary quality standards like triangulation in the social sciences and causal inference in the natural sciences. Such disciplinary discrepancies have remained largely ignored and implicit in PBL assessment practices. Traditional quantitative foresight approaches were mixed with qualitative discursive approaches, while the inherently different quality perceptions associated with these foresight cultures remained largely unnoticed. This finding suggests that both the modernist and reflexive logic of scientific advice to government is being mobilised without fundamental awareness of the differences between them in terms of epistemological underpinning. How the different quality perceptions attributed to the impact of stakeholder participation, whether positive or negative, related to a particular type of knowledge or way of producing this knowledge and assuring the quality hereof, appeared not to be a subject of debate. Yet, knowing that these different epistemic cultures co-exist is crucial to understanding and bridging, potential differences in conceptions of quality in an effective manner.

Conclusion: Learning by doing

Hence, the adoption of participatory assessment in government expert agencies is complex and is less like the replacement of one logic of scientific advice by another and more like an ongoing recombination of assessment approaches, roles and principles in day-to-day practices. The difference is that – under changing social order – the reflexive logic of scientific advice to government tends to become the more encompassing notion.

Our suggestion for practitioners in expert agencies is, therefore, to actively engage in processes of ‘learning by doing’ on the job. A ‘learning-by-doing’ attitude may stimulate practitioners to ‘problematise’ what is taken for granted. They may open up to new interpretive possibilities (e.g. via training, inspirational lectures, life-world experiences, experiments, reflective activities), while they assure a sense of continuity with traditional expectations and routine ways of working by taking on the ‘on the job’ lessons in their regular work.

Photo: Ulli Meissner,


A Dialogue Session: “Understanding learning for governing sustainability transitions” at the International Sustainability Transitions Conference (IST2017) in Gothenburg, Sweden

As another activity of the TIAS Learning Community, a well-attended and insightful dialogue session took place at the IST2017 conference, organized by Johannes Halbe, Barbara van Mierlo, Geeske Scholz, Joanne Vinke-de Kruijf and PJ Beers. The session began with two introductory talks on (1) opportunities for knowledge transfer from well-developed learning traditions (by Barbara van Mierlo) and (2) the role of social processes, leadership, and ownership (by Geeske Scholz). After these presentations, John Grin and Heila Lotz-Sisitka reflected on the potential of a learning approach for transition governance and transformative education. The discussion that followed was organized as a fishbowl conversation, i.e., members of the audience entered the stage to raise questions or provide comments. Issues that were raised and discussed included why transition scholars tend to use learning concepts often in an imprecise way. A need for integrative frameworks and knowledge translation was identified to address the latter issue. Discussions also revolved around what is needed for learning to occur and the role of power. Moreover, the learning community was challenged to define the practical utility of learning concepts in transition management. This challenge shows again the need for an intensive dialogue with transition scholars to improve mutual understanding as well as the importance of case study research to demonstrate the practical benefits of a learning approach.

Joanne Vinke-de Kruijf, TIAS Secretary

Towards standards for IA
A TIAS webinar on principles and standards took first steps towards potential standards for Integrated Assessments. It considered three high quality overviews, as follows. BellagioSTAMP, presented by Laszlo Pinter (Central European University and IISD): a set of principles, normative on what it takes for an assessment to be regarded as ‘integrated’, for example, in terms of thematic scope, time horizon and guiding vision, as well as participation and capacity to do another iteration in due time. Thus, the Bellagio principles are firmly normative. At the same time, they are highly aggregated in order to reflect the common gene in a wide range of assessment practices. Later articles helped to illustrate the Bellagio principles with examples from various lines of work. Interestingly, the Bellagio principles were revised after a decade, in light of much expanded practice, by a largely new team. The principles benefited greatly from this refresher and became shorter and more crisp.

Draft Integrated Environment Assessment Guidelines of UN Environment (formerly UNEP) were presented by Pierre Boileau (GEO head, UNEP). In particular, their purpose was explained, namely to position the range of assessments produced by UN Environment (UNEP) in a framework of consistent guidelines. That range extends from the global environment outlook series to so-called rapid response assessments. The guidelines put forward a decision tree: which combination of issues and constraints indicate which type of assessment to be produced. This is anchored in a theory of change outlined in the guidelines. The present guidelines build on the 2007 IEA training manual for the Global Environment Outlook, by UNEP and IISD. Interestingly, the 2007 manual was originally intended to assist the compilation of GEO main reports but found its widest application in the making of regional GEO reports and associated capacity building. It is hoped that the present draft guidlelines will be road tested by the authors of the global edition of GEO-6.

Soon-to be published findings of the FOGEAM project were presented by Martin Kowarsch and Pauline Riousset (Mercator Research Institute). This extensive project focuses on effectiveness of global environment-related assessments, in particular the assessments from 2005 onward, when their emphasis shifted from problem diagnosis to exploration of options. In this project, impact mechanisms were also conceptualized. Discourse analysis, document analysis and many interviews were conducted. The large body of empirical material of the project was also fed by eight workshops. The project positions global environment assessments as an element in deliberative processes about policy alternatives in light of societal goals.

In a brief discussion on next steps, one plea was widely supported, namely that any development of IA standards (be it principles, guidelines, product/process criteria, or even an ISO-type standard) should work alongside concrete evaluation of specific assessments. The webinar recording is available on

Jan Bakkes, TIAS Vice-President
Copyright: Ulli Meissner

IA News


SDG indicators: Report of a Dresden Nexus Conference session

Laszlo Pinter (TIAS Advisory board member, Central European University and International Institute for Sustainable Development) and Joanne Vinke-de Kruijf (TIAS Secretary, University of Twente)

In 2015, the member states of the United Nations agreed on the 2030 Agenda for Sustainable Development and a set of Sustainable Development Goals (SDG) applicable to all UN Member States. As part of the follow-up and review mechanisms of this agenda, member states are encouraged to conduct regular reviews of progress made towards achieving the SDGs at the national and subnational levels. These reviews are country-led and country-driven. They aim to provide input to the annual High Level Political Forum (HLPF). Against this background, the 2nd edition of the Dresden Nexus Conference (Dresden, 17-19 May 2017) featured a session on SDG monitoring and reporting with particular attention to indicators. The content of the session followed the topic of two recent TIAS webinars on SDGs that explored the role of IA in implementing the SDGs at the conceptual level and looked at the role of IA models. Hence, we would like to share some of the session results with the readership of the TIAS Newsletter.

The session started off with a presentation by Tamara Avellan (UNU Flores). In the study she introduced, Tamara and colleagues prepared an inventory of indicators used for tracking water quality. As a next step, they aim to investigate the practical applicability of these indicators and to identify capacity building needs. At the moment, no less than 474 water quality indicators are in use. Their study both showed that SDG performance measurement can build on a good base of existing tools and methods, but also illustrated their diversity. While helpful and necessary to measure what matters in different contexts, diversity also represent a challenge for standardization and comparability. The same obviously applies to SDG indicators.

Christian Kroll (Bertelsmann Stiftung) provided a presentation of SDG country performance index and dashboard they released last year. The index and dashboard are based on a selected set of indicators and show, respectively, the overall performance of countries on all indicators included and how well countries perform on specific goals. The index contributes to raising awareness, promotes the exchange of best practices and can be used as a research tool. Researchers at the Foundation are currently working on a new report, which will focus on international spillover effects (i.e. countries achieving their own goals at a cost to other countries e.g., by outsourcing polluting production or exporting waste).

Laszlo Pinter (CEU and IISD) introduced IISD’s new web-based tool to track SDG indicator reporting practices of countries and cities. Piloted for nine countries and selected Canadian cities, the tool provides a high-level overview of reporting on various SDGs and the similarity of indicators to the official set defined by the UN Statistical Commission (UNSC). Analysis of the data shows that while countries are good at reporting socio-economic indicators, reporting of environmental indicators are often lacking. The use of indicators reflects both the political commitment to the implementation of the SDGs and capacity constraints.

Finally, Sami Pirkkala (Prime Minister’s Office of Finland) provided an account of the progress made with the implementation of the SDGs in Finland. Finland is a forerunning country that has over twenty years of experience with implementing sustainable development using various policy frameworks. Sami stressed the role of national governments, who are responsible and accountable for the implementation and monitoring of SDGs and national parliaments. In Finland, a national implementation plan was recently submitted and will be reported on on an annual basis by the government to Parliament. Every four years, there will be an independent review and an evaluation of national implementation. Sets of indicators, which are linked to existing institutions, are being developed to support monitoring. Some of these indicators are global whereas others are country-specific. As for implementation at the regional and the local level, some cities in Finland are very proactive creating a huge potential for city-to-city learning. According to Finland’s experience, indicators are very much wanted but less often used.

Following the presentations there was a lively discussions about the use of indicators and indices. The index and dashboard developed by the Bertelsmann Stiftung was applauded but some of its aspects also challenged. For the Finnish government the index proved to be very useful since it provided insights into where Finland is doing well. However, participants questioned the equal weighting of goals and raised the need for making sure that low ranked countries are not demotivated. Also, as long as very few indicators are currently in use for the goals covered by the index, its reliability can be questioned. This also raises the question how to avoid cherry-picking of indicators.

Several suggestions were raised with respect to communicating and using the SDG index and similar indices.
  • When presenting an index pay attention not only to performance but also to efforts and actual progress made.
  • Provide users with an online platform where they can apply user-specific weightings (as is the case with the Better Life Index of the OECD or the the earlier Dashboard of Sustainability tool of IISD and the JRC).
  • Attach stories to the indicator scores to complement numbers with qualitative information in the form of a narrative.
  • Explore the use of indicators to support backcasting (reasoning back from future goals). Some countries are currently experimenting with this approach.
  • Overall, developing fully functional SDG indicator suites and an SDG index will require an extended process of experimentation, testing and learning.
For further information please refer to the Nexus conference website or contact the respective speakers.
Photo: Marten van den Heuvel (adapted)

New Publications

Kowarsch, M., Jabbour, J., Flachsland, C., Kok, M.T.J., Watson, R., Haas, P.M., et al. (2017). A road map for global environmental assessments. Nature Climate Change 7(6), pp.379-382. doi:10.1038/nclimate3307 (view-only link provided by Nature journal:
This publication provides a synthesis of the research project, "The Future of Global Environmental Assessment Making (FOGEAM)," initiated by UN Environment and MCC Berlin in 2013. Increasing demand for solution-oriented environmental assessments brings about significant opportunities and challenges at the science-policy-society interface. Solution-oriented assessments should enable inclusive deliberative learning processes about policy alternatives and their various practical consequences.

Pauliuk, S., Arvesen, A., Stadler, K. and Hertwich, E.G., 2017. Industrial ecology in integrated assessment models. Nature Climate Change, 7(1), pp.13-20.
Abstract: Technology-rich integrated assessment models (IAMs) address possible technology mixes and future costs of climate change mitigation by generating scenarios for the future industrial system. Industrial ecology (IE) focuses on the empirical analysis of this system. [The authors] conduct an in-depth review of five major IAMs from an IE perspective and reveal differences between the two fields regarding the modelling of linkages in the industrial system, focussing on AIM/CGE, GCAM, IMAGE, MESSAGE, and REMIND. IAMs ignore material cycles and recycling, incoherently describe the life-cycle impacts of technology, and miss linkages regarding buildings and infrastructure. Adding IE system linkages to IAMs adds new constraints and allows for studying new mitigation options, both of which may lead to more robust and policy-relevant mitigation scenarios.

Job Openings

IASS Potsdam is seeking a research associate (50%) to contribute to the “Umweltleitlinien deutscher Arktispolitik (Environmental guidelines of German Arctic policy)” project. In this project, environmental policy guidelines for activities in the Arctic as well as research papers on topics related to the Arctic environment will be developed in close cooperation with stakeholders from politics, business and society. Deadline for applications is July 14, 2017. More information can be found at:
Two PhD Positions are available in the junior research group “GLOCALPOWER – funds and tools for an African energy transition" at the University of Kassel. The project will focus on the transformation of energy systems from the perspectives of global environmental governance and political economy. It combines research on international organisations and green finance institutions on global level as well as local case studies in Ghana, South Africa, and Zambia. Deadline for applications is July 4, 2017. More information:
Two Postdoc Positions in Complex Social-ecological Systems are open in the Complex Social-Ecological Systems group at Oregon State University beginning autumn 2017 for two years each. For more information visit:

A Research Associate position is available at Stockholm Environment Institute (SEI), Bangkok, Thailand. SEI is looking for a Research Associate to undertake research activities and develop new research projects that focus on the centrality of human rights in climate change adaptation and natural resources management. Application Deadline: 16th July 2017. More information:
A Research Data Assistant position is available at Stockholm Environment Institute and the York School of Management in York, UK. This is a full time position for a 3 year fixed term. The Data Assistant will support interdisciplinary research on sustainable and resilient supply chains. This position will involve data preparation, analysis and management duties as well as develop methods and indicators for the analysis of risks and impacts associated with producer to consumer systems with a focus on food commodities. Application deadline: 10th July 2017. More information:



12-19th July 2017, Advanced Webinar: Remote Sensing of Drought. The webinar is organized by the US National Aeronautics and Space Administration (NASA) and will address themes related to capacity development, climate change, early warning, food security and agriculture, GIS and mapping and risk identification and assessment, and water-related hazards. More information

Photo: U. Meissner

26-29 September 2017, Healthy-Polis: Harnessing opportunities for improving urban health and wellbeing through environmental sustainability workshop. Coimbra, Portugal.

This workshop is part of the 14th International Conference on Urban Health - Health Equity: The New Urban Agenda and Sustainable Development Goals. The purpose is to identify and discuss innovation and standardisation of epidemiological, exposure and health impact assessment methods. In addition, the workshop will provide a forum for international, multi-disciplinary research collaboration on urban environmental health, health equity and sustainability. It will also contribute to the evaluation and promotion of environmental interventions, focusing on urban life-course approaches in response to environmental change. Finally, the workshop aims to promote integration of public health research and practice into urban research and data collection agendas, to address UN Sustainable Development Goal 11: Make cities inclusive, safe, resilient and sustainable. More information


TIAS Quarterly

TIAS Quarterly is the newsletter of The Integrated Assessment Society.
ISSN: 2077-2130
Editor: Caroline van Bers
Associate editors: Anna-Lena Guske, Caroline Lumosi, Joanne Vinke-de Kruijf,
Layout: Worldshaper photography & design - Fabian Heitmann, Caroline van Bers

Postal Address:
TIAS Secretariat
c/o Institute of Environmental Systems Research (USF)
Barbarastr 12
Osnabrück University
D-49069 Osnabrück, Germany

Phone: +49 (0)541 969 2505
E-Mail: info[at]

Become a TIAS member

TIAS Membership fees

Individuals: € 50 / US$ 65 annually
Developing country € 35 / US$ 40

Students: €15 / US$ 20 annually
Developing country €10 / US$10

Institutions: € 200 / US$ 250 annually;
Developing country € 150 / US$165
Copyright © 2017 The Integrated Assessment Society e. V., All rights reserved.