Evaluating Environment in International Development

Front matter

EVALUATING ENVIRONMENT IN INTERNATIONAL DEVELOPMENT

This book provides novel and in-depth perspectives on evaluating environment and sustainability issues in developing countries.

Evaluating Environment in International Development focuses on the approaches and experiences of leading international organizations, not-for-profits, and multilateral and bilateral aid agencies to illustrate how systematic evaluation is an essential tool for providing evidence for decision-makers. Moving beyond projects and programmes, it explores normative work on the environment as well as environmental consequences of economic and social development efforts. This new edition reflects on the 2030 Agenda for Sustainable Development and Sustainable Development Goals and considers how they have influenced efforts in a wide range of countries and what the implications are for evaluation. It also explores ways in which Big Data and geospatial approaches might be utilized.

PART I Introduction and conceptual background

1 Evaluating environment in international development: an introduction Juha I. Uitto

2 A global public goods perspective on environment and poverty Rob D. van den Berg

3 Evaluation at the nexus: evaluating sustainable development in the 2020s Andy Rowe

4 Poverty, climate change and disaster risk reduction: too complex to evaluate?

Hazel Todd and David Todd vi

Contents

PART II Approaches and challenges in evaluating environment and sustainable development

Using big data and geospatial approaches in evaluating environmental interventions Anupam Anand and Geeta Batra

Multiple actors and confounding factors: evaluating impact in complex social-ecological systems Aaron E. Zazueta and Jeneen R. Garcia

Assessing progress towards impacts in environmental programmes using the field review of outcomes

to impacts methodology

Meta-analysis of climate mitigation evaluations Christine Worlen

A programme theory approach to evaluating normative environmental interventions Segbedzi Norgbey and Michael Spilsbury

From evaluation of joint programmes to joint evaluation of SDGs-ready interventions: lessons from the joint GEF-UNDP evaluation of the Small Grants Programme

Evaluating the poverty—environment nexus in Africa Michael Stocking

Small grants, big impacts: aggregation challenges Sulan Chen and Juha I. Uitto

Green economy performance of environmental initiatives in Latin America and the Caribbean Ronal Gainza and Simon Lobach

Evaluating international support for transboundary aquifer management programmes Alan Fox

Contents vii

15 Disaster risk management in the SDG era 278 Vijayalakshmi Vadivelu

16 The Consultative Group on International Agricultural Research approach to evaluation of climate change, environment, and natural resource management 299 Roberto La Rovere

FIGURES

2.1 4.1 4.2 4.3 4.4

5.1

5.2 5.3 5.4 5.5 5.6 7.1

7.2

7.3

7.4

Evaluation Criteria

Potential Effects of Climate Change on Poverty Reduction Some Major Disasters in Asia-Pacific 1970-2016

Initial Theory of Change for DIMSOG

Overlaps Between Disaster Risk Reduction and Climate Change Adaptation

Globally Distributed GEF-Supported Protected Areas were Overlaid with Sites of Conservation Importance

Geocoded Location Information for GEF Project Implementation Areas Data Used in the Uganda Case Study

Trends and Comparison

Study Area Map of the 12 PAs within Kenya Included in the Analysis Land Cover Classifications for the Reserve

The Relationship between the Key Elements in a Project Theory of Change

Steps in the Field-Based Review of Outcomes to Impacts Assessment Process

Steps in Conducting a Field Review of Outcomes to Impacts Analysis with Group or Individual Key Informants

Theory of Change for Intermediate State 3

38 63 66 70

114 122

Figures ix

7.5 Theory of Change for Intermediate State 1 127 7.6 Theory of Change for Intermediate State 2 129 7.7 Overall Review of Outcomes to Impacts Assessment of SEYMEMP

Progress towards Impacts 132 7.8 Overall SEYMEMP Theory of Change 135 8.1 Evaluation Framework of Tokle and Uitto (2009) 142 8.2 Evaluation Framework of Tokle and Uitto (2009) with Stakeholder

Groups 143 8.3 Example for Mapping the Barriers in the Barrier Circle Diagram 150 8.4 The Intervention Circle 151 8.5 Barrier Circle and Intervention Circle Diagram 152

9.1 A Set of Generic Results Chains, which can also be Termed ‘Causal

Pathways’, Impact Pathways’ or a Theory of Change’ 161 9.2 A Schematic Single ‘Impact Pathway’ Showing Intermediate States,

Assumptions and Impact Drivers 162 9.3 ‘Synergies Decisions’ Theory of Change (Shown in Part) 166

9.4 Theory of Change — Joint Geophysical Imaging Methodology for Geothermal Reservoir Assessment 169

9.5 A Generalized Representation of Impact Pathways Relating the Centre for International Forestry Research Criteria and Indicators Project 171

9.6 Major Impact Pathways through Forest Stewardship Council Certification Processes 172

9.7 Theory of Change — Common Problems in Capturing Performance at the Programme Level 175

12.1 SGP Management Diagram 223

TABLES

3.1 4.1 4.2

4.3

7.1

7.2 7.3

13.2 13.3

Observations from CES Sustainability-Ready Stocktaking 50 Some Estimated Impacts of Disasters in the Region 67

Common Types and Methods of Evaluation at Various Levels of Intervention 71

The Use of “Nested Theories of Change” to Help Evaluate Different

Types of Issue 74 Definitions of Theory of Change Elements in the Outcomes—Impacts

Pathways 112 Field Review of Outcomes to Impacts Rating System 115

Global Environmental Benefits (to Biodiversity) for the SEYMEMP

Project 118 The SEYMEMP Outcomes—Impacts Theory of Change 120 Outcomes—Impacts Assessment Findings for Intermediate State 3 123 Available Information on Global Environmental Benefit Status 130 Example for Barrier Removal Strategies 147 SGP Portfolio by Focal Area 221 Grading System to Assess the ‘Green’ Level of Performance of

Environmental 241 Modalities of Compensation in the PES, Costa Rica 242

Cumulative Contributions to FONAFIFO from 1998 to June 2011 243

Tables xi

13.4 Contributions to FONAG by Constituents and Donors 245 13.5 Estimated Payback Periods for Domestic Solar Water Heaters, Barbados 246 13.6 Type of Funding by Initiative and Phase 250 13.7 Green Performance of the Initiatives 253

13.8 Strengths, Weaknesses, Challenges, Opportunities Matrix 256

Foreword

To understand and appreciate the importance of this book, I will offer readers my perspective on two things: (1) the significant contributions to evaluation found herein and (2) the global context within which those contributions occur. Let me begin with the global context.

The first edition of the book came out in 2014 before the Sustainable Development Goals (SDGs) had been adopted but when it was clear that the Millennium Development Goals (2000-2015) would not be met. We are now a third of the way through the SDG commitments (2015-2030) in which the trajectory towards attainment of those goals is dismal. Nowhere is this trend towards failure starker than on climate. As I write this in June 2020, global temperatures in May were the warmest on record: 0.63°C warmer than the 1981-2020 average for May; warmer by 0.05°C than May 2016, the previous warmest May; and warmer by 0.09°C than May 2017, the third warmest May. Earth’s carbon dioxide levels hit record high, despite a coronavirus-related emissions drop. There is more carbon dioxide in the air now than at any time in 3 million years. The Scripps Institution of Oceanography and the National Oceanic and Atmospheric Administration (NOAA) reported that the amount of CO, in the air in May 2020 hit an average of slightly greater than 417 parts per million (ppm). This is the highest monthly average value ever recorded and is up from 414.7 ppm in May of 2019.

Such specific data are usually avoided in a book because they will be out-of-date by the time of publication and subsequently whenever readers come to the book. I have included these data as an invitation to readers to update the trendlines to provide an immediate and timely context for the critical importance of evaluating environment in international development—and, in so doing, affirm that we are in the midst of a deepening global emergency.

Foreword xiii

Global emergency

Each year the Oxford Dictionary editors select a word of the year. The word for 2019 was “climate emergency.’

Climate emergency is defined as “a situation in which urgent action is required to reduce or halt climate change and avoid potentially irreversible environmental damage resulting from it.”

This year, heightened public awareness of climate science and the myriad implications for communities around the world has generated enormous discussion of what the UN Secretary-General has called ‘the defining issue of our time”....

Usage of the phrase climate emergency increased steeply over the course of 2019, and by September it was more than 100 times as common as it had been the previous year....

This data is significant because it indicates a growing shift in people's language choice in 2019, a conscious intensification that challenges accepted language use to reframe discussion of ‘the defining issue of our time’ with a new gravity and greater immediacy.

(Oxford Languages, 2019)

The Intergovernmental Panel on Climate Change has warned that 2030 is likely to be a point of no return on climate change. But the global emergency involves more than climate, as this books makes clear.

Beyond climate

In mid-June, coronavirus infections worldwide exceeded 9.1 million with nearly one-half million deaths. Worldwide economic depression looms from the pandemic. The global pandemic has given rise to another new word, infodemic, meaning an overwhelming amount of information about a problem including wide and rapid spread of misinformation. Also part of the global context as this is being written are worldwide protests against racism and injustice sparked by the murder in May, 2020, of George Floyd, an unarmed African-American man killed by a white police oficer in Minneapolis. These global patterns are deeply interconnected: the growing climate emergency, the global pandemic, the exacerbating worldwide infodemic, and protests in cities around the world against systemic and structural injustices.

Beyond just evaluating environment: a nexus perspective

I have provided this context to emphasize and highlight the significant perspective offered in this book. This book is about much more than evaluating environment. It is about environment at the nexus of humanity’s future. Evaluating sustainability xiv Foreword includes ecosystem health, more just and equitable societies, and evidence-based decision-making. The sustainability framework offered here integrates social, economic, and environmental goals and criteria. This holistic approach requires upgrading and enhancing environmental evaluation, but not as an isolated function. The fundamental framework offered here is one of nexus:

e the nexus of social, economic, and environmental interventions. e the nexus of environment and development.

e the nexus of development and sustainability.

e the nexus of human and natural systems.

e the nexus of environmental degradation and poverty.

e the nexus of evidence and decision-making.

e the nexus of local and global initiatives;

e the nexus of projects, programmes, and larger-scale initiatives;

e the nexus of multi-stakeholder governance; and

e the nexus of evaluation and action.

In essence, this second updated edition of the book, Evaluating Environment in International Development, demonstrates that a sustainable environment is at the centre of humanity’s future. Everything is ultimately connected to the environment. The book, then, is about evaluating those interconnections not environment as a stand-alone, isolated, or siloed focus.

Juha I. Uitto, the book’s editor, as director of the Independent Evaluation Office of the GER and Rob van den Berg, president of the International Development Evaluation Association (IDEAS) in 2019, gave voice and leadership to this perspective in crafting with others a Declaration on Evaluation for Transformational Change adopted by participants at the 2019 IDEAS conference in Prague. One of the ten items in that Declaration asserts:

Focus on sustainability. In all our evaluations, we commit to evaluating for social, environmental and economic sustainability and transformation, including by assessing contextual factors and systemic changes. We commit to assessing and highlighting, in all evaluations, unintended negative social, economic and environmental effects.

That is the perspective of this book. It is a breakthrough perspective for evaluation going forward in the context of interconnected global emergencies of climate, pandemic, infodemic, economic turbulence, and massive inequalities and injustices. Evaluating environmental sustainability is the thread that can intersect with, illuminate, and connect these global challenges and guide humanity’s responses. Time is of the essence. The point of no return looms large.

Michael Quinn Patton Author of Blue Marble Evaluation: Premises and Principles