Measuring impact to improve conservation results

As the second most heavily forested Amazonian country, Peru has attracted a steady stream of programmes aimed at reducing greenhouse gas emissions from land-use change, the country’s largest source. Despite these conservation efforts, however, annual forest loss has increased over the past two decades, from about 84,000 hectares in 2001 to a record 200,000 in 2020.

A new study by the Center for International Forestry Research and World Agroforestry (CIFOR-ICRAF) shows that many conservation projects and programmes in the country had at best a modest effect on forest loss, while the impact on the well-being of the communities involved ranged from positive to negative.

Reasons included flaws in the design of the programme, failure to deliver on benefits promised to communities, or the location of projects in remote areas facing little risk of deforestation. A key conclusion of the study is that forest conservation projects and programmes have tended to lack well-designed impact evaluations that could provide key evidence to make them more effective.

The study raises questions for policymakers about whether the benefits of conservation projects outweigh their costs, whether the areas being protected are those facing the greatest risk of deforestation, whether planners are clear about the outcomes they expect and how to measure them, and whether the programmes also result in social and economic benefits to participating communities.

“There is little evidence about the effectiveness of public policies for forest conservation in Peru,” said Renzo Giudice, a senior researcher at the Center for Development Research at the University of Bonn in Germany, who co-authored the study with Manuel Guariguata, a specialist in tropical forest ecology and forest management for production and conservation and senior associate at CIFOR-ICRAF.

“We’re still flying blind with regard to impact evaluations,” Giudice added.

Forest conservation programmes often produce reports and statistics about the forest area conserved, the number of communities involved and the number of new protected areas — from national to local — created every year, but little information about their ecological, economic and social impacts, he said.

Measuring those impacts is even more critical as the goals of conservation programmes expand from focusing on keeping forests intact to contributing to the social and economic well-being of forest-dwelling communities. And although those outcomes may not be measured systematically as part of the project evaluation, they have come under increasing scrutiny from academics.

A literature search turned up 17 impact evaluations of various types of conservation programmes and projects in Peru, all in Amazonian regions, including some conducted as part of the CIFOR-ICRAF Global Comparative Study on REDD+.

The projects were varied, and included reducing greenhouse gas emissions from deforestation and forest degradation (REDD+) in Brazil nut concessions and community forests, payment for environmental services related to water, community forest monitoring, conservation-based conditional cash transfers for Indigenous communities as part of Peru’s National Forest Conservation Program, and comparisons of forest conservation in various categories of protected areas.

The types of areas targeted ran the gamut from national parks and reserves managed by Peru’s National Service of Natural Protected Areas (SERNANP), to other types of protected areas, Indigenous communities and Brazil nut concessions.

Some of the projects were based on conservation incentives, such as payments to local communities, while others focused on disincentives, such as fines for infractions resulting in deforestation. Several focused on creating conditions conducive to conservation, such as titling of communal lands. Nevertheless, the projects evaluated represented only a small percentage of the many conservation projects implemented in Peru, pointing to a need for more evaluations, the authors said.

Most of the evaluations measured deforestation as an indicator of the project’s effectiveness. About 70% of them found that deforestation decreased as a result of the project, although most of the decrease was slight, while 25% found no effect, and in one case, deforestation increased. Far fewer evaluations looked at social and economic impacts, and those that did showed mixed results, including one indicating that extreme poverty in communities increased, on average, after national protected areas were created nearby.

Two programmes for reducing greenhouse gas emissions from deforestation and forest degradation (REDD+) showed virtually no impact on either deforestation or the well-being of the communities involved.

In some cases, the lack of impact appears to be related to failure to follow through on benefits promised to the participating communities, Giudice said. In others, it could be due to flaws in project design.

In one case, for example, communities receiving monetary compensation for conservation as part of Peru’s National Forest Conservation Program, “appear to have chosen [to protect] areas with less deforestation pressure compared to other areas,” he said. “That means the [programme’s] ability to reduce deforestation was low, because deforestation was already low in the areas affiliated with the programme.”

Nevertheless, policymakers face trade-offs. Areas with higher deforestation threats tend to have lower poverty levels, so efforts to benefit communities with greater monetary poverty could lead to projects in areas that were better protected to begin with, he said.

Errors in impact evaluation design can also make a project appear to have had more or less of an effect than it actually did. Impact evaluations attempt to determine whether a project has an effect that would not have occurred if the project had not been implemented. Because that is impossible to measure directly, managers must compare the project area to another area of similar characteristics that serves as a control.

Control areas must be chosen carefully, however. If a project in a protected area is compared against an unprotected zone that includes a road, for example, the project might appear to have an outsize impact because the forest in the control area is intersected by a road or faces other risks that the project site does not.

Giudice and Guariguata conclude their study with a series of recommendations, including more — and more rigorous— impact evaluations, focusing efforts on areas where risk of deforestation is greatest, ensuring that proposed sanctions are actually carried out and that economic or other benefits really reach communities, and supporting oversight agencies responsible for sanctioning illegal logging, as those fines appear to be an effective disincentive.

For Giudice, the need to ensure that conservation funds are invested well makes well-designed evaluations an ethical imperative, as well as a practical matter.

“If impacts aren’t measured effectively,” he said, “it’s impossible to learn and to improve.”

Global Biodiversity Information Facility

Related Posts