How the cost of a missed clearing changes the choice of forest map in Guyana
A decision analysis of two maps using a published reference sample, 2001–2017
In plain English
A satellite map can miss open land or send a ranger to check a patch that still has trees. In this sample, Guyana’s map missed 147 fewer hectares of open land and raised alarms on 126 more hectares of forest than the global map. Which choice suits a field team depends on what each mistake costs.
Weighted cost of map errors at three assumed costs per miss
Analysis steps
- Read reference labels
- Count the two error types
- Assign costs per hectare
- Compare maps
Read the full paper
Two forest maps agree with reference labels on more than 99% of the assessed area. Can that figure tell a monitoring team which map to use? Using the published Guyana evaluation, we assign separate weights to missed and false non-forest labels. The national map has fewer weighted errors when a missed hectare costs more than six sevenths of a falsely flagged hectare. Below that threshold the global map has fewer. The threshold describes the historical sample and the two stated costs.
Research question
How much weight must a team give to a missed hectare of non-forest before the national map has a lower error cost than Global Forest Change?
Data and methods
Galiatsatos and colleagues published a common reference assessment for the two maps [1, Table 3]. Their study adapted Global Forest Change, a satellite product developed by Hansen and colleagues [2], to Guyana’s forest definition. We extracted the four cells for each map. All entries measure hectares in the assessed sample. Forest and non-forest describe land cover over the cumulative comparison period.
A miss is a reference hectare of non-forest that the map labels forest. A false flag is a reference hectare of forest that the map labels non-forest. Set the cost per hectare of a false flag to one unit. Let r be the nonnegative cost per missed hectare in the same units. For each map, the total cost is r times missed hectares plus falsely flagged hectares. We compare the totals at three illustrative weights and solve for their intersection.
Results
Across 96,127 assessed hectares, the global map missed 522 hectares and falsely flagged 114. The national map missed 375 and falsely flagged 240. Among the 9,236 reference hectares of non-forest, the share missed fell from 5.65% to 4.06%. The national map therefore exchanged 126 additional false flags, measured in hectares, for 147 fewer missed hectares.
The cost difference, national minus global, is 126 − 147r. It equals zero at r = 6/7, approximately 0.857. With equal weights the national map has 615 error units and the global map has 636. Doubling the cost of a miss gives 990 and 1,158 units, respectively. At half the cost, the ranking reverses.
Discussion
A team that spends most of its time checking false alarms can favour a different map from a team tracing loss that an initial screen missed. The crossover gives both teams a question they can answer with local records. How much staff time or field expense does each kind of error incur per hectare? The comparison assumes that cost grows in direct proportion to area.
Limitations
The table combines 2001–2017. It cannot locate an individual clearing, date it or identify its cause. Hectares within one clearing share conditions. A trip cost per site would require event boundaries and distances. The published aggregate lacks the sampling-unit records needed to calculate a design-based confidence interval for the crossover [3]. These estimates do not rank current product versions.
Reproduce the analysis
Transcribe Table 3 of [1]. Read rows as map labels and columns as reference labels.
Check that each matrix sums to 96,127 ha and each reference non-forest column to 9,236 ha.
For each r in 0.5, 1 and 2, calculate 522r + 114 and 375r + 240. Equate these expressions to find the crossover.
Data table
| Map | Forest correct (ha) | Missed non-forest (ha) | False non-forest (ha) | Non-forest correct (ha) |
|---|---|---|---|---|
| Global Forest Change | 86777 | 522 | 114 | 8714 |
| Guyana-MRV | 86651 | 375 | 240 | 8861 |
References
- Galiatsatos et al. (2020). An Assessment of Global Forest Change Datasets for National Forest Monitoring and Reporting. Table 3.
- Hansen et al. (2013). High-resolution global maps of 21st-century forest cover change.
- Olofsson et al. (2014). Good practices for estimating area and assessing accuracy of land change.
Data credits · Galiatsatos et al. (2020), Table 3. CAIA weights and cost calculations. CC BY 4.0
Download the data and analysis code (ZIP)
Working paper · Internal review · How we check the research