Agriculture, forestry, and other land use is a key driver of climate change, accounting for 23% ( GtCO2eq ) of total anthropogenic emissions of greenhouse gases during 2007-2016 (IPCC_land_use_2019), largely driven by deforestation and forest degradation. Deforestation does not only release carbon (e.g., through slash-and-burn), but also destroys a multitude of other forest ecosystem services: preserving biodiversity, counteracting flooding and soil erosion, filtering water, and offering a livelihood for the local population.
The causes of deforestation are mostly economically driven: expansion of commercial or subsistence agriculture, logging, fuelwood collection, or livestock grazing (Hosonuma_2012)). To counteract the economic incentives, payments for ecosystem services (PES) (Wunder_2007) are increasingly (Donofrio_2019) provided to forest conserving or restoring landowners by international stakeholders (e.g., through the governmental UN-REDD program (Gibbs_2007) or the commercial voluntary carbon market (Donofrio_2019)). However, current methods for monitoring, reporting, and verification (MRV) of the landowner-provided forest ecosystem services are either based on 1) on-ground inspection, which is too expensive (USD 20-30k), delayed (up to two years), corruptible, and biased (GoldStandard_2017), 2) satellite, which is low-cost, but limited to the binary verification of forest/no-forest cover (Hansen_2013), or 3) drones.
Recent works haved proposed low-cost and accurate MRV via drones (Dao_2019; daogainforest; Lutjens_2019)
. Specifically, these works propose algorithms that estimate forest ecosystem services (e.g., stored forest carbon) from drone imagery that was reported by landowners. Replacing on-ground inspection with remote assessment via drones, however, opens up the possibility of untruthfully reported imagery. Given that the landowner is financially incentivized by PES to report higher forest ecosystem services value (e.g., higher forest cover or biodiversity), the possibility of false reporting is high, but not addressed by previous works. We hypothesize that the landowner could report untruthful imagery by altering the image 1) in location, 2) in time, or 3) with adversarial perturbation (e.g., via PGD(Madry_2018)), as displayed in Figure 2.
We propose TrueBranch, a metric learning-based algorithm that verifies the trustworthiness of reported drone imagery of forest conservation projects. Specifically, TrueBranch aims to verify the truthfulness of drone images via matching them with public satellite images. Matching is proposed to be done in a deeply learned feature space that, ideally, 1) allows for easy distinction of images with different value of forest ecosystem services, 2) is robust to adversarial perturbation of the drone image, and 3) generalizes the verification of drone images to other ecosystems (e.g., mangroves or peatlands). Figure 1 shows how TrueBranch is embedded in the scheme of automated MRV systems to achieve low-cost, accurate, and trustworthy MRV, which will promote international investments in forest conservation.
2 Attack vectors
Landowners are financially incentivized to report untruthful drone imagery that displays forest with higher ecosystem services value to receive higher PES. To reason about an algorithm that detects untruthful imagery, we classify common attack vectors with examples, as displayed inFigure 2:
Altering drone image location: The landowner has land with 50% forest cover and reports imagery from a neighbouring land with 80% forest cover to receive higher valuation.
Altering drone image time stamp: The landowner reports imagery (e.g., with altered time stamp metadata) from previous flights, before their land has been logged or cleared.
Altering drone image values
: The landowner tricks a neural network-based forest valuation algorithm into estimating higher ecosystem value by altering the image values with sophisticated attacks such as human-imperceptibleadversarial perturbation (e.g via PGD (Madry_2018)).
A combination of the above.
3.1 Challenges of detecting attack vectors with nominal metrics
We ran preliminary experiments to investigate if we can detect attack vectors (i.e., untruthfully reported imagery). Figure 5 shows all used publicly available drone (0.3m/px) and satellite (4m/px) imagery from OpenForest and PlanetLabs, respectively. Figure 3 suggests that nominal distance metrics (here, MSE in pixel or RESICS-45 feature space (i.e., the activation layer of a classification network, trained on satellite imagery (neumann2019indomain))) are not sufficient to reliably separate truthful from untruthful drone imagery by using the corresponding low-resolution satellite imagery.
3.2 Detecting attack vectors with a learned metric
Based on our preliminary results, we design TrueBranch to leverage the triplet loss function(Schroff_2015) from the field of metric learning. The loss function is designed to pull truthful images closer to the true satellite image and push untruthful images away from it. The result of training with triplet loss should be a deeply learned feature space (i.e., learned metric) in which truthful and untruthful imagery is separated by a large margin, as displayed in Figure 4. In comparison to the nominal metrics from section 3.1, the learned metric is also designed to be robust to adversarially perturbed images (sec. 2.3), as shown by mao2019metric
. As metric learning requires large amounts of imagery and high-resolution drone imagery is scarce, future works could augment drone imagery by satellite imagery, enhanced by super resolution algorithms(deudon2020highresnet). Note that the metric learning-approach enables TrueBranch to be extended from the verification of imagery of forest conservation projects to a multitude of other ecosystem conservation projects (e.g., mangroves, peatlands, wetlands, etc.) by adding the respective imagery.
Truthfully reported imagery is a vital requirement to issue Payments for Ecosystem Services (PES) based on trusted and automatic valuation of ecosystem services (e.g., stored forest carbon). In this paper, we list possible attack vectors and propose TrueBranch, a metric learning-based algorithm that uses public satellite data and metric learning to distinguish truthfully reported from untruthfully reported imagery.
The authors are thankful for the guidance and advise by the mentors (Prof. Dava Newman, Forrest Meyen, Adam de Sola Pool), support from the local community (Sandro Pimentel, La Niebla Forest), academic collaborators (Prof. Pedro Brancalion, Prof. Paulo Guilherme Molin), non-governmental institutions (WWF Peru and Brazil) and CONAF (Daniel Montaner, Cesar Mattar, Jose Antonio Prado). Part of this research has been developed as part of the OpenSurface platform and a real-world pilot in Chile, which was launched at the COP25 United Nation’s Climate Summit. OpenSurface is funded by IDBLab and EIT Climate-KIC.