DeepAI
Log In Sign Up

IGrow: A Smart Agriculture Solution to Autonomous Greenhouse Control

Agriculture is the foundation of human civilization. However, the rapid increase and aging of the global population pose challenges on this cornerstone by demanding more healthy and fresh food. Internet of Things (IoT) technology makes modern autonomous greenhouse a viable and reliable engine of food production. However, the educated and skilled labor capable of overseeing high-tech greenhouses is scarce. Artificial intelligence (AI) and cloud computing technologies are promising solutions for precision control and high-efficiency production in such controlled environments. In this paper, we propose a smart agriculture solution, namely iGrow: (1) we use IoT and cloud computing technologies to measure, collect, and manage growing data, to support iteration of our decision-making AI module, which consists of an incremental model and an optimization algorithm; (2) we propose a three-stage incremental model based on accumulating data, enabling growers/central computers to schedule control strategies conveniently and at low cost; (3) we propose a model-based iterative optimization algorithm, which can dynamically optimize the greenhouse control strategy in real-time production. In the simulated experiment, evaluation results show the accuracy of our incremental model is comparable to an advanced tomato simulator, while our optimization algorithms can beat the champion of the 2nd Autonomous Greenhouse Challenge. Compelling results from the A/B test in real greenhouses demonstrate that our solution significantly increases production (commercially sellable fruits) (+ 10.15 and net profit (+ 87.07 experts.

READ FULL TEXT VIEW PDF

page 1

page 2

page 3

page 4

01/26/2022

A Simulator-based Planning Framework for Optimizing Autonomous Greenhouse Control Strategy

The rapidly growing global population presents challenges and demands fo...
01/08/2019

Smart-Edge-CoCaCo: AI-Enabled Smart Edge with Joint Computation, Caching, and Communication in Heterogeneous IoT

The development of mobile communication technology, hardware, distribute...
06/18/2020

Pervasive Communications Technologies For Managing Pandemics

Pandemics always have had serious consequences unless they were effectiv...
07/04/2022

The Dichotomy of Cloud and IoT: Cloud-Assisted IoT From a Security Perspective

In recent years, the existence of a significant cross-impact between Clo...
08/02/2019

Distributed Deep Convolutional Neural Networks for the Internet-of-Things

Due to the high demand in computation and memory, deep learning solution...
03/23/2018

Welfare Without Taxation - Autonomous production revenues for Universal Basic Income

In the face of shifting means of production from manual human labor to l...
09/21/2022

Artificial Intelligence and Innovation to Reduce the Impact of Extreme Weather Events on Sustainable Production

Frequent occurrences of extreme weather events substantially impact the ...

1. Introduction

With global challenges in food, energy, and water, the digital transformation of the agricultural industry is extremely urgently in need. In order to meet these challenges, smart agriculture solutions, based on automation and intelligent decision-making, have attracted more and more attention(Brajović et al., 2015; Mekala and Viswanathan, 2017; Hemming et al., 2020).

Since governments have begun to launch development plans (van Rijswick, 2018), agricultural digitization has begun to take root globally. Modern high-tech greenhouses equipped with sensors and actuators can achieve a high level of autonomous control by integrating IoT and edge computing technologies. With an efficient control strategy, autonomous greenhouses ensure high crop production at a relatively low resource cost. However, it would require experienced and skilled labors to make optimal decisions based on the high volume of digital information, not to mention the challenges to scale (Brain, [n.d.]).

With the development of AI, new technologies are being applied in the agricultural field (Liakos et al., 2018). To the best of our knowledge, there is no mature AI-based solution for Autonomous Greenhouse Control (AGC). The planning algorithms based on forwarding search have achieved remarkable success in the AI field (Brajović et al., 2015). However, these planning algorithms all rely on the prior knowledge of environmental dynamics of the underlying system (Vemula et al., 2016), such as game rules or precise simulator dynamics. Therefore, with safety concern, they cannot be directly applied to real-world applications, such as robotics, industrial control, or controlled-environment agriculture. Moreover, sample efficiency (high cost, long cycle, sparse reward) has always been one of the main challenges of building data-driven solutions in agriculture. A simulation model is able to generate virtual trajectories and enables more efficient evaluation and iteration of planting strategies. More importantly, it provides a testbed for optimization of AI algorithms, which has the potential to take over the decision-making process of AGC, securing reliable outcomes while saving considerable labor.

In the past few decades, dynamic climate models (Tantau, 1985; Van Straten et al., 2010) and crop models (Gary et al., 1998; Marcelis et al., 2008)

of greenhouse simulation have been developed. However, the existing models mainly rely on the domain knowledge in physics and biology specific to greenhouse setups, which is difficult to transfer to other scenarios. Nowadays, neural networks (NN) and deep learning have replaced expert systems in many fields 

(Chellapilla et al., 1999; Imrak, 2008) by virtue of their generalization and expressive power. However, the application of AI algorithms on greenhouse simulation and control remains underexplored.

The quantity and quality of data are key to the success of data-driven AI approaches in addressing the AGC problem. IoT technologies enable measurement of multimodal data (Mekala and Viswanathan, 2017), which can be efficiently stored and processed via the cloud platform (Keerthi and Kodandaramaiah, 2015). Thus, the combination of IoT and the cloud provides an ideal solution for agricultural data collection and management. However, limited by the sensor capacity, only partial observations of the greenhouse states can be obtained. Any AI models trained based on such collected data inevitably lead to the simulation-to-reality gap (Jakobi et al., 1995). By incorporating an incremental scheme in our framework, the hope is that by continuously accumulating sensor data, the proposed AI algorithms can keep improving and eventually become deployable in the large-scale production environment.

In this paper, we model the AGC problem and propose a smart agriculture solution, namely iGrow, which is based on AI, IoT, and cloud-native technologies, as shown in Figure 1. By autonomous greenhouse control, iGrow has the potential of saving considerable labor costs, thus significantly releasing the labor shortage problem. We test our framework in both simulated and real greenhouses scenarios. The experimental results demonstrate the technical and economic value of our intelligent agricultural decision-making AI module for the optimization of the AGC problem.

Figure 1. An overview of iGrow. Specifically, iGrow optimizes a strategy under an NN-based simulator, remotely deploys the strategy to greenhouse, collects data via sensors, and then utilizes these data to update the current simulator incrementally. The overall procedure is in an iterative manner.

The main contributions of our work are summarized as follows:

  • To our best knowledge, for the first time, the optimization problem of autonomous greenhouse control (AGC) is formulated as a Markov decision process (MDP) problem, which is solved by our proposed bi-level optimization algorithm.

  • We are the first to propose an NN-based three-stage simulation model to approximate the transition dynamics of the AGC problem, which is used as a testbed for the optimization of control strategies. With a real agricultural dataset, we validate that the accuracy of the NN-based simulator is comparable to that of the state-of-the-art simulator based on expert knowledge and rules.

  • We propose a model-based iterative heuristic control algorithm, with provably super-human performance in both simulation and reality.

  • We test our solution in both simulated and real autonomous greenhouses. Particularly, during the A/B test in real greenhouses, the experimental results demonstrate the performance of iGrow solution statistically significantly () exceeds that of planting experts with an average improvement of 10.15% in production and 87.07% in net profit.

2. Related Work

Digital agriculture: IoT with cloud computing technology has been used to provide smart agricultural solutions (Mekala and Viswanathan, 2017). Some applications (Liqiang et al., 2011; Zhu et al., 2011) use wireless sensor networks for monitoring crop fields. Thus, managers can monitor different environmental parameters efficiently (Danita et al., 2018) via devices such as light, temperature, relative humidity, and soil moisture sensors. These sensors are used to measure and collect information, which will further be logged and stored online using IoT and cloud computing (Brajović et al., 2015). Automated irrigation systems (Parameswaran and Sivaprasath, 2016) have been developed and used for the remote control to minimize human involvement. However, to our best knowledge, an end-to-end solution, such as closed-loop remote control and decision-making system of autonomous greenhouses, has not yet been fully developed and validated.

Greenhouse simulation: Simulation of a modern greenhouse can be divided into two parts: modeling of the underlying (1) physical dynamics (climate model) and (2) biological process (crop model). There is a set of research focusing on the design of mechanistic greenhouse climate models to simulate physical dynamics (such as temperature, CO concentration, humidity, etc.) in greenhouses (Dincer and Cengel, 2001; Wallach et al., 2018; Piñón et al., 2005; Van Beveren et al., 2015). Integrating the first and second laws of thermodynamics into the model helps improve simulation accuracy (Dincer and Cengel, 2001)

. The nonlinear least square is the most used method for parameter estimation 

(Wallach et al., 2018)

. The dynamic greenhouse climate model has been used to combine model predictive control (MPC) and evolutionary algorithm to solve a greenhouse open-loop optimization control problem 

(Piñón et al., 2005; Van Beveren et al., 2015). Crop modeling is the other essential part of the greenhouse simulation, which can be used to predict yield, growth, and nutrient relations of the greenhouse-grown crops (Marcelis et al., 2008). Researchers establish crop models by analyzing crop growth mechanisms (Kuijpers et al., 2019; Bertin and Heuvelink, 1993; Marcelis et al., 2008; De Visser et al., 2014); for example, exploring the relationship between leaf photosynthesis efficiency and dry matter (Bertin and Heuvelink, 1993). Combining the crop model with the dynamic greenhouse climate model enables growers to explore and optimize greenhouse planting strategies (De Visser et al., 2014). As far as we know, AI approaches for modeling greenhouse planting processes remain underexplored.

AI for agriculture

: AI-powered agriculture has been studied extensively since the turn of the millennium. Researchers have tried various machine learning methods in agriculture, for which a review can be found in 

(Liakos et al., 2018)

. Recently, deep learning techniques and reinforcement learning (RL) techniques have also been applied in agriculture. For deep learning in agricultural applications,  

(Kamilaris and Prenafeta-Boldú, 2018) provides an excellent survey to start with, and applications with deep learning include crop classification (Kussul et al., 2017), crop disease detection (Ferentinos, 2018), and remote crop count (Rahnemoonfar and Sheppard, 2017). When it comes to RL and agriculture, researchers focus on operational management (i.e., learning a good decision-making sequence to maximize an objective). For example,  (Garcia, 1999) studies wheat crop management with RL techniques. RL has also been applied to resource consumption management. For instance,  (Bergez et al., 2001; Bhattacharya et al., 2003) used RL to optimize maize irrigation and water system management in agriculture. However, few studies focus on optimizing automated greenhouse planting strategies via simulator-based AI algorithms (An et al., 2021).

3. Problem Statement

In this work, we formulate the greenhouse control problem as a stochastic MDP optimization problem. In the following section, we give the necessary background of MDP and formally introduce the problem of AGC under weather uncertainty.

3.1. MDP introduction

An MDP can be denoted by a tuple , where and represent the state and action spaces, respectively. The dynamics or transition distribution are denoted as , the initial state is assumed to follow the initial state distribution , gives the reward function of executing action to transition from state to , and is the discount factor. A policy on maps a state

to a probability distribution over

that generates trajectories as , where , , , and is the terminate time step.

MDP optimization problem. Find a policy that maximizes the cumulative expected return given :

(1)

3.2. Formulation of the AGC optimization problem

An autonomous greenhouse planting process can be denoted by a tuple , where denotes greenhouse climate, is the crop growth state, represents the production, specifies the control setpoints, and represents the outside weather.

The objective of AGC is to find a policy that both improves crop yields and reduces expenses, such as resource consumption and labor costs, and this can be viewed as a specific instance of the MDP optimization problem. There are many factors involved in the AGC optimization problem, but we can only obtain partial observations from the greenhouse due to the limitation of the sensors. As a result, we focus on several observable factors that have a significant impact on crop cultivation, and omit other variables. Within this context, we are able to specify the state space, the action space, the reward function, and the transition function of the AGC MDP as follows.

  • State space: In the AGC MDP, each state consists of a four tuple , and details of each element are shown in Table 1. It is noteworthy that each variable of the state is a number with different units, which encode the different information determining the growth status of the crop.

  • Action space: Only four essential control variables are involved in the AGC MDP, including temperature, CO concentration, lighting, and irrigation. We present basic statistics in Table 2. Notice that those action variables could affect the state variables, except for the outside weather , which is beyond control.

  • Reward function: Since AGC aims to weigh crop yields against total expenses, we denote the cumulative return by the crop gains minus the cost of control policy consisting of resource consumption, labor costs and etc. By setting , the reward function can be converted into the formulation: .

  • Transition function: The transition function in AGC is assumed to be unknown, so we seek for a model of the dynamics as the approximation of instead.

Category Name Description Unit
Outside
weather
()
Iglob outside solar radiation
Tout outside temperature
RHout outside humidity %
COout outside CO concentration
Windsp outside wind speed
Tsky virtual sky temperature
Indoor
climate
()
AirT indoor temperature
AirRH indoor humidity %
AirCO indoor CO concentration
PAR photosynthetically active radiation
Crop
state
()
LAI leaf area index -
PlantLoad number of growing fruits
NetGrowth photosynthesis net growth
Production () FW fruit fresh weight
Table 1. Basic state variables are defined in our model.
Action (setpoint) Units
indoor temperature
indoor CO concentration
illumination on/off -
irrigation on/off -
Table 2. Basic actions are defined in our model.

3.3. Bi-level optimization

Let be the parameterization of the predictive model , and parameterize the control strategy by parameters . In AGC, should be learned before the optimization of strategy in consideration of the sample inefficiency of crop planting. Within this context, the AGC optimization can be formulated as:

(2)

where is the training loss on a given dataset . The above problem formulation is actually consistent with bi-level optimization (Wen and Hsu, 1991) in a broader scope since both and need to be optimized in order to achieve better strategies. Basically, and are treated respectively as upper-level and lower-level variables that are optimized in an interleaving way. In particular, the lower-level optimization represents the simulator optimization with continuous data collection, while the upper-level optimization corresponding to the strategy iteration.

4. Methodology

As shown in Figure 1, we lay out our framework to the AGC optimization problem. The proposed solution consists of several essential components and each of them plays a different role in bi-level optimization. To be specific, the decision-making module firstly leverages AI algorithms to optimize a strategy under a given simulator, which actually is a predictive model learned from greenhouse planting data in our context. Then, the strategy is deployed to the greenhouse, supported by IoT and cloud technologies, and the corresponding planting records will be collected and stored in the database. We recall that the continuous collection of new planting data will undoubtedly contribute to the modification of the current simulator, which is beneficial to the further iterative optimization of strategy.

4.1. Incremental model

To solve an MDP optimization problem, the state transition function is required. However, the greenhouse planting process is stochastic and dynamic, so it is reasonable to assume the to be unknown in the AGC optimization problem. To address this issue, existing works simulate the process based on empirical formulas including physics and biology domain knowledge (Van Straten et al., 2010; Marcelis et al., 2009). Compared to traditional artificial models, the data-driven methods can model the planting dynamic with less prior knowledge, as well as perform with high accuracy and generalization (Imrak, 2008). Thus, we build an NN-based three-stage simulation model to approximate . The detailed modeling process is described below.

Greenhouse climate module: Crop growth is mainly determined by the indoor climate. Therefore, we need to build a model to simulate greenhouse climate change. At time , indoor climate is influenced by previous outside weather , control setpoints , and indoor climate . Thereby, the indoor climate change can be formalized as , where represents the transition function of indoor climate change. Generalized, the dynamic greenhouse climate change problem can be defined by a tuple , and the climate change is mapped by , where , , , (see Table 1 and Table 2 for details). We propose an NN-based structure to represent model , where represents network parameters. The model structure is shown in Figure 2

. We adopt the mean-square error (MSE) as the loss function:

where represents the average loss of samples and and represent real and prediction value of the -th greenhouse climate variable of the -th sample, respectively. Additionally, we assume greenhouse climate state transit once per hour, which is considered to be accurate enough to approximate reality.

Growth state module: According to the greenhouse climate model, we can obtain the indoor climate at time , which affects current crop growth state . Their relationship can be expressed as , where represents the transition function of crop growth state. Furthermore, the problem of crop growth can be defined by , and the growth process is given by , where and . Similarly, as shown in Figure 2, we build the state transition model using NNs, where represents corresponding network parameters. Besides, loss function and the frequency of state transition are similar to greenhouse climate models.

Production module: Production increase is infinitesimal within one hour. Thus, we focus on daily production, which is more reasonable. Production is determined by crop growth state. Therefore, the process of production accumulation process can be generalized as , where and . However, the state transition frequency of the growth state module is hourly. We extend

to a vector

, where represents the growth state of the -th hour of the day . Because crop growth state is accumulated over time in our model, we only need to consider in the production module. Thus, production of day can be denoted by , where represents the transition function of production. Similarly, we use as NNs parameters and the MSE as loss function to simulate production accumulation process. The structure is shown in Figure 2 as well.

Figure 2. Incremental Model Network Structure. (a) and (b) represent the greenhouse climate module and the crop growth module, respectively, which simulate the greenhouse planting process at the hour-level. (c) represents the production module, determined by crop growth state, and updates once a day.

Compared with one-module modeling, our model imposes constraints on the network structure and provides implicit regularization, which reduces the demand for data and prevents overfitting.

Limited by sensor technology, we cannot obtain a full observation of the greenhouse planting process. Thus, the bias between and exists objectively. Theorem 1 (Janner et al., 2019) indicates that as long as we improve the returns under the model by more than , we can guarantee improvement in .

According to the central limit theorem 

(Rosenblatt, 1956), we can easily deduce that and , where represents the scale of the dataset. This inference manifests that model is able to be accurate enough as long as enough real data. Therefore, we introduce the concept of incremental model on the basis of the three-stage model, that is, streaming update the model with the newly collected data.

Theorem 1 ().

Let the expected Kullback-Leibler (KL) divergence between the two transition distributions be bound at each time step by and the policy divergence be bound by . Then, the real returns and model returns of the policy are bound as:

(3)

4.2. Optimization algorithms

AI algorithms are known to be an efficient way to solve the MDP problem (Kolobov, 2012), and we consider two classical AI algorithms, including a heuristic algorithm and a reinforcement learning algorithm, to solve the AGC optimization problem.

Soft Actor-Critic: In this work, we adopt Soft Actor-Critic (SAC) (Haarnoja et al., 2018), a state-of-the-art off-policy reinforcement-learning algorithm, to optimize the AGC strategy. SAC interacts with the simulator constantly to collect transition tuples and store them. SAC alternates between a soft policy evaluation and a soft policy improvement. During soft policy evaluation, an NN-based soft Q-function is updated by minimizing a soft Bellman residual. During soft policy improvement, the policy is optimized by maximizing an entropy objective. The optimization process is given in Algorithm 1.

Input: policy , Q-function and replay buffer
Output: Optimal after epochs
1 for  iterations do
2       for each step  do
3            
4            
5            
6            
7      for each gradient step do
8             Sample random minibatch
9             BellmanResidual()
10             Minimize()
11             KL-divergence()
12             Minimize()
13            
14      
Algorithm 1 Soft Actor-Critic
Input: Initial population size , iteration limit , and the composition of polysomy
Output: Optimal after iterations
1
2
3
4
5
6 for  iterations do
7      
8       for each  do
9            
10            
11            
12      
13      
14      
15      
16      
17      
18
19
Algorithm 2

Elitist Genetic Algorithm

Elitist Genetic Algorithm: Classical heuristic algorithms, such as the genetic algorithm, simulate biological evolution to solve deterministic optimization problems (Ponsich et al., 2008). In this paper, we apply an elitist genetic algorithm (EGA) (Rani et al., 2019), which can accelerate the convergence of the AGC strategy. Because the state space of each variable is different, we adopt multiple chromosomes to encode these variables. That is, in one cycle, the same variables share the same chromosomes by concatenation. The optimization procedure is summarized in Algorithm 2.

In the actual planting process, we take the final accumulated net profit as the objective. Therefore, we have to optimize the AGC strategy throughout the whole planting period. The strategy requires the weather forecast model , which makes predictions for future weather to support optimization.

4.3. Closed-loop iterative control of iGrow

The inherent model bias in the simulation can be exploited by the optimization algorithms, leading to large reality gap between the actual and simulated policy performance (Janner et al., 2019). To alleviate this problem, we propose an iterative scheme to enable continual but potentially asynchronous evolution of both the model and the strategy. As shown in Algorithm 3, for every time steps, the control strategy is updated on the current simulation model. The setpoints produced by the latest strategy are fed to the real greenhouses and new planting data are collected to expand the training dataset. And for every time steps, the simulation model will in turn be updated by fine-tuning on the latest data buffer. The proposed procedure resembles lifelong learning (Field, 2000), with the flexibility of updating the model and strategy at different paces to ensure stable and optimal progression.

5. Simulation Experiment

In this section, we train the incremental model via real planting data. Based on the model, we compare our AI algorithms with the logged strategies of participants in an Autonomous Greenhouse Challenge111https://www.wur.nl/en/project/autonomous-greenhouses-2nd-edition.htm organized by Wageningen University & Research (WUR).

5.1. Dataset

We use the dataset collected from the competition above. Five teams used AI algorithms to control individual greenhouses to grow tomatoes for 166 days remotely. A team of experienced growers also participated in the competition as reference. This dataset contains a comprehensive record of the sensing and control data used during the competition, such as greenhouse temperature and CO levels.

However, only six trajectories are far from satisfactory for training a robust model. WUR developed a greenhouse tomato simulator based on empirical formula to approximate the real greenhouse planting process (Hemming et al., 2020). We randomly simulate thousands of control strategies to record their rollouts, and collect these trajectories as the training set of our model.

In the simulated experiment, the calculation for net profit is the same as that of the Autonomous Greenhouse Challenge222https://www.kaggle.com/piantic/autonomous-greenhouse-challengeagc-2nd-2019.

Input: Dataset , model , the update period of strategy, and the update period of model
Output: Updated model and updated dataset
1 Initialize
// A complete greenhouse planting period
2 for  do
3       if  then
4             update by simulating from current on
5            
6      if  then
7             update by
8            
9      
       // Perform action, then calculate reward and collect observations in real greenhouse
10      
       // Log planting process
11      
12      
Algorithm 3 Closed-loop iterative control of iGrow

5.2. Hardware, software and parameters

The hardware configuration is: Intel (R) Core (TM) i7-9700K, one GeForce RTX 2080 Ti GPU, and the computing memory is 12 GB. We implement the model in a Python 3.6.13, PyTorch 1.7.1 environment. In the training phase of neural networks, the batch size is 256, the learning rate is set to be 1e-3, and the number of epoch is 150.

5.3. Simulation models comparison

In our previous work (An et al., 2021), we have experimentally demonstrated that the simulation accuracy of a data-driven NN-based model can gradually improve with the amount of training data. Therefore, we randomly choose 1000 virtual planting trajectories generated by the WUR simulator as the training set . Similarly, we construct the testing set with 50 virtual planting trajectories. First, based on the new dataset, we use the method in (An et al., 2021) to train a baseline model. Next, to study the role of real data, we added real planting data of 5 participants (except the champion team: Automatoes) to . To ensure the fairness of the comparison, we randomly remove five trajectories from the training set, that is . Accordingly, the training set of the incremental model is

. We use supervised learning to train the incremental model. Specifically, we adopt the oversampling technique 

(Barua et al., 2012) to address the dilemma of the imbalanced dataset. According to Table 3, the fitting accuracy of the baseline and incremental models both are satisfactory. The overall goodness-of-fit () of the baseline and incremental models are 0.9110.009 and 0.9060.046, respectively. From this point of view, the baseline is slightly better than the incremental model.

However, the testing set, consisting of the virtual planting trajectories generated by the WUR simulator, is naturally limited by the inherent gap of simulation to reality. Therefore, the accuracy of the baseline model maybe inferior to the incremental model in the real scenario. To verify this conjecture, we input the planting strategies of the champion team into different simulators to get different virtual planting trajectories, and then compare them with the real planting trajectory. Figure 3 indicates that the two models we adopted both achieve comparable simulation accuracy to the WUR simulator. Moreover, we observe that the incremental model is more accurate than the baseline model, which demonstrates that the real data is crucial in model calibration. This also shows that lower-level optimization makes sense.

Figure 3. Comparing the accuracy of different models on Automatoes. (a) represents net profit simulated by the same strategy on different models. (b) represents the accumulated absolute error of different models compared to the ground-truth net profit.
Figure 4. Comparison of the three methods. (a) represents the performance of different methods on our incremental model. (b) and (c) show the 120-hour control strategies of temperature and illumination provided by EGA and Automatoes.

5.4. Comparison methods

  • Automatoes strategy: Automatoes is the champion of the Autonomous Greenhouse Challenge. The organizer briefly introduces their algorithm in (Hemming et al., 2020). They combine a predictive control algorithm with expert knowledge. Since the algorithm details are unknown, we directly conduct their public setpoints on our incremental model.

  • EGA: EGA is an improved genetic algorithm that improves convergence by adopting an elite retention strategy.

  • SAC: SAC is one of the current state-of-the-art off-policy actor-critic reinforcement-learning algorithms for continuous control problems.

According to Figure 4(a), we observe that: (1) The performance of SAC is inferior to that of EGA. It is mainly because the delayed reward is especially problematic in AGC, resulting in a severe challenge to RL algorithms. For example, current control setpoints may not be reflected until a month later; (2) The Automatoes strategy performs the worst simulation result, since the Automatoes integrates prior knowledge for their decision-making, rather than using a pure AI method. In this experiment, the access to our simulation model is unlimited, enabling two AI algorithms to fully exploit their optimization capabilities to obtain more fine-grained control policies, for instance, to achieve hourly regulation as shown in Figure 4(b).

Method AirT AirRH AirCO PAR
Baseline 0.915 0.707 0.930 1.000
Incremental 0.986 0.941 0.988 1.000
Method LAI PlantLoad NetGrowth FW
Baseline 0.927 0.935 0.959 0.917
Incremental 0.669 0.849 0.939 0.880
Table 3. of different variables in the incremental model, where, , represents the goodness of fit of the simulation results of real observations, and closer to 1 means better performance (McDonald, 1989).

6. Pilot Experiment

In the pilot phase, we use the model-based iterative EGA to optimize an AGC strategy and combine IoT with cloud-native technologies to deploy the smart agriculture solution, i.e., iGrow.

6.1. Deployment overview

We chose tomatoes as the experimental crop in the pilot project since they are one of the main greenhouse crops worldwide. Our pilot program started in October 2019 and is still ongoing. Both the control and experimental groups are required to manage the tomato crops remotely in autonomous greenhouse compartments to maximize net profit. As mentioned in Section  3.2, net profit is a balance between gains and expenses. Gains are calculated according to the production and the selling price, while total expenses include resource consumption (electricity, heat, CO, and water), crop maintenance costs, and equipment amortization (by apportioning the purchase price of equipment to six years). Additionally, daily crop maintenance (crop management, fertilization and etc.) of all compartments are equally treated by labors.

The control group’s compartments are managed by planting experts with growing experience of more than 20 years, while the experimental group is controlled by iGrow, whose incremental simulation model is not static but is updated monthly using the recently collected planting data. Note that all compartments are equipped with the same materials for a fair comparison. In addition, the control and the experimental groups need to control the crop growth remotely according to the measured greenhouse information monitored by the sensors.

6.2. Pilot project setup

The growers have to determine setpoints in advance, which will be sent to the cloud platform through an application programming interface (API) (the details of the controlled variables are shown in Table 1). The cloud is responsible for generating instructions for the actuators according to the currently measured data of the greenhouse by sending setpoints every 10 minutes.

The deployed sensors are shown in Supplementary Figure S2. These sensors continuously measure the data with frequency between every 1-3 seconds. Once the measured data is updated, it will be transmitted to the cloud platform via an embedded sensor system. Cloud-computing technology provides data cleaning, visualization, and weather forecasts. Growers can monitor the entire process in real-time through the graphical interface as shown in Supplementary Figure S3. Also, the measured data can be downloaded from cloud storage.

As described in Section 4.3, we use IoT with cloud-native technology to accumulate planting data. Furthermore, the decision-making module of iGrow will optimize and update control strategies based upon the incremental model on a daily basis. During the optimization process, the previously predicted states of the simulation model are continuously replaced by the real states recorded by the sensors to calibrate the model errors and re-optimize the control strategy.

6.3. Case study

We have been conducting 4 pilot experiments through some real autonomous greenhouses in Liaoyang City, Liaoning Province, China (See Supplementary Figure S1). iGrow solution has been deployed in over 20 compartments, each of which covers an area of 1 Mu (1/15 hectare). In each pilot experiment, we all take A/B tests where the control groups hire expert growers for greenhouse management, whereas the experimental groups rely on the iGrow platform for automated greenhouse control and management. The first phase of the pilot project333https://www.tencent.com/en-us/articles/2201057.html ran from October 2019 to March 2020, and the experimental group and the control group controlled 2 compartments and 1 compartment, respectively. The experimental group outperformed the control group by 4,000 RMB (500 euro) in net profit on average per season. In the second phase, from March 2020 to July 2020, we scaled up the size to 2 and 5 compartments for the control and experimental group, respectively. In this paper, we utilize the results of the second phase for analysis and seek some insights.

Note that we assume the results of the control group represent the average level of growing experts, and we adopt an independent sample T-test for the experimental group to verify the superiority of iGrow. We recall that when the result of the T-test satisfies the condition

, this indicates that the result is statistically significant. Some key statistics are given as follows:

  • Gains: The experimental group average increases by 26.30% compared to the control group. It is because production and price, which depends on the weight and the sweetness of the fruits, improve statistically significantly.

  • Costs: The average total costs of the control and experimental groups are 3,270.04 euro/Mu and 3,223.29 euro/Mu, respectively. Although the experimental group consumes more energy, the harvests from its compartments all finished one week earlier than the control group (see Figure 5(a)), saving daily crop maintenance costs.

  • Net profit: Compared to the control group, the experimental group shows increased gains and reduced costs, resulting in an 87.07% improvement in net profit. See Figure 5 and Table 4 for more details.

Figure 5. Comparison of the harvests between the control (controlled by growing experts) and the experimental (controlled by iGrow) group during the second phase of the pilot project.
Economic
Control
Group
Experimental
Group
T-test
Energy Cost 13.90 0.23 50.11 20.48 -260.57% 2e-2
Crop Maintenance
Cost
1543.96 0.00 1457.19 5.1 5.62% 5e-6
Equipment
Emortization
1171.88 0.00 1711.88 0.00 0.00% -
Total Cost 3269.74 0.23 3219.18 18.14 1.55% 5e-3
Price 0.45 0.00 0.49 0.01 10.63% 5e-4
Production 16.64 0.80 18.33 0.68 10.15% 8e-3
Gains 4768.30 9.79 6106.88 189.09 28.07% 1e-4
Net Profit 1498.56 10.03 2887.7 183.45 92.70% 1e-4
  • Relative Improvement: the percentage improvement of the experimental group
    relative to the control group on the specified index.

Table 4. Overall economic comparison during the second phase of the pilot project.

We plot the pair relationship among four states (See complete results from Supplementary Figure S4 to S13): AirT, AirCO, AirPAR, and AirRH, which are the controlled variables under planting experts and the iGrow platform. According to Figure 6, we observe some useful insights: (1) The greenhouse controlled by the control group exceeds 30 C more frequently, which is not suitable for crop growth, sometimes even harmful to crop health (Siscaro et al., 2019); (2) Between 17-27 C, photosynthetic enzymes are more active (Shamshiri et al., 2018). In the meantime, the greenhouse controlled by the experimental group shows higher CO concentration and PAR, contributing to enhancing photosynthetic intensity; (3) The greenhouse controlled by the control group has more humidity above 70%, which is harmful to crop health (Shamshiri et al., 2018).

Figure 6. The relationship among AirT, AirCO, AirPAR, and AirRH under the control strategies.

7. Conclusion

In this paper, we formulate autonomous greenhouse control as a stochastic MDP optimization problem and propose a smart agriculture solution, namely iGrow. We build cloud IoT infrastructure to collect and manage planting data. These data are used to update our AI decision-making module consisting of a simulation model and a control strategy. Both simulated and pilot results demonstrate the superiority of our solution.

Our solution has potential to improve the automation level of greenhouse climate control and boost growing efficiency in real production environments. Moreover, our modeling methodology provides a paradigm to build an accurate digital twin of the corresponding real-world applications so that AI algorithms can be developed and tested without a real environment.

References

  • (1)
  • An et al. (2021) Zhicheng An, Xiaoyan Cao, Yao Yao, Wanpeng Zhang, Lanqing Li, Yue Wang, Shihui Guo, and Dijun Luo. 2021. A Simulator-based Planning Framework for Optimizing Autonomous Greenhouse Control Strategy. In Proceedings of the International Conference on Automated Planning and Scheduling, Vol. 31. 436–444.
  • Barua et al. (2012) Sukarna Barua, Md Monirul Islam, Xin Yao, and Kazuyuki Murase. 2012. MWMOTE–majority weighted minority oversampling technique for imbalanced data set learning. IEEE Transactions on knowledge and data engineering 26, 2 (2012), 405–425.
  • Bergez et al. (2001) Jacques-Eric Bergez, M Eigenraam, and F Garcia. 2001. Comparison between dynamic programming and reinforcement learning: a case study on maize irrigation management. In Proceedings of the 3rd European Conference on Information Technology in Agriculture (EFITA01), Montpellier (FR) pp

    . Citeseer, 343–348.

  • Bertin and Heuvelink (1993) Nadia Bertin and Egbert Heuvelink. 1993. Dry-matter production in a tomato crop: comparison of two simulation models. Journal of Horticultural Science 68, 6 (1993), 995–1011.
  • Bhattacharya et al. (2003) B Bhattacharya, AH Lobbrecht, and DP Solomatine. 2003. Neural networks and reinforcement learning in control of water systems. Journal of Water Resources Planning and Management 129, 6 (2003), 458–465.
  • Brain ([n.d.]) D Brain. [n.d.]. What Is the Current State of Labor in the Greenhouse Industry. Greenhouse Grower. Available online: https://www. greenhousegrower. com/management/what-is-the-current-state-of-laborin-the-greenhouse-industry/(accessed on 15 November 2018) ([n. d.]).
  • Brajović et al. (2015) Miloš Brajović, Stefan Vujović, and Slobodan Đukanović. 2015. An overview of smart irrigation software. In 2015 4th Mediterranean Conference on Embedded Computing (MECO). IEEE, 353–356.
  • Chellapilla et al. (1999) Chellapilla, Kumar, Fogel, and B. David. 1999. Evolving Neural Networks to Play Checkers Without Relying on Expert Knowledge. IEEE Transactions on Neural Networks (1999).
  • Danita et al. (2018) M Danita, Blessy Mathew, Nithila Shereen, Namrata Sharon, and J John Paul. 2018. IoT based automated greenhouse monitoring system. In 2018 Second International Conference on Intelligent Computing and Control Systems (ICICCS). IEEE, 1933–1937.
  • De Visser et al. (2014) Pieter Huibert Bram De Visser, Gerie van der Heijden, and Gerhard Buck-Sorlin. 2014. Optimizing illumination in the greenhouse using a 3D model of tomato and a ray tracer. Frontiers in Plant Science 5 (2014), 48.
  • Dincer and Cengel (2001) Ibrahim Dincer and Yunus A Cengel. 2001. Energy, entropy and exergy concepts and their roles in thermal engineering. Entropy 3, 3 (2001), 116–149.
  • Ferentinos (2018) Konstantinos P Ferentinos. 2018. Deep learning models for plant disease detection and diagnosis. Computers and Electronics in Agriculture 145 (2018), 311–318.
  • Field (2000) John Field. 2000. Lifelong learning and the new educational order. ERIC.
  • Garcia (1999) Frédérick Garcia. 1999. Use of reinforcement learning and simulation to optimize wheat crop technical management. In Proceedings of the International Congress on Modelling and Simulation (MODSIM’99) Hamilton, New-Zealand. 801–806.
  • Gary et al. (1998) Christian Gary, JW Jones, and Marc Tchamitchian. 1998. Crop modelling in horticulture: state of the art. Scientia Horticulturae 74, 1-2 (1998), 3–20.
  • Haarnoja et al. (2018) Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel, et al. 2018. Soft actor-critic algorithms and applications. arXiv preprint arXiv:1812.05905 (2018).
  • Hemming et al. (2020) Silke Hemming, Feije de Zwart, Anne Elings, Anna Petropoulou, and Isabella Righini. 2020. Cherry Tomato Production in Intelligent Greenhouses—Sensors and AI for Control of Climate, Irrigation, Crop Yield, and Quality. Sensors 20, 22 (2020), 6430.
  • Imrak (2008) C. Erdem Imrak. 2008. Artificial neural networks application in duplex/triplex elevator group control system. Strojniski Vestnik 54, 2 (2008), 103–114.
  • Jakobi et al. (1995) Nick Jakobi, Phil Husbands, and Inman Harvey. 1995. Noise and the reality gap: The use of simulation in evolutionary robotics. In European Conference on Artificial Life. Springer, 704–720.
  • Janner et al. (2019) Michael Janner, Justin Fu, Marvin Zhang, and Sergey Levine. 2019. When to trust your model: Model-based policy optimization. arXiv preprint arXiv:1906.08253 (2019).
  • Kamilaris and Prenafeta-Boldú (2018) Andreas Kamilaris and Francesc X Prenafeta-Boldú. 2018. Deep learning in agriculture: A survey. Computers and electronics in agriculture 147 (2018), 70–90.
  • Keerthi and Kodandaramaiah (2015) V Keerthi and GN Kodandaramaiah. 2015. Cloud IoT based greenhouse monitoring system. International Journal of Engineering Research and Applications 5, 10 (2015), 35–41.
  • Kolobov (2012) Andrey Kolobov. 2012. Planning with Markov decision processes: An AI perspective. Synthesis Lectures on Artificial Intelligence and Machine Learning 6, 1 (2012), 1–210.
  • Kuijpers et al. (2019) Wouter JP Kuijpers, Marinus JG van de Molengraft, Simon van Mourik, Albertus van’t Ooster, Silke Hemming, and Eldert J van Henten. 2019. Model selection with a common structure: Tomato crop growth models. biosystems engineering 187 (2019), 247–257.
  • Kussul et al. (2017) Nataliia Kussul, Mykola Lavreniuk, Sergii Skakun, and Andrii Shelestov. 2017. Deep learning classification of land cover and crop types using remote sensing data. IEEE Geoscience and Remote Sensing Letters 14, 5 (2017), 778–782.
  • Liakos et al. (2018) Konstantinos G Liakos, Patrizia Busato, Dimitrios Moshou, Simon Pearson, and Dionysis Bochtis. 2018. Machine learning in agriculture: A review. Sensors 18, 8 (2018), 2674.
  • Liqiang et al. (2011) Zhao Liqiang, Yin Shouyi, Liu Leibo, Zhang Zhen, and Wei Shaojun. 2011. A crop monitoring system based on wireless sensor network. Procedia Environmental Sciences 11 (2011), 558–565.
  • Marcelis et al. (2008) LFM Marcelis, A Elings, PHB De Visser, and E Heuvelink. 2008. Simulating growth and development of tomato crop. In International Symposium on Tomato in the Tropics 821. 101–110.
  • Marcelis et al. (2009) L. F. M Marcelis, A Elings, P. H. B. De Visser, E Heuvelink, G Fischer, S Magnitskiy, and S Nicola. 2009. Simulating growth and development of tomato crop.. In International Symposium on Tomato in the Tropics.
  • McDonald (1989) Roderick P McDonald. 1989. An index of goodness-of-fit based on noncentrality. Journal of classification 6, 1 (1989), 97–103.
  • Mekala and Viswanathan (2017) Mahammad Shareef Mekala and P Viswanathan. 2017. A Survey: Smart agriculture IoT with cloud computing. In 2017 international conference on microelectronic devices, circuits and systems (ICMDCS). IEEE, 1–7.
  • Parameswaran and Sivaprasath (2016) G Parameswaran and K Sivaprasath. 2016. Arduino based smart drip irrigation system using Internet of Things. International Journal of Engineering Science and Computing 6, 5 (2016), 5518–5521.
  • Piñón et al. (2005) Sandra Piñón, EF Camacho, B Kuchen, and M Peña. 2005. Constrained predictive control of a greenhouse. Computers and Electronics in Agriculture 49, 3 (2005), 317–329.
  • Ponsich et al. (2008) A. Ponsich, C. Azzaro-Pantel, S. Domenech, and L. Pibouleau. 2008. Constraint handling strategies in Genetic Algorithms application to optimal batch plant design. Chemical Engineering & Processing Process Intensification 47, 3 (2008), 420–434.
  • Rahnemoonfar and Sheppard (2017) Maryam Rahnemoonfar and Clay Sheppard. 2017. Deep count: fruit counting based on deep simulated learning. Sensors 17, 4 (2017), 905.
  • Rani et al. (2019) Shweta Rani, Bharti Suri, and Rinkaj Goyal. 2019. On the effectiveness of using elitist genetic algorithm in mutation testing. Symmetry 11, 9 (2019), 1145.
  • Rosenblatt (1956) Murray Rosenblatt. 1956. A central limit theorem and a strong mixing condition. Proceedings of the National Academy of Sciences of the United States of America 42, 1 (1956), 43.
  • Shamshiri et al. (2018) Redmond Ramin Shamshiri, James W Jones, Kelly R Thorp, Desa Ahmad, H Che Man, and Sima Taheri. 2018. Review of optimum temperature, humidity, and vapour pressure deficit for microclimate evaluation and control in greenhouse cultivation of tomato: a review. International agrophysics 32, 2 (2018).
  • Siscaro et al. (2019) Gaetano Siscaro, Carmelo Lo Pumo, Giovanna Tropea Garzia, Simona Tortorici, Antonio Gugliuzzo, Michele Ricupero, Antonio Biondi, and Lucia Zappalà. 2019. Temperature and tomato variety influence the development and the plant damage induced by the zoophytophagous mirid bug Nesidiocoris tenuis. Journal of Pest Science 92, 3 (2019), 1049–1056.
  • Tantau (1985) H-J Tantau. 1985. Analysis and synthesis of climate control algorithms. In Symposium Greenhouse Climate and its Control 174. 375–380.
  • Van Beveren et al. (2015) PJM Van Beveren, J Bontsema, G Van Straten, and EJ Van Henten. 2015. Optimal control of greenhouse climate using minimal energy and grower defined bounds. Applied Energy 159 (2015), 509–519.
  • van Rijswick (2018) Cindy van Rijswick. 2018. World Vegetable Map 2018: More than Just a Local Affair. Website. https://research.rabobank.com/far/en/sectors/regional-food-agri/world_vegetable_map_2018.html.
  • Van Straten et al. (2010) Gerrit Van Straten, Gerard van Willigenburg, Eldert van Henten, and Rachel van Ooteghem. 2010. Optimal control of greenhouse cultivation. CRC press.
  • Vemula et al. (2016) Anirudh Vemula, Katharina Muelling, and Jean Oh. 2016. Path planning in dynamic environments with adaptive dimensionality. arXiv preprint arXiv:1605.06853 (2016).
  • Wallach et al. (2018) Daniel Wallach, David Makowski, James W Jones, and Francois Brun. 2018. Working with dynamic crop models: methods, tools and examples for agriculture and environment. Academic Press.
  • Wen and Hsu (1991) Ue-Pyng Wen and Shuh-Tzy Hsu. 1991. Linear bi-level programming problems—a review. Journal of the Operational Research Society 42, 2 (1991), 125–133.
  • Zhu et al. (2011) Yingli Zhu, Jingjiang Song, and Fuzhou Dong. 2011. Applications of wireless sensor network in the agriculture environment monitoring. Procedia Engineering 16 (2011), 608–614.