Model Embedded DRL for Intelligent Greenhouse Control

12/01/2019 ∙ by Tinghao Zhang, et al. ∙ 13

Greenhouse environment is the key to influence crops production. However, it is difficult for classical control methods to give precise environment setpoints, such as temperature, humidity, light intensity and carbon dioxide concentration for greenhouse because it is uncertain nonlinear system. Therefore, an intelligent close loop control framework based on model embedded deep reinforcement learning (MEDRL) is designed for greenhouse environment control. Specifically, computer vision algorithms are used to recognize growing periods and sex of crops, followed by the crop growth models, which can be trained with different growing periods and sex. These model outputs combined with the cost factor provide the setpoints for greenhouse and feedback to the control system in real-time. The whole MEDRL system has capability to conduct optimization control precisely and conveniently, and costs will be greatly reduced compared with traditional greenhouse control approaches.



There are no comments yet.


page 2

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Greenhouses have been widely used in agriculture, especially in norther part of China, because of its high efficiency and less weather dependency. Traditional greenhouse management requires operators to monitor crops status by going to the greenhouses in person. Besides, operators can hardly offer optimization control for the greenhouse environment (GE) until they have accumulated enough experience about the crops growth. Once the environment of greenhouse changes, previous experience may not be effective any longer. Therefore, traditional control methods are neither manpower nor financial efficiency. How to regulate the greenhouse conveniently and efficiently has attracted many researchers’ attention. Recent years, fast development of internet of things (IoT) allows to supervise the greenhouses by remote monitor[7, 6, 11]. Data collected by sensing network are uploaded to cloud platform, where it can be monitored in real time. At the same time, operators can remotely control the GE.

In terms of control methods, expert systems can make decisions like human and were used to control the environment for certain crops[5, 14]. These systems only considered one or two environmental factors, hence limiting its accuracy of GE control. Recently some researchers have applied popular control algorithms like proportional integral differential (PID) algorithm into greenhouse control[15, 3], while others have trained ANN with the help of other computing algorithms and gotten the control models for GE[8, 1]. Literature in [13]

used machine learning to estimate the temperatures, energy lost and exchange based on outdoor environment. However, few literatures have given suggestions on setpoints determination for greenhouse. What’s more, none of them has considered the relation between crop growth and GE control. Specifically, the demand of crop growth on the environment can be various under different conditions. What’s more, the selection of setpoints for GE is a trade-off between crop growth and costs to achieve optimization choice for smart greenhouse cultivation systems.

In this paper, we propose to apply MEDRL to determine the setpoints for GE. Embedded models are trained to forecast the crop growth so that DRL can analyze complex cultivation systems more deeply and balance crop growth and costs when making decisions. The ideas of growth modeling are greatly adaptive and can be used in various kinds of crops. Besides, image processing algorithms are employed to extract specific features of crops, which will be a part of input for DRL models, from images of crops. In order to formulate MEDRL models as most accuracy as possible, pattern recognition based on computer vision techniques is adopted to identify growing periods and other differences (e.g. sex) of the crops. Crops will have different MEDRL models based on these differences. For implementation, we designed a hardware framework with three feasible schemes for choosing control devices. In conclusion, the MEDRL-based greenhouse environment control system combined with crop growth embedded modeling is proposed in this paper, and also being implemented in the reality.

Figure 1: MEDRL Framework for Greenhouse Environment Control

2 Framework of MEDRL Model

2.1 DRL model for Greenhouse Environment

The environment in greenhouse is very complex, so even experts in this field don’t have enough confidence to decide the most reasonable setpoints for GE. Advanced techniques are necessary for deeply understanding greenhouse systems.

Reinforcement learning (RL), a novel machine learning algorithm, has been proved to be effective in optimization control[12]

. DRL combines the merits between deep learning (DL) and RL

[10], and has been successfully applied into many research areas such as autonomous driving[4], game playing[2], decision making problems[9] and so on. Therefore, DRL is applied in this paper to analyze the greenhouse system and provide setpoints for each environment factors.

In order to apply DRL in smart greenhouse system, three aspects of information are needed to be defined: state, action and reward function (The reward function). As mentioned above, the state is the input of the DRL model and should describe the controlled environment as accurately as possible. Action represents what the agent will do after receiving the state. The reward function is to evaluate this action.

In this paper, the state is the morphological characteristics of the crops, such as leaf area, stem length, number of leaves and volume of flow. Extracting these information from the crop images may require image processing algorithms. In the respect of the action, four physical variables, including carbon dioxide levels, light intensity, temperature and humidity, are taken into consideration. The reward function contains two parts: growth situation and corresponding costs,


where denotes the reward function, is the costs, is growth situation, and are the coefficients. Intuitively, it is a trade-off problem to decide the values of and .

It should be noted that crops have different growing periods. For example, Industrial hemp, has four growing periods during its life, germination, seedling, mature, and blooming stages. For those periods that haven’t bloom yet, the sum of stem length and the number of leaves is regarded as the growth situation. In contrast, the volume of the flower represents the growth situation. Costs mainly come from electricity bills when controlling sensors, cameras and heaters. Generating carbon dioxide or controlling air flow also has costs. The values of and can be determined by further tests. Figure 1 shows the framework of the DRL control for GE, where T means temperature, H means humility, and L means light intensity.

Figure 2: Training the Crop Growth Model

2.2 Crop Growth Modeling

As is shown in Figure 1, a crop growth model is embedded in the DRL model. Specifically, the agent gives the setpoints for each environment variable during the DRL training period, and the growth situation can be obtained by a embedded crop growth model. Then the growth situation combined with costs is used to calculate the reward value. However, most simulation approaches for the crop growth are based on expert models, which need rich knowledge about agriculture and are not universal for different kinds of crops. Therefore, we resorted to ML algorithms to forecast the crop growth under different crop environments.

Figure 3: Industrial Hemp in Different Growing Periods: (a) Female Plants; (b) Male Plants.
Figure 4: Growing Period and Sex Classification using CNN

Specifically, the collected data are divided into different classes based on their growing periods, and each growing period has its own growth model. Because the growth situation is required after the agent takes an action, the input of the model is supposed to have the same variables as the action has. Morphological characteristics are also be included as input. Moreover, the crop growth usually takes a long time, and not only current environment but also the historical environment may have influence on it. Therefore, the input can have both current and historical information of the crops. Besides traditional ML algorithms, recurrent neural network (RNN) and its improvement like long short-term memory network (LSTM) are also a good choice for us to train the model, as it can analyze the data on the current and past time.

Figure 2 shows the way to train the model, where denotes the number of samples that used to train the model in one step, and

denotes the total number of the samples. The loss function is defined as the mean square error between the true and predicted values. With the help of embedded models, DRL can be trained in a more reasonable way.

3 Growing Period and Sex Classification

Unique DRL and growth models should be formulated for different growing periods, as crops physiology varies. In addition, sex classification also plays an important role in practice. Take industrial hemp as an example, female and hermaphrodite crops are known to have much higher economic value than male ones so it’s crucial to control the ratio of male and female crops. Therefore, growing periods and sex classification are very necessary to provide precise control and increase the production.

However, it’s difficult for people without professional experience to recognize growing periods. For instance, Figure 3 is the photos of the industrial hemp under plastic shed. Most people would feel that the crops with the same sex in different GPs have little difference. Therefore, we apply deep learning (DL) techniques to conduct this pattern recognition task automatically.

Convolutional Neural Network (CNN) has been widely used and has shown a better performance over many traditional ML algorithms in the computer vision area, so we apply CNN to conduct growing periods and sex classification. Figure 4 is a framework of the recognition process, where and are two thresholds. Sex of industrial hemp can be known in the mature stage, so classification is conducted during that period. Noted that for some kinds of crops like industrial hemp, the seedling period and germination period can be basically recognized by stem length. Therefore, we simplify the task into a binary classification problem, which saves the training costs without reducing the recognition rate.

4 Hardware Configuration

We designed a overall hardware framework for implementing proposed control system as shown in Figure 5. Environment factors including carbon dioxide, light intensity, temperature and humidity are collected by sensors and sent to the microcontroller unit (MCU). Raspberry Pi (RPi) is the connection between MCU and the platform. After receiving the data, MCU sends the data to the cloud platform through RPi. Images of the crops collected by cameras will be sent to RPi directly. Image processing, DRL control, growing periods classification are all conducted in the cloud platform. The setpoints given by DRL models are in turn received by MCU through RPi. Based on the current intensity and types of devices, we came up with three types of control devices. Four types of controllers will then tune the GE to reach the setpoints. In addition, sensors network configuration is shown in Figure 6. The connection approaches between different sensors and devices include I2C, serial port protocol, and Wifi or IoT.

Figure 5: Overall Hardware Framework
Figure 6: Sensor Network Architecture

Precise control is required after DRL offers the setpoints for GE. A mixture of three schemes are used in this control system.

  • Control Cabinet with Strong Current Control: Strong current control has high stability and technological maturity. Large electric fans, heaters and artificial illuminators can also be driven by it. However, it is not easy to realize intelligent control by related algorithms.

  • STM32 & RTOS with Weak Current Precise Control: STM32 with RTOS is very precise, but the capability to drive high power loads is very poor. In this case, most controllers are low powered and control circuit will be complicated.

  • Arduino with Weak Current Imprecise Control

    : The precision of Arduino is not as high as that of STM32. But it’s easy for people to learn it because of its open source.

The hardware framework is currently being built based on above schemes.

5 Conclusion

In this work, a model embedded deep reinforcement learning (MEDRL) control framework was proposed for intelligent greenhouse environment. The embedded crop growth models were formulated using ML algorithms, which enjoy more generalization ability than traditional experts models and can be simply migrated in many kinds of crops. Besides, computer vision algorithms were applied to recognize, and we will formulate crop growth models and DRL models for the crops with different growing periods and sex, respectively. It was a refinement of the modeling process and also enhances reliability for DRL control. A hardware framework is being implemented in a greenhouse environment with industrial hemp growing. The whole system has a great potential to control the costs of greenhouse production, hence a good application in real life.


  • [1] Castañeda-Miranda, A., and Castaño, V. M. Smart frost control in greenhouses by neural networks models. Computers and Electronics in Agriculture 137 (2017), 102 – 114.
  • [2] Chaplot, D. S., and Lample, G. Arnold: An autonomous agent to play fps games.
  • [3] Chaudhary, G., Kaur, S., Mehta, B., and Tewani, R. Observer based fuzzy and pid controlled smart greenhouse. Journal of Statistics and Management Systems 22, 2 (2019), 393–401.
  • [4] Hilleli, B., and El-Yaniv, R. Toward deep reinforcement learning without a simulator: An autonomous steering example.
  • [5] Kano, A., and Shimaji, H. Green environmental control system with a crop model and an expert system. Acta Hortic. 230 (1988), 229–236.
  • [6] Kodali, R. K., Jain, V., and Karagwal, S. Iot based smart greenhouse. In 2016 IEEE Region 10 Humanitarian Technology Conference (R10-HTC) (Dec 2016), pp. 1–6.
  • [7] Li, R., Sha, X., and Lin, K. Smart greenhouse: A real-time mobile intelligent monitoring system based on wsn. In 2014 International Wireless Communications and Mobile Computing Conference (IWCMC) (Aug 2014), pp. 1152–1156.
  • [8] Luan, X., Shi, P., and Liu, F. Robust adaptive control for greenhouse climate using neural networks. International Journal of Robust and Nonlinear Control 21, 7 (2011), 815–826.
  • [9] Lyu, D., Yang, F., Liu, B., and Yoon, D. Logic-based sequential decision-making.

    Proceedings of the AAAI Conference on Artificial Intelligence 33

    (07 2019), 9995–9996.
  • [10] Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A., Veness, J., Bellemare, M., Graves, A., Riedmiller, M., Fidjeland, A., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., and Hassabis, D. Human-level control through deep reinforcement learning. Nature 518 (02 2015), 529–33.
  • [11] Reka, S. S., Chezian, B. K., and Chandra, S. S. A novel approach of iot-based smart greenhouse farming system. In Green Buildings and Sustainable Engineering (2019), pp. 227–235.
  • [12] Sutton, R. S., Barto, A. G., and Williams, R. J. Reinforcement learning is direct adaptive optimal control. IEEE Control Systems Magazine 12, 2 (April 1992), 19–22.
  • [13] Taki, M., Mehdizadeh, S. A., Rohani, A., Rahnama, M., and Rahmati-Joneidabad, M. Applied machine learning in greenhouse simulation; new application and analysis. Information Processing in Agriculture 5, 2 (2018), 253 – 268.
  • [14] Tchamitchian, M., Martin-Clouaire, R., Lagier, J., Jeannequin, B., and Mercier, S. Serriste: A daily set point determination software for glasshouse tomato production. Computers and Electronics in Agriculture 50, 1 (2006), 25 – 47.
  • [15] Wang, W., Xu, L., and Hu, H. Neuron adaptive pid control for greenhouse environment. Journal of Industrial and Production Engineering 32, 5 (2015), 291–297.