Active Learning with Gaussian Processes for High Throughput Phenotyping

01/21/2019 ∙ by Sumit Kumar, et al. ∙ Carnegie Mellon University 0

A looming question that must be solved before robotic plant phenotyping capabilities can have significant impact to crop improvement programs is scalability. High Throughput Phenotyping (HTP) uses robotic technologies to analyze crops in order to determine species with favorable traits, however, the current practices rely on exhaustive coverage and data collection from the entire crop field being monitored under the breeding experiment. This works well in relatively small agricultural fields but can not be scaled to the larger ones, thus limiting the progress of genetics research. In this work, we propose an active learning algorithm to enable an autonomous system to collect the most informative samples in order to accurately learn the distribution of phenotypes in the field with the help of a Gaussian Process model. We demonstrate the superior performance of our proposed algorithm compared to the current practices on sorghum phenotype data collection.



There are no comments yet.


This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

World population is projected to reach billion by , and yields of most staple crops are not increasing at a fast enough rate to meet the corresponding nutritional needs. To make matters worse, the total cultivable landmass is more likely to decline due to a number of factors like ever-increasing urbanization, non-biodegradable waste generation, etc. Hence, it is important to find genetic varieties of crops with favorable traits like disease resistance, high yield, etc. via plant breeding processes furbank2011phenomics to ensure food security for the next generations. Plant phenotyping evaluates crops based on physical characteristics to support plant breeding and genetic activities. Since the current standard practice in collecting phenotype data needs human specialists to assess thousands of plants, phenotyping is currently the bottleneck in the breeding process. To increase efficiency, High Throughput Phenotyping (HTP) uses sensors and robotic platforms to gather phenotype data mueller2017robotanist .

Current field phenotyping systems rely on exhaustive coverage of the crop field being studied in the breeding experiment. This works well in relatively small proof-of-concept trials but requires significant time and resources for a large field which in turn limits the progress of genetics research. This is especially true for ground-based platforms which can collect higher quality data than aerial platforms, but have much lower coverage rates. With the aim to accelerate the breeding process, we propose an active learning algorithm for an autonomous ground robot to collect a subset of samples with high utility in a short time. The agent models the distribution of a target phenotype of interest in the field with a Gaussian Process and actively selects the next locations to sample instead of exhaustively visiting the whole field.

In HTP, phenotypes like plant height, stalk width, etc. are measured with the help of state-of-the-art computer vision and deep learning techniques 

jenkins2017online ; baweja2018stalknet ; kayacan2018embedded on images captured by on-board cameras. Note that images can be captured by the robot even when it is in motion. This means that as the robot is travelling from one place to another, it can gather phenotype measurements from all the sampling locations or plots along the path. This is different from conventional adaptive sampling where the robot selects some locations to collect high utility data and plans a path to reach them by optimizing a cost function like path length. In this model, the robot learns about the distribution of a target variable of interest by taking measurements at the selected locations and there is no information gain as the robot is travelling from one location to another as it is not taking any measurements along the way.

In contrast, in this work, we consider the case where the robot collects data (through images) not only at some selected locations but also while moving along the planned path in the field as shown in Figure 1. Formally, there are two types of measurements:

  • Static measurements: The robot stops at a sampling location or plot in the field and takes images after orienting itself properly with respect to the plants in order to minimize the measurement noise and gather highly accurate data.

  • Mobile measurements: As the robot is travelling from one point to another in the field, it takes phenotype measurements from the images captured from all the plots along the path without any adjustments.

Naturally, the static measurements are more accurate than the mobile ones due to localization errors and noisy images resulting from robot’s motion on the uneven terrains of the field. In fact, the larger the speed of the robot, the noisier is the mobile data. However, gathering static measurements requires more time and resources than mobile ones. As a result, there is a trade-off between the quality of data and the required resources. Hence, it is important to develop statistical machine learning models and adaptive sampling algorithms to incorporate information gain also from mobile measurements to build autonomous systems capable of high throughput phenotyping.

This paper makes the following contributions:

  • We present an active learning algorithm with a Gaussian Process model that incorporates the two types of measurements with different noise levels and determines high utility samples in order to actively learn the distribution of phenotypes in the field. (Section 3.1)

  • We propose an informative planning algorithm to determine the path with maximum joint information gain resulting from both static and mobile measurements. (Section 3.2)

  • On a sorghum phenotype dataset collected by a robotic platform from a field in South Carolina, USA, we perform a detailed comparative analysis of our proposed algorithm with the current practices in crop phenotyping. (Section 4)

  • We have open-sourced our code repository, simulation environment and the sorghum dataset111Our github repository can be found at for the research community to carry out further work in this direction.

2 Related Work

Gaussian process rasmussen2004gaussian

is a probabilistic non-parametric method widely used for modelling a scalar phenomenon as a multivariate normal distribution over the observable features. By utilizing the uncertainty estimate of this Bayesian model, many approaches have been proposed to address the problem of selecting a subset of available samples with the highest utilities to efficiently learn the scalar field 

krause2008near ; guestrin2005near . Planning paths for a robot to navigate to those sampling locations is called Informative Planning binney2013optimizing ; ma2017informative ; marchant2012bayesian ; marchant2014sequential ; lawrance2017fast . Several metrics have been used to quantify the information gain of a sample or a set of samples, such as mutual information luo2016distributed , entropy lu2014autonomous , Fisher information levine2010information

and the average reduction in variance 

binney2012branch . In this work, we use the entropy as the information-theoretic criteria.

Informative planning and adaptive sampling are widely studied topics in the robotics and machine learning communities. Informative planning algorithms based on a recursive-greedy style  singh2007efficient ; meliou2007nonmyopic have been proposed with a sequential allocation mechanism in order to obtain subsequent waypoints. Low et al. low2009multi presented an entropy based framework which uses dynamic programming to obtain a batch of waypoints for determining the optimal path. Ma et al. ma2016information also proposed a similar informative planning method based on dynamic programming in order to compute the informative waypoints. All these approaches consider a single source of data acquisition and has the same measurement uncertainty for all the samples. Here, we consider two types of measurements with different noise levels resulting in a data-dependent noise model and seek to examine how the noisy data affects the ability of the agent to learn the environmental model. Note that our proposed method is directly applicable in situations where the robot has two sources of data acquisition - an expensive but accurate primary source and a noisier secondary source. For example, a robot equipped with the task of mapping the distribution of temperature in an indoor environment can take accurate thermometer readings from some locations and can also get temperature estimates from images captured by on-board cameras using deep learning techniques chu2018visual . Similarly, in crop phenotyping, penetrometers can be used for accurate measurement of stalk width and images can also provide the same estimates with the help of computer vision techniques baweja2018stalknet .

Binney at al. binney2012branch proposed a branch and bound algorithm for informative planning in a grid environment where each cell is a sampling location similar to the setup considered in this work. However, their proposed algorithm performs exhaustive search over a finite horizon and hence is limited to only small grids. In this work, we decompose the informative planning problem into two parts where the robot first selects some discrete locations to collect high quality measurements (static samples) and then plans a path to maximize information gain from comparatively noisier measurements (mobile samples) along the path too. As a result, our method can be applied to much larger environments than the ones considered in their work.

Mueller et al. mueller2017robotanist presented a robotic platform for efficient crop phenotyping. The robot is capable of collecting physiological and morphological traits of crops. For example, the robot can measure the number and width of stalks, the height of plants, the area and color of leaves, etc. in each plot. The collected data can be used in various ways such as evaluating crops based on physical characteristics and can also be relayed to geneticists who use it to validate hypotheses and determine future plant crosses. Although the robotic platform enables high-throughput and robust phenotyping compared to human labor, its efficiency can be significantly increased by collecting high utility data in a short time which will enable the deployment of such systems to large fields where exhaustive coverage is not possible. Also, the amount of data needed to analyze crop genetics can be significantly reduced by learning the distribution of phenotypes via GP models so as to estimate unobserved data as shown in this work.

(a) Environment
(b) Iteration 1
(c) Iteration 2
Figure 1: (a) The grid environment used in simulation. The red cell depicts the current position of the robot. The black cells are obstacles whereas the rest is free space. The sampling locations or plots in the field are shown in orange. (b) The robots determines some most informative sensing locations (shown as dark green cells) to collect high quality phenotype measurements. It then plans a path from its current position to visit all the sampling locations. As the robot traverses the planned path (shown as a sequence of green arrows), it gathers noisy measurements from all the plots along the way. (c) After completing the traversal, it repeats this process from its current position. For picture clarity, all the static samples collected in previous iterations are shown in light green color. The dark orange colored regions are the areas visited by the agent so far.

3 Methodology

In this section, we describe our Gaussian Process model for incorporating both static and mobile measurements and informative planning algorithm to find the optimal path in the environment obtained by maximizing the joint information gain from both types of samples.

3.1 Gaussian Process Model

Let be the set of all sampling data points in the environment where

represents an observable feature vector. There exists a latent function

that maps the input to the objective value . In our work, the agricultural field is divided into plots and each plot represents a data point. The feature vector comprises of location, vegetation index and mean leaf angle density of the plants in a plot whereas the target function is the mean stalk height in the plot. Formally, and . In Section 5.2, we give a detailed reasoning behind selecting these features as input and output variables of the GP model.

After sampling a set of data points and observing the corresponding target values , where is a noisy estimate of the true value , the robot uses Gaussian Process regression to learn the underlying mapping

assuming the joint distribution of the observed readings is Gaussian. A GP is a distribution over functions fully defined by a mean function

and a covariance function :


The mean function is typically assumed to be zero without any loss of generality. The covariance function , also known as kernel, encodes assumptions about the structure of the latent function

. It describes the relation between two data points and typically has some free hyperparameters to control this relation. We used the popular Matern 

minasny2005matern Kernel as our covariance function:


where is the gamma function, is the modified Bessel function, and and are the output-scale and the length-scale parameters of the kernel respectively. In this work, we used .

Since there are two types of samples here, the associated measurement noise is not same for all the data points but instead depends on the type of sampling - static or mobile. We denote the variances associated with static and mobile measurements as and respectively. One can see that depends on the quality of images captured while the robot is in motion which in turn depends on the speed of robot. For simplicity, we assume that the robot moves at a uniform speed and hence is constant. Furthermore, is also a constant as the robot is at rest while collecting static samples in order to minimize measurement errors and get high quality data.

As the robot collects mobile data while moving in the environment, it is possible that it has acquired multiple measurements for the same feature vector . This happens when the robot travels through a region it has visited before. Since Gaussian Processes are non-parametric method, there can not be two different target values for the same input . We combine all the mobile measurements into a single equivalent measurement by taking their mean value:

where is the number of mobile measurements for data point and are the observed values. This averaging step reduces the noise in the observed data by biasing it towards the true mean value. Furthermore, a robot may have acquired a static measurement for a same data point

for which it has gathered mobile measurement(s) before. Instead of substituting all the comparatively noisy mobile data with a single static data, we fuse them together as the product of the two probability density functions:

Essentially, the cumulative output value is a weighted mean of the measurement from the two sources where the weights are the precision of each source and the resulting variance drops below the measurement variance of each of the sources. If a data point has been sampled by only one medium (static or mobile), then the associated data-dependent noise is either or

depending on the type of sampling. The covariance due to the sample-dependent measurement noise can be represented by using a white noise kernel:


where is the Kronecker delta function which is if and otherwise. We model the inter-sample covariance function as the sum of Matern covariance function and the white noise covariance function:


The hyperparameters of the model can be estimated by maximizing the log marginal likelihood of the observed data as done in  rasmussen2004gaussian . With the GP model fully defined, we can now write the posterior target distribution of a set of samples conditioned on the sampled set as:


where is the pairwise covariance matrix whose element is the covariance between the sample of and the sample of as defined in Equation 4.

The notion of differential entropy is often used in spatial statistic optimization problems to specify the informativeness of a set of samples given a set of previously collected samples. Formally, the entropy of a set conditioned on a sampled set is defined as:

where the conditional covariance matrix is estimated from the posterior distributed as described in Equation 5.

(a) Environment
(b) Graph
Figure 2: (a) In the first step of Informative Planning, the robot determines the most informative samples (here ) in the field which are shown in green. (b) In the second step, it constructs a graph whose nodes are the entry and exit of all the narrow corridors in the field shown in orange plus the static sampling locations determined in the first step shown in green. Path planning in the original grid environment is equivalent to that on this graph however significantly faster.

3.2 Informative Planning

We now describe our proposed two-step informative planning algorithm which takes into account both static and mobile measurements to find the most informative path in the environment. In this work, we use differential entropy of a set of samples as its information gain metric.

In the first step, the robot selects a set of points (see Figure 1) from the unsampled space with the maximum entropy conditioned on the already sampled set . Formally,


where is the power set of . Since, this is an NP-Hard problem guestrin2005near , we use the greedy strategy proposed by Krause et al. krause2008near where the sample is selected from the unsampled set as the one which results in the maximum information gain:


where . The white noise associated with these samples is the static measurement noise, i.e., .

In the next step, the robot plans a path from its current location (see Figure 1) to collect data from all the selected data points while maximizing the joint information gain from both static and mobile samples encountered in the path.

Let, be the set of all possible paths originating from a location in the environment and passing through the plot of all samples . Also, let be the set of mobile samples along the path . The robot determines the most informative path while satisfying the budget constraints:

where is the cost of the path considered to be the length here and

is the budget or the maximum allowed path length. Using the chain rule of entropy, we can write:


The robot aims to find the path which results in the maximum information gain from mobile samples conditioned on the previously sampled set and the static samples . This is expected as all the candidate paths have the same set of static samples so the best path is the one which results in maximum conditional information gain from mobile samples. Normally, the path optimization criteria in adaptive sampling applications is the cost of travel and once a set of sampling locations is determined, the planner generates the shortest route to traverse all of them ma2017informative . However, in our case, due to the presence of an additional medium of gathering data, the robot seeks to find the path which results in joint maximum information gain from both the mediums.

We set the budget to be the shortest path length plus a slack term :


This adaptive budget ensures that there exists at least one path which satisfies all the required conditions. The slack term controls the freedom given to the robot to explore areas not necessarily along its shortest path. If we enforce a constant budget instead, the robot may go to far off places if the shortest path length is much smaller than the budget. Also, there may not be any feasible path if the budget is inadequate. Moreover, in the adaptive sampling cases where the robot is required to do multiple traversals, having a budget that ensures the existence of at least one feasible path while also providing some freedom to explore distant areas is desirable.

Since there is very narrow space between two rows of crops in the field, it is not possible for the robot to take a turn without damaging the plants. This means that once the robot enters the narrow region (corridor) between two rows of crops, it can only move in its heading direction till it reaches the end of it where it can either take a turn or keep moving forward (see Figure 2). This no U-turn constraint significantly simplifies the planning problem by reducing the search space from the entire field to a small graph as shown in Figure 2. The nodes of this graph are the intersection points or junctions in the environment and edges are the straight line paths connecting them. To find the optimal path from the robot’s initial location traversing through the set of location of all the static samples denoted by , we add the initial position of robot and the waypoints to the graph . One can easily see that path planning on this graph is equivalent to that on the original grid environment but highly efficient.

The robot runs a graph search on to determine the set of potential paths which satisfy the budget constraints and then selects the optimal path as per equation 8

. As is common in path planning, we use a heuristic to approximate the goal distances to focus the search in order to obtain solutions faster than uninformed search method 

nilsson1971problem . In our case, there is no explicit goal node but a goal state achieved when all the waypoints have been visited.

We use a simple method to find a lower bound on the distance to cover all the remaining sampling locations. Let, the current position of the robot and the waypoints left to be visited be and respectively. We denote and . We compute a lower bound on the path cost by determining the bounding box formed by and the set of waypoints . Note that the agent’s path will touch or intersect all edges of this bounding box in order to visit all the remaining waypoints. In other words, the agent has to travel at least the distance from its current position to the nearest edge and then to the opposite edge along each of the two axes. Formally, the proposed heuristic cost-to-go can be written as:

The first two terms are the dimensions of the bounding box whereas the last two terms are the distance of the agent from the nearest edge along each axes. This lower bound on the cost-to-go can be further tightened by incorporating the no turn constraint and the initial orientation of the agent in the analysis, however, for the size of environment considered in this work, the proposed heuristic was able to significantly speed up the search and provide optimal plans in real-time.

In this work, we exhaustively compute all the paths which satisfy the budget constraints, however, this can be computationally very expensive as the size of the environment increases. For the crop field considered here, exhaustive evaluation was computationally tractable. For larger environments, however, this step can be the computational bottleneck in the entire pipeline. One way to solve this problem is to construct the desired path in segments. Instead of determining all possible paths from the start location at once, one can first find an order of visiting waypoints by minimizing the total travel cost, for example, and then select the optimal path between two consecutive waypoints in the determined order.

4 Data collection

The phenotype measurements are collected by a robotic platform (see Figure  4) from a sorghum field in South Carolina, USA. The sorghum field we are studying is laid out in the form of a grid of plots. For taking phenotype measurements, the robot exhaustively visits all the plots from North to South (see Figure 4) and captures images in each plot from its on-board cameras. After reaching the end of a column, it shifts two columns East and then moves from South to North till it reaches the end. The robot continues this process till it has covered the whole field. As one can see, this exhaustive coverage strategy is time and fuel consuming as the robot has to stop or move slowly in each plot to get accurate measurements. The objective of this work is to develop an intelligent and adaptive strategy to enable fast data acquisition for further use by scientists to validate genetic hypotheses and determine desirable plant crosses.

As the robot drives through the field, it measures the width, height and the number of stalks in each plot from the images of stalk regions of plants as shown in Figure 5. It also measures the leaf angle, surface area of leafs and the vegetation index which is the ratio of dry to green leaf area from the images of leafs as shown in Figures 5, 5 and 5 respectively. We have developed sophisticated deep learning and computer vision techniques in our previous works to extract these measurements directly from images.

We are interested in studying these phenotype measurements and determining their correlation with the yield produced at the end of the harvest season. With the help of such a correlation model, scientists will be able to determine early in the season which genetic varieties of a crop are expected to give high yield and need not have to wait till harvest to obtain the results of their experiments. Furthermore, by actively taking high utility measurements from some plots only and not the entire field, the data acquisition process can be accelerated and the research work can be further extended to large agricultural fields.

Figure 3: The figure shows the data collection process. The robot (shown in red) takes phenotypes measurements from all the plots in the first column. After that, it shifts to the next free column containing an array of plants and continues in the same manner till it has covered the whole field.
Figure 4: Our robotic platform Robotanist in a sorghum field in South Carolina, USA.
(a) Detected stalks
(b) Leaf angle
(c) Leaf area
(d) Vegetation index
Figure 5: (a) The robot detects stalks and measures their width, height and count in each plot from images. Similarly, (b) leaf angle, (c) leaf area and (d) vegetation index or the ratio of dry to green leaf area are measured from the captured images.

5 Analysis

In this section, we describe our simulation environment, implementation details of our algorithm and the experiments we performed to evaluate the efficacy of our proposed algorithm.

5.1 Implementation

For rapid testing and experimentation purposes, we have built a grid-based simulation environment (shown in Figure 1) which imitates the actual layout of plots in the sorghum field. Each obstacle-free column represents an array of crops planted with intermittent spacing allowing the robot to shift to an adjacent array. The measurement received by a robot in simulation is drawn from a normal distribution with mean as the ground truth value and variance as either static variance or mobile variance depending on the type of sampling. Formally,

where and are the static and mobile measurements respectively and is the ground truth field data for the sample .

Our GP model as described in Section 3.1 is built on top of GPyTorch gardner2018gpytorch

which is a Gaussian Process library implemented using PyTorch 

paszke2017automatic . We used zero mean and Matern covariance function with (see Equation 2) as the mean and covariance modules of the GP model respectively. The hyperparameters of the GP model are learned by maximizing the log likelihood of a subset of training data and are fixed thereafter. The informative planning and graph search algorithms are implemented with the help of NetworkX  hagberg2008exploring package. Our simulation environment, code repository and the sorghum dataset are open-sourced and can be found at

5.2 Experiments

Since true yield data will be available only after the harvest season, we used the mean stalk height in each plot as a metric for quality of produce from that plot. The density of crops is more or less same throughout the field, so the average plant height in a plot directly reflects the expected yield at harvest. The agent aims to actively learn the distribution of mean stalk height as a function of location, mean vegetation index and mean leaf angle density using a GP model. We included the last two features in the set of input variables as sorghum phenotypes are poorly correlated with location as suggested by crop scientists and no meaningful mapping can be learned between them. Both leaf angle and vegetation index are measured from the images of leafs which can be captured by aerial vehicles from the top in a short time. So, we reasonably assumed that the ground robot has access to mean vegetation index and mean leaf angle density in each plot prior to starting its adaptive sampling routine.

In each simulation, the robot starts from the top left corner of the grid as shown in Figure 1 and performs a total of iterations. In each iteration, it determines most informative static sampling locations and the corresponding optimal path (see Figure 1). After collecting all the data from the planned path, it updates its GP model. The robot repeats this process in the next iteration (see Figure 1).

5.2.1 Comparison with baselines

We compared our proposed Maximum Entropy algorithm (MaxEnt) as described in Section 3.2 against four baseline methods:

  • Naive static: The robot sequentially visits all the plots in each column as shown in Figure 4. In order to get accurate measurements, the robot slows down or stops in each plot. In this case, the white noise variance is equal to the static measurement variance for all the samples, i.e., .

  • Naive mobile: This strategy is same as Naive static except that the agent does not stop in each plot and measures phenotypes while moving. The data collected in this process is noisier than that in the previous strategy. Here, the white noise variance for all the samples is the mobile measurement variance, i.e., where .

  • Shortest: Like MaxEnt, in this case, the agent determines the set of all the feasible paths as described in Section 3.2. It then selects the path with the shortest length.

  • Equi-sample: After computing the set of all the feasible paths , the agent selects the path which has the same number of sampling locations as the one selected by MaxEnt algorithm.

Note that MaxEnt, Shortest and Equi-sample are informative strategies, i.e., they adaptively select the next sensing locations and plan paths to maximize information gain instead of naively covering the entire field as done by Naive static and Naive mobile strategies. In Shortest and Equi-sample strategies, ties among multiple candidate paths are broken arbitrarily.

We ran simulations to compare the strategies. In each simulation, out of sensing locations, we reserved a set of randomly chosen locations as a test set denoted by . The robot can not observe this test set while learning the distribution of target variable. The metric used to quantify the performance of a strategy is Mean Absolute Error (MAE) between the true phenotype values and the ones predicted by the model on the test set samples:

where is the predicted mean value by the GP model and is the ground truth phenotype value collected from the field.

For this experiment, we set , and . We would like to mention that the true values of stalk width varies from cm to cm and we did not perform any normalization on the dataset.

(a) Test MAE v/s distance
(b) Number of samples v/s distance
Figure 6: The plot shows the (a) MAE of agent’s prediction on the test set against distance travelled and (b) number of samples collected against distance travelled. For clarity of the picture, only confidence interval is shown.

The graphic plot of agent’s prediction against distance travelled is shown in Figure 6. Each grid cell in the environment corresponds to 1 unit distance. Note that for the same distance, the informative strategies gather less number of samples than the naive ones (see Figure 6) as the planned path often requires travelling along a row in the grid (see Figure 1) where there are no sampling sites. On the other hand, the naive strategies move to the next column only after sampling all the plots in the current column (see Figure 4). In each iteration, the agent collects a variable number of samples, so we decided to use the distance covered as our independent axis in the plot in order to ensure fair comparison among all the strategies.

Unlike the naive strategies, all informative strategies are able to quickly predict the phenotype distribution by actively visiting places with high utility. Our proposed MaxEnt consistently achieves the lowest prediction error on the test set indicating its ability to accurately estimate the target distribution. Shortest and Equi-sample strategies are also able to learn the target distribution and closely match the performance of MaxEnt. On the other hand, the two naive strategies perform poorly and even after covering more than half of the field and collecting more samples that the informative strategies, they are unable to match the predictive performance of the informative strategies. The experiment validates that the agent can learn the target distribution in a short time by actively sensing locations and does not need to exhaustively cover the entire field and collect data from all the plots as done in the current practices.

5.2.2 Effect of noise ratio

We define the noise ratio as the ratio of mobile to static measurement noise, i.e., . Assuming a fixed static measurement noise , the noise ratio depends on the speed of the robot. The faster the motion of robot, the larger is the noise ratio and vice-versa. This is because if the robot is moving fast, then the measurement uncertainty will increase due to increase in localization errors and poor image quality. However, moving too slow is also undesirable as it slows down the data collection process, hence there is a trade-off between the quality of collected data and the time consumed in the process.

We compare the predictive performance of MaxEnt on the test set samples against different values of noise ratio. For this experiment, we set and . We ran simulations and the results obtained are shown in Table 1.

MaxEnt prediction with different
MAE 4.08(0.70) 4.10(0.77) 4.09(0.78) 4.40(0.68)
Table 1: MAE on the test set for and averaged over simulations. In each simulation, the robot travels

distance units. The first term is the mean whereas the one inside the parentheses is the standard deviation of the metric.

We observe that there is not much significant difference in the predictive accuracy of the learned model till . This is an interesting finding as it suggests that the robot does not need to stop or move extremely slowly in each plot in order to learn an accurate model of the distribution of phenotypes in the field but an intelligent and adaptive combination of static and mobile samples can also achieve the same level of accuracy. Since, noise ratio depends on speed, one can estimate the speed at which the robot should move in the field in order to collect samples rapidly without deteriorating the quality of learned GP model. However, there are many other factors that affect the measurement noise like terrain complexity, camera quality, weather conditions, etc. and an in-depth analysis of their effect along with the speed of robot is left for future work.

5.2.3 Effect of budget

In the previous experiments, we set which means that the robot does not have any extra budget to explore nearby areas in each iteration. We would like to mention that even in this case, there exists many candidate paths of the same length because of the structured warehouse-like layout of the environment. Here, we want to understand how does some slack on the maximum allowed path length or extra budget affects the quality of the learned model. Increasing slack does provide some freedom to the robot to explore areas not necessarily along its shortest path, however, that comes at the expense of increased computational cost. Hence, it is necessary to determine how much additional budget, if any, is useful in adaptive crop phenotyping applications while maintaining tractable computational requirements.

We compare the final predictions made by MaxEnt algorithm on the samples in the test set for different values of slack length . We set and for this experiment and ran simulations. The results are shown in Table 2.

MaxEnt prediction with different slack
MAE 4.22(0.58) 4.00(0.51) 4.08(0.54) 4.19(0.40)
Table 2: MAE on the test set for and for different slack values averaged over simulations. In each simulation, the robot travels distance units. The first term is the mean whereas the one inside the parentheses is the standard deviation of the reported metric.

We observe an interesting trend in the final predictions made by the learned model. Increasing the slack value improves the performance of the model, however, the increase is not monotonic, i.e., providing additional budget does not necessarily results in increased performance of the learned GP model for the same distance travelled. This is because samples lying nearby along a path exhibit high correlation due to the local smoothness modelling assumptions of the GP kernel, hence, the information gained along a path does not always increase with increase in path length. As a result, a large budget does not necessarily lead to large improvement in model performance although it significantly increases the total computational cost of evaluating feasible paths. On the other hand, a small extra budget is sufficient for the agent to gather any useful additional information helpful in learning an accurate model of the target distribution while maintaining a reasonable computational load.

6 Conclusion

We presented an active learning framework that alternates between adaptively sampling plots with high utility and learning a GP model of the target distribution for High Throughput Phenotyping. We have also released our code repository for further work by other researchers. Through simulation experiments, we demonstrated the superior performance of our proposed approach compared to the current practices. We also performed several ablation experiments to understand the contribution of various components in the whole planning and learning pipeline.

We believe this is just the beginning and there is a lot of work to be done for developing reliable robotics and machine learning technologies to accelerate the breeding process. Here, we used only one phenotype (mean stalk height) as an indicator of crop health. In future work, we will analyze correlations between multiple phenotypes by modelling them in a Multi-task Gaussian Process framework for active learning and efficient data gathering. Also, we will deploy a team of robots working in a fully decentralized manner to adaptively estimate the quality of produce.


  • [1] Robert T Furbank and Mark Tester. Phenomics–technologies to relieve the phenotyping bottleneck. Trends in Plant Science, 16(12):635–644, 2011.
  • [2] Tim Mueller-Sim, Merritt Jenkins, Justin Abel, and George Kantor. The robotanist: a ground-based agricultural robot for high-throughput crop phenotyping. In Proceedings of IEEE International Conference on Robotics and Automation, pages 3634–3639, 2017.
  • [3] Merritt Jenkins and George Kantor. Online detection of occluded plant stalks for manipulation. In Intelligent Robots and Systems (IROS), 2017 IEEE/RSJ International Conference on, pages 5162–5167. IEEE, 2017.
  • [4] Harjatin Singh Baweja, Tanvir Parhar, Omeed Mirbod, and Stephen Nuske. Stalknet: A deep learning pipeline for high-throughput measurement of plant stalk count and stalk width. In Field and Service Robotics, pages 271–284. Springer, 2018.
  • [5] Erkan Kayacan, Zhongzhong Zhang, and Girish Chowdhary. Embedded high precision control and corn stand counting algorithms for an ultra-compact 3d printed field robot. Proceedings of Robotics: Science and Systems. Pittsburgh, Pennsylvania, 2018.
  • [6] Carl Edward Rasmussen. Gaussian processes in machine learning. In Advanced lectures on machine learning, pages 63–71. Springer, 2004.
  • [7] Andreas Krause, Ajit Singh, and Carlos Guestrin. Near-optimal sensor placements in gaussian processes: Theory, efficient algorithms and empirical studies. Journal of Machine Learning Research, 9(Feb):235–284, 2008.
  • [8] Carlos Guestrin, Andreas Krause, and Ajit Paul Singh. Near-optimal sensor placements in gaussian processes. In Proceedings of the 22nd international conference on Machine learning, pages 265–272. ACM, 2005.
  • [9] Jonathan Binney, Andreas Krause, and Gaurav S Sukhatme. Optimizing waypoints for monitoring spatiotemporal phenomena. The International Journal of Robotics Research, 32(8):873–888, 2013.
  • [10] Kai-Chieh Ma, Lantao Liu, and Gaurav S Sukhatme. Informative planning and online learning with sparse gaussian processes. In Robotics and Automation (ICRA), 2017 IEEE International Conference on, pages 4292–4298. IEEE, 2017.
  • [11] Roman Marchant and Fabio Ramos. Bayesian optimisation for intelligent environmental monitoring. In Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on, pages 2242–2249. IEEE, 2012.
  • [12] Roman Marchant, Fabio Ramos, Scott Sanner, et al. Sequential bayesian optimisation for spatial-temporal monitoring. In UAI, pages 553–562, 2014.
  • [13] Nicholas RJ Lawrance, Jen Jen Chung, and Geoffrey A Hollinger. Fast marching adaptive sampling. IEEE Robotics and Automation Letters, 2(2):696–703, 2017.
  • [14] Wenhao Luo, Shehzaman S Khatib, Sasanka Nagavalli, Nilanjan Chakraborty, and Katia Sycara. Distributed knowledge leader selection for multi-robot environmental sampling under bandwidth constraints. In Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on, pages 5751–5757. IEEE, 2016.
  • [15] Wenjie Lu. Autonomous sensor path planning and control for active information gathering. PhD thesis, 2014.
  • [16] Daniel S Levine et al. Information-rich path planning under general constraints using rapidly-exploring random trees. PhD thesis, Massachusetts Institute of Technology, 2010.
  • [17] Jonathan Binney and Gaurav S Sukhatme. Branch and bound for informative path planning. In Robotics and Automation (ICRA), 2012 IEEE International Conference on, pages 2147–2154. IEEE, 2012.
  • [18] Amarjeet Singh, Andreas Krause, Carlos Guestrin, William J Kaiser, and Maxim A Batalin. Efficient planning of informative paths for multiple robots. In IJCAI, volume 7, pages 2204–2211, 2007.
  • [19] Alexandra Meliou, Andreas Krause, Carlos Guestrin, and Joseph M Hellerstein. Nonmyopic informative path planning in spatio-temporal models. In AAAI, volume 10, pages 16–7, 2007.
  • [20] Kian Hsiang Low. Multi-robot adaptive exploration and mapping for environmental sensing applications. Ph. D. dissertation, 2009.
  • [21] Kai-Chieh Ma, Lantao Liu, and Gaurav S Sukhatme. An information-driven and disturbance-aware planning method for long-term ocean monitoring. In Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on, pages 2102–2108. IEEE, 2016.
  • [22] Wei-Ta Chu, Kai-Chia Ho, and Ali Borji. Visual weather temperature prediction. arXiv preprint arXiv:1801.08267, 2018.
  • [23] Budiman Minasny and Alex B McBratney. The matérn function as a general model for soil variograms. Geoderma, 128(3-4):192–207, 2005.
  • [24] Nils Robert Nilsson.

    Problem-solving methods in artificial intelligence.

  • [25] Jacob R Gardner, Geoff Pleiss, David Bindel, Kilian Q Weinberger, and Andrew Gordon Wilson. Gpytorch: Blackbox matrix-matrix gaussian process inference with gpu acceleration. In NIPS, 2018.
  • [26] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
  • [27] Aric Hagberg, Pieter Swart, and Daniel S Chult. Exploring network structure, dynamics, and function using networkx. Technical report, Los Alamos National Lab.(LANL), Los Alamos, NM (United States), 2008.