Abstract
An optimal control framework to support the management and control of resources in a wide range of problems arising in agriculture is discussed. Lessons extracted from past research on the weed control problem and a survey of a vast body of pertinent literature led to the specification of key requirements to be met by a suitable optimization framework. The proposed layered control structure—including planning, coordination, and execution layers—relies on a set of nested optimization processes of which an “infinite horizon” Model Predictive Control scheme plays a key role in planning and coordination. Some challenges and recent results on the Pontryagin Maximum Principle for infinite horizon optimal control are also discussed.
1. Introduction
This paper concerns a framework based on dynamic optimization and, in particular, optimal control to support the management and control of resources in a wide range of problems arising in agriculture.
The relevance of optimizing resources in agriculture is undeniable. In fact, the need to feed a growing population on earth while keeping a sustainable equilibrium with the environment is one of the most critical challenges that humankind is facing [1, 2]. Being a fact that this general issue concerns the general problem of food production resources and of global resources management of the earth, and thus, transcends the strict context of agriculture [1–6], it is absolutely clear that the scarcity of resources and the multiple, mostly adverse, impacts in the environment call for advanced management practices in which optimization should be increasingly pervasive and effective [7, 8]. It is not surprising that the research community has been enthusiastically embracing the huge challenges arising in the extremely wide range of research and management problems arising in this context, as it is attested by a vast amount of technical literature.
These challenges span the whole spectrum of relevant issues: from the optimization problem formulation—encompassing the specification of the scope of the interactions to be considered, the modeling of system's dynamics, the problem constraints and performance criteria, and the optimization time horizon—to the approaches to solve it—which may draw from optimization and control theory results, notably the characterization of solutions to their approximation, analysis of sensitivity and robustness of the solution, and the numerical issues arising in the computation procedures—, and to the framework to integrate the generated output in appropriate decision-making and control support systems. All these aspects play an important role in ensuring that this elaborated framework yields outcomes that do not defraud the natural end users' expectations.
This paper reports the recent collaborative research effort between University of S. Paulo (USP) and University of Porto (UP). It draws heavily from the research work that has been taking place at UNESP and USP [9–11], with a significant collaboration with EMBRAPA, on optimizing the usage of multiple agrochemicals to control the proliferation of weeds (whose growth models exhibit resistance) to maximize the yield of crops. This experience, coupled with some optimal control theory developments, notably in optimal feedback control, infinite horizon optimal control, and Model Predictive Control [9, 12–16], at FEUP and UNESP, constitutes the basis for the optimal control framework proposed here.
This paper is organized as follows: in Section 2, we will provide an overview of some research work carried out in optimal control as well as in optimization based decision support systems that have been developed for agriculture. Given the vast amount of literature, we are forced to address a small sample of research work selected for its pertinence to the framework proposed in this paper so that the general key contributions are spanned. In ensuing Section 3, a control problem addressed by USP and UNESP [10, 11] for minimizing the use of agrochemicals in containing weeds while considering the phenomenon of resistance is considered. Here, the benefits of optimization are clear for both the economic and the environment perspectives, in spite of the fact that no specific ingredients were considered for the later. Then, in Section 4, the key challenges and a set of requirements for the framework proposed here are discussed in the light of the lessons extracted in the previous section and of some of the state-of-the-art. Based on this discussion, a layered control architecture based on nested Model Predictive Control (MPC) schemes [13] is presented. This framework involves a number of ingredients developed in previous research work of the authors, notably a MPC framework approximating an infinite horizon optimal control problem as well as the characterization of the associated necessary conditions of optimality [9]. Finally, some brief conclusions are stated in Section 5.
2. Overview of Optimal Control in Agriculture
In this section, we present an overview of a selected number of contributions on how the optimal control paradigm has been considered to solve decision and control problems arising in agriculture resources management. Our selection criteria focus mainly on the effort to address the challenges arising in fulfilling the requirements of such a diverse and complex application domain. Besides the issues intrinsic to optimal control, such as adequate characterization of solutions to these problems with highly nonlinear dynamics and a wide variety of constraints, and computational tractability of the ensuing computational algorithms, there are many other considerations which depend essentially of the problem domain and are critical to meet the expectations naturally associated with the features of this dynamic optimization paradigm. These issues arise at two generally interdependent, levels: (i) specification of the scope of the problem, and (ii) once its scope is defined. In the later, the modeling of wealth of dynamics and constraints so that the essentially relevant features are captured and, at the same time, the tractability constraints are satisfied, and specification of performance functionals and time horizons to be considered are key challenges. In what concerns the former, the extremely vast array of highly intertwined contexts that can be considered, economics, environment, climate, ecology, natural resources (notably, water, and soil), spatial (from farm level to a region or country), and time scales (from short horizon of a few production cycles to long horizon returns), calls for challenges arising in dealing with concentrated or distributed control problem formulation, specification of architectures, notably the delimitation of monolithic, and the organization of complex systems in multiple interacting subsystems. Another, independent class of issues concerns the deployment of the control design process in the context of automation and decision-making processes that naturally arise at the various levels.
The promising perspectives of optimal control to determine control strategies for resources management in agriculture were tangibly recognized in the pioneer work of Zilberman in [8] back in 1982. In this seminal article, he provides a few sketchy statements of optimal control problems in agricultural economics in which the versatility of the formulation is regarded as a key advantage in capturing the physical dynamics for monolithic and distributed systems and systems with delays. It pinpoints some key challenges as, notably, the need of accurate models what, in turn, entails the need of massive data, as well as the need to tackle the usual high computational complexity. However, the author rightly asserts that these drawbacks will be overcome by the technological advancements in sensing, automation, and computation which will enable the exploitation of the relevance of optimal control theory. Although in a more restricted context, this vein is in some sense prolonged in [7]. This paper discusses a number of pertinent issues concerning the role and potential in the engineering of systems for greenhouse climate control in precision agriculture of some techniques of control theory organized in two large classes: conventional control and optimal control. After pointing out some challenges which include modeling, variability, uncertainties, perturbations, and computational complexity, the authors draw the attention for the increasing relevance of Model Predictive Control in addressing most of these challenges, particularly in the light of the recent developments in sensing and computational advances.
In light of these considerations, we organize this review as follows. We start by examining the effort in straightforward monolithic optimal control problems, and then we move on to “generalizations” of this to more complex contexts from which the need of architectural considerations will arise, and, finally, we will consider work concerning decision-making and automation processes that are relevant to optimal control.
2.1. Optimal Control Problems
In [17], an optimal control problem is formulated in order to find the combination of chemical and nonchemical control measures optimizing the long-term tradeoff between short-term profits and long-term adverse effects of herbicide resistance of ryegrass which is quite common in wheat crops in Australia. A simple discrete time, yet representative, optimal control problem is formulated whose solution shows that, for the considered control costs structure, the optimal strategy consists on a combination of both nonchemical and chemical controls. However, as wheat crop-field-level studies in [18], ryegrass herbicide resistance may develop very rapidly if a sufficiently high weed mortality is not guaranteed. This may happen if herbicide rates are very low and no additional control measures are adopted.
In [19], both static and dynamic approaches to weed control by applying a single herbicide are considered. For the dynamic case, optimal control problems are formulated for both deterministic and stochastic contexts. The Pontryagin Maximum Principle and dynamic programming are used in solving the considered optimal control problems. On the top of relevant considerations concerning the effects of uncertainties and the effect of the weather conditions in the herbicide dose response function, the authors conclude for the representative case of wild oats that the dynamic approach is significantly superior and, thus, a weed integrated management system should incorporate such approaches to determine a mix of weed control options achieving the optimal level of control.
In [20], very simple infinite horizon optimal control problems—deterministic and stochastic—for the optimal management of invasive weeds are considered. The Pontryagin Maximum Principle is used to determine bang-bag type of controls expressed in terms of simple rules specifying the types of land where an invasive weed should be first prevented from establishment and under what conditions control should be initiated. By weighing the marginal benefits and costs of various control actions, three potential control strategies are possible: containment, eradication, and no action.
In [10, 11], a dynamic optimization problem is formulated to determine the optimal—in the sense of maximizing the crop economic return over a number of production cycles while minimizing environmental impacts—application of two herbicides to control the infestation of the weed Bidens Subalternans in a corn crop, by taking into account the resistance effect that develops over time. Since integer and continuous variables are present, the problem is cast in the framework of mixed-integer nonlinear programming (MINLP) and solved with a branch and bound method. Thus, this work improves in different ways on the ones in [17, 19] and will be the subject of a more detailed discussion in the next section.
With the goal of maximizing crop returns with minimal environmental effects of pesticides, in [21], the authors formulate an optimal control problem to define optimal management strategies for controlling multiple interacting pest populations affecting the agricultural production. The Pontryagin Maximum Principle is used to derive theoretical solutions for the problem which illustrates the impact of multiple species on optimal pest management strategies.
Optimal control plays a key role in the definition of a management strategy developed in [22] to maintain the density of the pest population in the equilibrium below some economic injury level. The optimal pest strategy is organized in two components. While in the first one, the Maximum Principle of Pontryagin is used to drive the pest ecosystem equilibrium to the desired level, dynamic programming is used to stabilize the ecosystem level by minimizing a quadratic cost functional penalizing deviations of this level.
In [23], an optimal control problem is formulated to determine the optimum crop return by applying pesticides and fertilizers in a ecosystem with two species in a predator-prey relationship: mice feed on grain and grain feeds on a resource. A thorough analysis based on the Maximum Principle of Pontryagin enables the authors to discuss optimal control policies leading to a short-run ecosystem equilibrium that depends on the use of fertilizers and pesticides, and thus, to the corresponding agricultural production function. The authors conclude that if present-value profit is sought, then pest eradication is never optimal and, depending on the prices of pesticide and grain, it is either optimal to use no pesticide or a moderate amount of pesticide or to apply a chattering control. However, if only the steady state of mice population is taken into account, then it may be optimal to eradicate mice or, depending on the prices as well as on the shape of the grain production function, to use no or a moderate amount of pesticide.
Impulsive pest control strategies have also been considered in order to address problems in which it is important to consider control actions for problems with some very fast dynamics leading to discontinuous trajectories. An integrated pest management model impulsively controlled by biologically and chemical means is considered in [24]. The controls act in a periodic fashion, and a nonlinear incidence rate accounts for the effect in the dynamics caused by the biological control application. The Floquet theory for impulsive ordinary differential equations is employed to obtain a global asymptotic stability condition of the susceptible pest-eradication solution in terms of a certain inequality. Along the same vein, the Floquet theory is also used in [25] for sugarcane borer pest control. In [26, 27], the authors present an analysis of the dynamics of an impulse control model for pest control by combining pesticides and biological control that takes into account the pest population age structure and delays. It appears that there is still a large scope for the application of sophisticated results available in the impulsive optimal control literature (see [28, 29], and references therein).
While the spatial distribution was assumed irrelevant in the research so far considered in this survey, this is not the case in many real-life problems. The dynamics of both crops, weeds, and pests might differ in space due to terrain morphology, variations of the soil composition, sun exposure, and ground water distribution, among others. In [30], the optimal control of a spatially distributed bio-invasion is considered being the considered area organized in a finite number of disjoint cells which may either have been invaded or not. The optimal control policy is obtained by solving an integer-programming problem in which the cost function is the sum over time and space of control costs and invasion damages, and the constraints include boundary conditions, the discrete bioinvasion dynamics, and control constraints. The authors reach very interesting conclusions on how the optimality of the control policy depends strongly on its spatial distribution. Delays should also be incorporated in modeling, and, thus, have a significant influence in the definition of optimal control strategies. An interesting problem of controlling and eradicating the infestation of wood boring beetle is considered in [31]. A key feature of the model consists in incorporating two delays: detection of infested trees and the development time of the beetle from egg to adult. After a mathematically rigorous modeling, the authors present and discuss a strategy for the eradication of the infestation which essentially consists on eliminating the infested trees. Some simulation results are presented.
In [32], an optimal control problem is formulated and the Maximum Principle of Pontryagin is used to define a feedback crop thinning strategy that maximizes profits in the agriculture, horticulture, and viticulture settings. This is achieved by, when required, thinning the crop until the optimum natural growth and/or decay equilibrium is achieved. In [33], a simple discrete time optimal control problem is formulated to find control strategies for the photoperiod and light intensity that maximize the wheat crop growth in a fully controlled environment—an advanced life support system. Two different dynamic control models were considered, and conclusions on the robustness of the solutions are discussed.
A finite horizon optimal control problem that takes into account both the profits from agriculture as well as the costs associated with soil erosion and lake restoration from phosphorus runoff is considered in [34]. A method based on problem decomposition with simpler subproblems coupled by constraints is used to solve this problem and leads to the conclusion that the problems of soil erosion and phosphorus runoff from agricultural land cannot be dealt separately and points to the need of an integrated approach to soil management and pollution control with on-site and off-site measures in order to obtain an efficient resource allocation management strategy.
In [35], the problem of optimal carbon sequestration in agricultural soils (either by reduced tillage, organic farming, and other carbon input to the soil techniques, switch to perennial crops, etc.) that takes into account the fact that carbon desequestration is far faster than sequestration is formulated. A two-stage optimal control problem with a dissymmetric dynamic process is solved to determine the optimal sequestration/desequestration strategy for specific benefit, damage, and cost functions. An optimal carbon sequestration strategy is also considered in [36] by solving an optimal control problem in which a model of carbon sequestration and energy abatement is used to explore the potential role of forests in greenhouse gas mitigation. An optimal strategy to switch the use of land from forest to agriculture while ensuring sustainable development is considered in [3] by formulating and solving an optimal control problem. Here, it is shown that there is a minimum level of agricultural land reconversion rate required to reduce forest land conversion, and, moreover, the success in the fight against deforestation ensuring economic growth requires the development of off farm activities, the increase in agricultural yield, and the adoption of labour intensive technologies.
2.2. Control Architectures
The abstract preference for optimization based approaches is consensual. However, the path to their deployment is far from simple and from consensual. It is intuitive that a systemic methodology is essential to ensure that models incorporate all the significantly determining factors from an exhaustive analysis, which might encompass a number issues that lie of social, legal, and administrative natures that have to be complied with. Thus, an important challenge is how to address the large complexity in a proper and insightful characterization of the control problem solution based on the state-of-the-art optimal control results with practical tractable computational procedures.
One of the key issues is that we are dealing with an important economic problem requiring the optimal management of alternative production technologies which will have to be associated with intertemporal control strategies for alternate dynamical systems. References [37–40] surveyed in this subsection deal with the advances in optimal control that resorted to architectural arrangements in order to deal with these issues.
In [39], a systemic approach is adopted in order to formulate optimization and optimal control problems to optimize the economic valuable botanical yield components based on a functional-structural plant growth model in terms of the source-sink dynamics. This model encompasses all pertinent ingredients which include both botanical (the set of organs of the plant) and ecological (all biotic factors affecting the plant’s growth, such as, among others, weeds and pests) yield components, as well as all other environmental factors (water, light, temperature, humidity, soil composition, etc.). In order to deal with the complexity, the author organizes the overall problem in a hierarchic structure that takes into account the network of interacting factors considered in the overall model. First, for a given set of environmental conditions, the endogenous factors are optimized, and, then, for the given plant genotype, the exogenous environmental factors are optimally controlled. Regarding environmental biotic factors, the interactions between the plant growth and pests which, in turn, have their own development dynamics, were considered. A key challenge here is to ensure the compatibility of this model with the plant model in terms of spatial and temporal scales. The optimization results can provide guidelines to improve cultivation modes.
The architectural arrangement adopted in [39] follows closely the one proposed in [40] in a more abstract context in economics. Here, in order to optimize investment decisions of a firm whose capital goods are subject to a delivery lag, an optimal control problem with a performance functional that depends on arguments with time-delay is formulated. The author shows that, under a set of reasonable assumptions, this problem can be converted in a two-stage optimal control problem, and necessary conditions are presented for a general two-stage problem with an adjustable switching time. The two-stage optimal control problem has been also considered to deal with the “new technology adoption problem” challenge in [38]. Here, an optimal solution to utility maximization needs to contain information on the time, at which the technology change will take place along with the optimal control strategies before and after the change. In order to incorporate the fact that the technology change can only occur if the “new technology” value function dominates the “old technology” value function after the switch, the value function is characterized by using the fact that its hypograph is a viability kernel of a certain auxiliary problem. In [37], a flexible and efficient computational algorithm based on a set of necessary conditions derived for finite-time multiple-phase systems is presented. Based on necessary conditions of Optimal Control Theory to multiple-phase problems, the insight into their practical management provided by this algorithm is illustrated with an application to a complex crop rotation problem.
Another key general feature that calls for appropriate architectural arrangement when optimizing the management of resources for complex problems such as the ones arising in agriculture is due to the fact that usually decisions affect multiple conflicting interests, different sets of stakeholders, and generate impacts that, in general, manifest in different time horizons.
A simple example of an instance of these challenges was addressed in [4]. Here, optimal control modeling was used to analyze in a dynamic framework how public resources should be allocated to small-scale water protection efforts in agriculture or alternatively to investments in large-scale waste water treatment plants to control point source loads. In the considered context, it is shown how environmental damage from nutrient concentration affects the optimal timing of investment in waste water treatment capacity, and the optimal timing of investment, the rate of nutrient load reduction from point versus nonpoint sources, and the optimal switching policies from control nonpoint pollution only to control of both nonpoint and point sources are determined. However, it is clear that the analysis becomes extremely more complex if a much more general single framework encompassing various of the management and control problems alluded to in the previous subsection is considered.
An excellent analysis revealing the complexity of decision-making and control in agriculture is provided in [41]. Performed in the context of the Australian agriculture, this research shows the need of increasingly adaptive policies in order to take into account the evolution of perceptions of the state of the system and of the intervening processes in the multiple spatial and temporal scales, as well as the increasingly important environment changes of which climate variability plays a prominent role. The results of the analysis show that the usual assumption of perfect information and excessive adaptive performance leads to optimization-driven decisions and strategies that will not be sustainable in the longer term. Thus, the author strongly recommends the adoption of frameworks to represent processes allowing more complexity able to deal with limited information, risk aversion, cross-scale interactions, misattribution of cause and effect, background context, and goals identification. The conclusions do not suggest to preclude or lessen the role of optimization but rather point out to the need to preserve the ability to robustly adapt with a reference to a goal which ultimately should result from social and political considerations.
Although in a much wider context than the one considered in this paper, the work in [1] bears a significant relevance for the definition of an overall large-scale decision-making framework in agriculture as it constitutes a serious warning to the single minded optimization driven design of management and control support systems. In [1], it is argued that the optimal development path in the sense of the maximin criterion of intergenerational justice is too demanding to be practical and even too costly for the economically less competitive. Thus, a development policy that follows the optimal growth approach but adopts certain measures to mitigate both the intergenerational and intragenerational welfare inequalities is suggested.
Finally, a note on the wide range of tools that, over the years, have been developed to support the multiple activities required for decision-making in agriculture in the context of the significant complexity of the processes that have to be taken into account. These range, among others, from weather and climate forecasts [42–44], modeling of a wide range of intervening phenomena (like pests propagation, etc.) [43], to dedicated optimization and intelligent decision-making [43, 45], possibly under partial information [46], and taking into account trends in climate change [44]. This small sample of recent works reveals the huge complexity and sophistication of the tools currently available to support operational management and decision-making and, at the same time, the huge challenges to be faced in the design of a framework to support the optimization of resources in agriculture in the various scopes in which the problem can be considered.
3. Weed Optimal Management
Here, we report on the research undertaken at USP and UNESP with close cooperation with EMBRAPA in which optimal control is used in order to find the control strategy for the application of two herbicides to contain the growth of a given species of weeds so that the economic return of the crop is maximized while minimizing the environmental impacts of the used agrochemicals.
Weeds grow naturally in agricultural soils and are highly competitive for the resources required for the crop growth, thus leading to a reduction in the yield and quality. Moreover, a realistic weed population model should contain both susceptible and resistant subpopulations. The weed resistance to a herbicide is the ability, of some biotypes to survive and reproduce after exposure to this herbicide. The continued use of a given herbicide will provoke the manifestation of resistant biotypes.
A herbicide based weed control strategy should aim at maximizing the crop economic returns with minimal use of herbicides, in order to minimize environmental impacts of excessive agrochemicals, and retard the weed resistance evolution. Thus, the optimal control problem should contain all the ingredients required to achieve these goals by seeking the minimum of the cost functional subject to the various constraints. Its formulation for the particular case of a corn crop and the very common in weed Brazil Bidens Subalternans follows in the next subsections.
3.1. Weed Population Modeling
The population model of the weed Bidens Subalternans follows closely the one defined in [19] for which the resistance phenomenon is considered.
Let denote the production cycle. Let , , and be, at cycle , the seed bank density, the seeds resulting from weed reproduction, and the new seeds added to the seed bank, in , respectively. Let be the fraction of seeds germination. Let and be, respectively, the density seedlings and of mature plants in cycle given in . Let be the dose of herbicide used in cycle , given in liter ha−1. Then, the basic Biden Subalternans population model is given by where is the cycle germination rate of weed seeds, , , and are regression coefficients, is the survival rate of new seeds, is seed removal at harvest , is the import of seeds , is the death rate of dormant seeds, and is the herbicide dose response function, which, according to [47], is given by for some parameters , , , , and which can be determined experimentally.
In order to incorporate the resistance effect, the dominant allele with nuclear resistance of the Bidens Subalternans [48] considered the Fisher, Haldane, and Wright equation [49] for the principles of population genetics yields where and the fractions of dominant and recessive allele in cycle , is the selective pressure function, and is a genetic “buffer”.
Now, by putting together the previous equations, the following nonlinear dynamic control system reflecting the weed population model with the effect of herbicide resistance in the case of a nuclear dominant allele is obtained: where is the control variable, and is given by
with , and is the herbicide resistance .
Remark that this is one of the simplest models incorporating resistance: single herbicide, no biological control, and temporal and spatially homogeneous dynamics. The equilibrium analysis in [10] reveals that this dynamic system has 3 points of equilibrium, one unstable and two stable.
3.2. Problem Formulation
By denoting the control process by , where and are, respectively, the state and the control variables, the Optimal Control Problem for the case of a single herbicide can be stated as follows: where the function in the dynamic equation (8) is specified by (4) and (5), and the running performance functional in (7) is given by , being the output price per unit of the commodity, the production function, the unit cost of weed control, is the fixed costs for the weed control input and production, and the discount rate or opportunity weight. In the control constraints (10), , are herbicide application parameters, and is a certain herbicide application production cycle-dependent constraint.
The optimal control strategy is the herbicide application rate maximizing the performance functional over a given period subject to the given control constraints and the state constraint (9). It is important to note that neither long- nor short-term environmental impacts are explicitly considered in this problem.
The extension of this problem to the case of multiple herbicides, say , is straightforward from the formal point of view: where the new index dependent parameters have the same meaning as in the corresponding single herbicide formulation. Now, for the herbicide , the function defining the dynamics in (12) is given by (4), and the function in (13) by (5).
It is important to remark that now the control space is enlarged in order to allow for the rotation of herbicides in different production cycles with only one herbicide used in each production cycle, and it is assumed that the effect of different herbicides is considered merely additive. This assumption has not been validated experimentally yet and is the subject of current scrutiny.
3.3. Solution Approach and Results
The single herbicide optimal control problem is solved by formulating it as a standard Nonlinear Programming (NLP) problem to which commercially available software packages proved to be efficient.
The approach to the multiple herbicide problem consists in, first, reformulating as a Mixed Integer Nonlinear Programming (MINLP) problem, and, after the elimination of control constraints via a quadratic penalization method, in solving it by the Branch and Bound Method and an Active Set Algorithm (check [10] for details). At each stage of the branch and bound method, an NLP relaxed subproblem is solved. The relaxed problem has the same structure as the multiple herbicide optimal control problem but with the constraints replaced by .
In Figures 1 and 2, we present results on the optimal control strategy and the associated seed bank and resistance evolution for the infestation control of Bidens Subalternans species in a corn crop with a single herbicide, either nicosulfuron or atrazine, for a 10-year period.
Results involving the combination of these two herbicides were also obtained but with a preliminary character. In fact, these results, shown in Figures 3 and 4, obtained with mixed chemicals do not use a correct dose response function. Due to the encountered modeling challenges, the dose response function for mixed herbicide concentration has not been developed yet, and, instead, an interpolation of the dose response functions obtained in a greenhouse environment for single herbicide cases was used. In this preliminary context, which serves as a mere indication for additional ongoing research, the results shown in Figures 3, 4, and 5 were obtained.
Finally, again with the preliminary character remarked above, a comparison of net values for each one of the above cases and for the conventional current practice is shown in Figure 5.
Although limited, a preliminary comparison of the proposed optimization approach with most of the current practice reveals that the optimized mixed herbicide concentration to control the weed infestation yields greater economic returns and that the weed resistance evolution was retarded. In spite of the absence of any specific ingredients for environment recovery (either via adequate constraints or specific terms in the performance functional), the optimization of herbicide application for weed control appears to be more environment friendly than the conventional herbicide usage in the absence of any optimization.
4. A Dynamic Optimization Framework
In this section, we discuss issues for the definition of a dynamic optimization framework based on optimal control. This discussion builds on the lessons extracted from the research reported in the previous section and draws requirements and ingredients from the selected state-of-the-art addressed in Section 2. Some of the challenges identified in Section 2 are discussed in the next subsection from which a number of key requirements to be satisfied by the optimization framework will be extracted. Then, we present a number of ingredients and the architectural arrangement designed to satisfy the identified requirements. Finally, we present some optimal control theoretic developments that will provide the foundations underlying the proposed dynamic optimization framework.
4.1. Challenges and Requirements
In spite of the limited scope of the comparison with alternative approaches for weed management, the conclusions obtained from Section 3 point to a clear advantage of using optimal control. It is also clear from the research effort that the development of a practical tool based on the approach—something that was outside the scope of the work done—would require additional research and development work along a number of directions. Among these, detailed modeling of endogenous and exogenous intervening factors features prominently. In what concerns the former, one could single out the following issues.(i)Detailed consideration of the reproduction cycle phases of crops and weeds. (ii)Dose response function for different herbicides, notably, the interaction between the effects both in the effectiveness in eliminating weeds and in the evolution of the emergence of the weeds' resistance. (iii)Sensitivity of the solution with respect to parameters and modeling uncertainties. With respect to the second class of issues, pertinent issues such as the impact in the solution obtained with our approach of the (i)spatial heterogeneity, such as solar and wind exposure, terrain morphology, soil composition, and groundwater distribution; (ii)meteorological variability; (iii)ecosystem interactions—multiple competing species of weeds, effect of multiple pests—which are not only unavoidable but may lead to the consideration of alternative biological control strategies, possibly coupled with the application of herbicides;(iv)incorporation of environmental effects—soil, groundwater, air, and carbon emission/sequestration—which might make sense to take into account in very different time horizons; (v)boundary effects with neighboring agricultural production facilities; were not considered at all and will certainly lead to improvements and to the application scope of the designed approach to the weed optimal control problem. It is also clear that both sets of issues require not only a much stronger interdisciplinary research program but also a stronger collaborative effort with stakeholders, from which EMBRAPA is clearly singled out.
These concerns together with a compilation extracted from the review of the current state-of-the-art led to the specification of a number of requirements to be satisfied by a comprehensive dynamic optimization based framework to support decision-making and control.(1)Long (say, infinite) time and short time horizon optimal control strategies should be articulated in spite of the, possibly conflicting, goals to be considered in the different time horizons. In this context, the framework should lead to decision-making and control strategies that are optimal or tend to optimality in what concerns both the short-term returns and in the reinvestment of part of these returns to guarantee sustainability which might, for example, involve environment recovery. (2)Scalability in time and space. Scalability is required not only to deal with complexity but also with the heterogeneity of issues to be addressed due to very diverse process dynamics to be considered, goals to be targeted, and performance criteria to be optimized that emerge when both spatial and time scales change. Modularity is an important feature enabling this requirement to be fulfilled. As it has been recognized in some of the surveyed literature, for example, [38–40], a multistage structure is required to coordinate the various modules in order to ensure local strategies contributing to common goals. (3)Coordinated decentralization of the decision and control system. This requirement emerges from the need to take into account local specific issues at a given time scale with shared challenging issues that arise at a possibly different time scale. It should enable to organize the system in a discrete set of “independent” nodes, each one acting with partial information but also in such a way—induced by certain coordinating indicators—as to automatically generate consensus at a higher scale. Pests and weeds, ecosystem's level interactions, effects, and cross-effects of agrochemicals both in the environment, crops, weeds, pests (with particular interest for phenomena of resistance), and soil dynamics are, among others, important phenomena that propagate both spatially and temporally that call for concerted management and control actions at appropriate scales. (4)Adaptivity to take into account climate change trends, other environment changes, economic and social trends, and prospective technological developments. The need of this requirement is well explained in [41]. In fact, decisions are usually made by considering forecasts involving data from the past. By considering models incorporating the most update perceptions provided by the multiple pertinent sciences and technologies for data gathering and actuation, the optimization underlying the processes to support decision-making and control will yield results better adjusted to expectations. (5)Robustness of the solution with respect to modeling uncertainties and perturbations. Data gathering, sensing and computational limitations, and the huge number of factors, some of which unexpected, intervening in the natural and human originated processes of interest entails the omnipresence of modeling uncertainties and perturbations. This requirement is fulfilled by appropriate control systems design which might include multiple feedback control loops as well as appropriate choices of targets and performance criteria. These requirements are described here in general and relatively abstract terms. Obviously, they have to be further detailed in the specific context in which an optimal control based management and control support tool would be designed. This task will be part of the tool in question design activity.
4.2. Control Architecture
In order to fulfill the requirements above, a general layered control architecture depicted in the diagram below is proposed.
It involves the following layers.(i)Planning layer. It considers more global issues and takes information from the overall “system” via the coordination layer and from pertinent external sources in order to generate long-term planning targets that are sent in to the coordination layer. The planning horizon is not necessarily predetermined, and the planning layer will be run whenever significant inconsistencies are detected in the current plan as a result of the scrutiny of the execution provided by the coordination layer as well as significant evolution of pertinent externally generated knowledge. The data at the disposal in the planning layer also enables the update of global models that might be required to preserve meaningful planning targets in the light of prevailing pertinent data and the execution performance of the various subsystems. (ii)Coordination layer. It receives the planning targets from the planning layer and generates shorter term targets for each one of the subsystems being coordinated. It also receives status data from each one of these subsystems, integrates and provides feedback data to the planning layer. Moreover, the coordination layer may play a role in a decentralized generation of “consensus” among the various subsystems inasmuch as the planning targets are not at stake. Notice that these operations may involve optimization procedures. (iii)Subsystem layer or execution layer. In each one of these subsystems, a resources optimization (possibly optimal control) sectorial problem is solved by taking into account the local performance functional, dynamics and constraints, and the indicators provided by the coordination layer and, possibly, with interaction with some neighboring subsystems. In general, the strategies are designed to maximize returns (usually in an economic sense), being “common good” goals, such as environment sustainability, taken into account (in the sense of long-term optimization) via the “guidance” provided by the Coordination Layer. The shifting of coordinating targets might require either operational or structural changes at the subsystem which will have to be taken into account by the corresponding model update.
A number of observations are in order. First this modular organization into subsystems allows to accommodate the spatial heterogeneity as well as the evolution over time that follows from the evolution of the social economical and environmental contexts as well as of the underlying knowledge. This ensures that adaptivity is built in the process of decision-making and control strategies generation at the various levels, as well as the scalability requirement. With respect to this point, we note that the operation of composition can be defined in the sense that this layered structure can be regarded as a single block embodying a certain subsystem in a wider structure with the same architecture. The coordination layer ensures the articulation of short-term goals at subsystem level with long-term goals addressed by the planning level. Remark that the optimization problem solved at this level takes into account the physical and economic feasibility constraints at the subsystem level via the coordination layer. Since each subsystem generates control strategies by solving an optimization problem with “local” data, the indicator targets provided by the coordination layer reveals that, in general, decision-making is decentralized. Finally, the built in consensus generation capability and the adopted optimization procedures at the various levels of the proposed decision-making architecture entails the robustness requirements.
Optimal Control theory has a rich arsenal of tools to address the multiple challenges outlined above. Besides the wide range of paradigms, which include continuum time, discrete event, and hybrid systems, the formulation of the optimal control problem is very general and flexible. It encompasses a wide class of performance functionals, controlled dynamics (ordinary and partial differential equations, among others), state constraints (to be satisfied at given points in time and over time subsets), control spaces (from measurable functions to measures) and constraints, mixed state and control constraints, and isoperimetric constraints, among others. On the other hand, for the most significant formulations, there is a sophisticated body of results that have been proved under very weak assumptions on the data of the problem, thus ensuring their applicability to a wide range of applications. Focussing on the characterization and synthesis of control strategies, we single out the following: (i)Maximum Principle of Pontryagin [28, 50–54], which yields an open-loop control strategy by maximizing the so-called Pontryagin function. This involves an adjoint function which has the useful interpretation of propagating back in time the gradient of the cost functional at the optimum and can be regarded as the gradient of the Value Function (the optimum cost to go) along the optimal trajectory almost everywhere with respect to time;(ii)dynamic programming [54], which provides both a technique for verification of optimality, as well as, a means for the synthesis of the optimal control strategy in a state feedback form. It is out of the scope of this paper to detail the above results, and the readers are directed to the indicated references. The Maximum Principle of Pontryagin will be briefly addressed in the next section. Any one of these classes of optimal control results can be used to provide the control synthesis required in blocks of the above described structure. We consider the so-called Model Predictive Control (MPC), also known as Receding Horizon Control, Scheme (see [15] and references therein), using a combination of these results in order to provide the feedback control synthesis for the both the higher and the lower levels of the hierarchic control structure. While the former integrates the planning and the coordination layers, the later is constituted by the various Management and Control Subsystems as depicted in Figure 6 at the lower level of the hierarchic control structure. The heterogeneity of the Management and Control Subsystems might dictate diverse optimization approaches, such as Dynamic Programming, Mathematical programming, that better fulfills the respective specific requirements.
While the optimization problems, possibly of the MPC type, at the low level of the proposed control architecture is solved for a certain finite time horizon, the MPC scheme adopted for the high level is solved for a very long—ideally infinite—time horizon and its performance functional involves both a running cost and another term that is a function of the ideal desired target state for the overall system. This state could, for example, correspond to an optimal sustainable equilibrium to be attained or approximated asymptotically. A detailed description of this MPC scheme as well as some of the challenges inherent to the approximation of infinite horizon optimization problems by finite horizon ones are given in the next subsection. In [55], the relevance of considering infinite horizon in optimal planning in economics is discussed and the technical challenges are pointed out.
The high level MPC will generate a number of indicators that will translate into targets or constraints to be satisfied or approximated by the optimization problems of the Management and Control Systems at the low level in each one of the production cycles of a certain, fixed, or variable finite time horizon. Remark that the specification of models, functionals, and targets—for example, the desired long-term equilibrium for the high level—can be incorporated in optimization processes at both levels of the structure as a result of the evolution of knowledge and of the effectiveness of the deployed control and management strategies. These changes in the formulation of the optimization problems can be either event-driven in the case of disruptive developments or the result of a periodic review, being the rate at which these changes take place such that the overall stability of the scheme is maintained, and promote the adaptivity requirement of the overall system. On the other hand, the feedback nature of the MPC scheme will endow the overall system with robustness to perturbations.
4.3. Model Predictive Control and Infinite Horizon Optimal Control
In this subsection, we will discuss issues concerning the Model Predictive Control (MPC) scheme considered at the high level of the overall control structure as well as the issues that arise from the fact that it is designed to approximate an infinite horizon optimal control problem.
Essentially, the very basic MPC scheme consists of a recursive procedure in which, once sampled, the state of the system at the initial time, say , the optimal control problem is solved in a given optimization horizon and, then, the obtained control applied during a much shorter time interval (usually designated by control application horizon). At this point, the state of the system is sampled, the whole optimization horizon slides of time units, and the process is repeated. Thus, this MPC scheme is as follows.(1)Initialization: , . (2)Solve the optimal control problem over to obtain an optimal reference open-loop control process . (3)Apply the computed optimal control during the control application time horizon . (4)Sample the state variable at to obtain . (5)Slide the time origin by time units, let , and go to step (2).
Remark that even if the solution to the optimal control problem is open-loop, the periodic sampling of the state variable together with the computation of the associated solution to ensures the closure of the control loop to the required extent. Many variants of this scheme are possible. For example, feedback control can be used in the interval by replacing steps (2) and (3) by, respectively, the following: (2′) Solve over to obtain an optimal reference trajectory . (3′) Compute an optimal feedback control during to track restricted to this time interval. If sampling at step (4) reveals no significant deviations of the sampled state from the expected optimal value , then the usually computationally heavy step (2) can be skipped. Other variations of the scheme—important to ensure adaptivity of the optimization process—may include the possibility of changing either or both time horizons, adapting the model dynamics (say, by running model estimation procedures with the gathered data), and modifying constraint functionals and sets, as well as performance functionals at appropriate times. These ingredients enter in the specification of the optimal control problem , being one of the simplest formulations as follows: where defines the controlled dynamics of the system, is the endpoint cost functional, is the running cost function, and and are, respectively, the pointwise state and control constraints. Optimality conditions are currently available for this problem under substantially weak assumptions on its data.
For further details on the MPC scheme, we point out [12, 13, 15, 56] and the references therein.
Since one key objective of the proposed resources optimization framework is to reconcile long-term goals—such as environmental sustainability, with short-term goals, typically dictated by economic competitiveness—the MPC scheme proposed to the high level of the control structure should generate strategies that asymptotically approximate the solution to an “infinite horizon” optimal control problem that drives the system to the desired equilibrium point that solves a problem of the type,
The function is the term in the performance functional that forces the system to be driven to the desired long-term equilibrium. In order for the MPC scheme to yield solutions approximating the ones of the infinite horizon optimal control problem, the associated optimization problem is defined as follows: where the function is a value function defined by
Under appropriate assumptions, the value function can be obtained by solving an Hamilton-Jacobi partial differential equation: For a good reference see [54]. In general, a solution to this partial differential equations in the conventional sense fails to exist, and the type of solution and the notion of derivative that have to be considered may depend on the ingredients of the problem. Moreover, the huge difficulties arising in the computational tractability in solving this equation are well known (for computational approaches and tools, see [57] and references therein). On top of this, we have the issue of solving this equation in an infinite horizon context. This constitutes a huge challenge in the current state-of-the-art in Optimal Control theory.
In order to investigate an alternative approach to this problem, we have been looking at necessary conditions of optimality for a class of infinite horizon optimal control problems that seem to be particularly well suited for the applications considered here [9]. In this paper, we consider the following problem: where and are compact sets and the remaining ingredients are as mentioned previously. In spite of the significant body of literature on this class of problems (see [9, 55] and references therein), the degenerative effect of the infinite horizon still constitutes a huge challenge. Our goal consists in deriving a Maximum Principle exhibiting boundary conditions at the final endpoint with maximal information. For this purpose, we consider to be an equilibrium point as , that is, there exists a feasible control process such that , and , introduced the notion of directional inclusion at infinity.
Let , a.e. in and be a pointed cone. We say that directionally at infinity, that is, , if where and Then, our necessary conditions of optimality in the form of a Maximum Principle can be stated as follows.
Let the control process be a solution to . Then, there exists a multiplier satisfying the nontriviality condition , as well as the maximum condition, that is, maximizes . in the map Here, denotes either or .
5. Conclusions
In this work, we discussed an optimal control framework to support resources management and control in agriculture and discussed how it satisfies key requirements extracted from previous research on optimal weed control and from some pertinent literature. The proposed multilayer control structure relies on nested optimization processes, being the optimization at the planning and control layers provided by an “infinite horizon” Model Predictive Control scheme. The underlying optimality conditions were briefly outlined and some of the key challenges listed. In fact, we regard this paper as a roadmap pointing out to a number of research issues which may lead to tools to design decision-making, management, and control support systems with the versatility required by the huge diversity and complexity of the problems arising in the management of resources in agriculture.
Acknowledgments
All the authors gratefully acknowledge the support of University of São Paulo and of University of Porto under the joint project USP-UP “Controle para a Optimização de Recursos na Exploração Agrícola.” The first three authors also acknowledge the support of the FCT funded R&D project PEST-OE/EEI/UI0147/2011. The second two authors also acknowledge the support of FAPESP Grant no. 10/12360-3 and CNPq Grant no. 306466/2010-4, and the last author was supported by CNPq Grant no. 309335/2010-4 and São Paulo State Foundation (FAPESP/CEPID) Grant no. 2013/07375-0.