#### Abstract

The lack of online information on some bioprocess variables and the presence of model and parametric uncertainties pose significant challenges to the design of efficient closed-loop control strategies. To address this issue, this work proposes an online state estimator based on a Radial Basis Function (RBF) neural network that operates in closed loop together with a control law derived on a linear algebra-based design strategy. The proposed methodology is applied to a class of nonlinear systems with three types of uncertainties: (i) time-varying parameters, (ii) uncertain nonlinearities, and (iii) unmodeled dynamics. To reduce the effect of uncertainties on the bioreactor, some integrators of the tracking error are introduced, which in turn allow the derivation of the proper control actions. This new control scheme guarantees that all signals are uniformly and ultimately bounded, and the tracking error converges to small values. The effectiveness of the proposed approach is illustrated on the basis of simulated experiments on a fed-batch bioreactor, and its performance is compared with two controllers available in the literature.

#### 1. Introduction

In the last decade, several control strategies applied to different bioprocesses have been extensively investigated (e.g., [1–5]). In the particular case of fed-batch bioreactors, some nonlinear control strategies were developed to improve the process efficiency and the “batch-to-batch” reproducibility [6], as well as to deal with the unavoidable uncertainties typically present in bioprocesses [7].

Many articles have investigated the effects of model uncertainties on the performance of model-based nonlinear control strategies typically applied to bioprocesses. For example, the “yield coefficient of cell to glucose” in a bioprocess is a time-varying parameter often assumed as a constant. Also, the “specific growth rate” can be represented through the Monod-Haldane model [8], but it is possible that the model becomes inaccurate because of yeast growth inhibition due to an eventual presence of ethanol. In principle, three main types of uncertainties can be envisaged [9–13]: uncertain and/or time-varying model parameters, uncertain nonlinearities, and unmodeled dynamics. Additionally, nonmodeled external perturbations can disturb the process. In general, model uncertainties and external perturbations can cause severe risks in bioreactors, such as leading to undesirable operating conditions [14–16]. For instance, inadequate modeling of the inhibitory effect of a substrate can lead to a structural instability in the dynamical behavior of a bioprocess [15].

Adaptive control strategies are a valuable choice to control bioreactors due to their ability to compensate for parametric uncertainties. The Nussbaum gain method has been applied to control a bioprocess of unknown and time-varying control gain and parameters [17]. A fed-batch bioreactor control with simultaneous dynamic identification of unknown state and parameters has been investigated through an algorithm based on the auxiliary model principle with adaptive controllers [18]. A control strategy was proposed to track a desired profile for the biomass of a yeast culture in a high-density fed-batch bioreactor [19]. This last strategy was based on a model that included an uncertainty term estimated through an adaptive technique. Also, an adaptive nonlinear feedback controller has been designed to automatically estimate uncertain time-varying parameters in order to guarantee the closed-loop control of a waste water plant [20].

Fuzzy control systems have been successfully applied to several bioreactors. For instance, [12] considered an adaptive fuzzy-decentralized control for a system with unknown nonlinear uncertainties and dynamical uncertainties. The fuzzy system was used to estimate the nonlinear uncertainties. Also, a fuzzy controller for the alcoholic fermentation with* Saccharomyces cerevisiae *has been proposed [21]. It combined the capability of handling uncertainties with the ability of predictive control to improve the plant performance, making use of a neural network model of the nonlinear system. Optimal control strategies have also been applied to bioprocesses [22, 23]. In order to compensate for process uncertainties, this type of controllers incorporates an additional term to the bioprocess model (typically, in the specific growth rate), which is estimated through an optimization technique. Alternative strategies for controlling bioreactors under model uncertainties are trajectory-based control [24], hybrid control [25], and model predictive control [26]. Additionally, due to the inherent nature of a bioprocess, a control scheme based on a stochastic model seems to be adequate. The specific growth rate can be modeled as affected by a white noise representing environment uncertainties, and the mathematical model consists of a set of stochastic differential equations [27] (some applications can be consulted in [28, 29]). The application of this approach onto a real plant is quite difficult due to its complex design and implementation.

Based on a linear algebra methodology, a control strategy has been applied for trajectory tracking in a bioprocess [30]. The technique was used to calculate the control actions that ensure the tracking of independently determined optimal trajectories of some process variables. To this effect, a discrete approximation of the original process model must first be derived, and the control actions are then calculated through a least square problem. This methodology has been applied to a fed-batch bioprocess designed to produce a secreted protein, and it was originally modeled by Park and Ramirez [31]. Optimal profiles of such bioprocess have also been published [32].

Many nonlinear control techniques of bioprocesses assume that the feedback variables can be measured online [33, 34]. However, such hypothesis is often unrealistic and therefore some process variables must be estimated through virtual sensors (or soft sensors). There are several kinds of virtual sensors published in the literature which exhibit different abilities to deal with model mismatches and perturbations. In this work, an artificial intelligence-based observer has been chosen because this type of estimator is especially recommended for highly nonlinear systems in the presence of uncertainties [35].

This paper aims at designing a controller for the multivariable tracking of optimal profiles when the process is affected by uncertainties in the dynamic model and/or in the model parameters. Unmeasured process variables are estimated through a RBF neural network. The proposed approach is evaluated onto a bioprocess originally modeled by [31], with the optimal profiles determined by [32] in absence of model errors. Brief descriptions of the fed-batch bioreactor and the controller derived in [30] are presented in Appendix A.

In comparison with [30], the proposed approach prevents the polynomial uncertainties from affecting the tracking control. To achieve this aim, an integrative term is incorporated in the linear algebra methodology originally proposed. Then, the conditions in order for a system of linear equations to have exact solution are analyzed. The control action is obtained by solving the linear system, even though the original process model is nonlinear.

The main contribution is that the original method proposed herein ensures the convergence to zero of the tracking errors under additive uncertainties. Another important contribution is the inclusion of RBF neural network estimators: the control issue is addressed considering more realistic conditions with nonmeasurable variables. Further, the stability analysis is made using Lyapunov theory in discrete time.

The article is organized as follows. In Section 2, the controller design methodology and the convergence to zero of the tracking errors are presented. Section 3 shows briefly the description of the RBF estimator. A method to tune the controller is presented in Section 4, along with four testing cases designed to check the effectiveness of the new control law. Main conclusions of the work are summarized in Section 5.

#### 2. Controller Design

Derivation of the control strategy is based on the mathematical model presented in Appendix A. The effect of an additive uncertainty () is modeled by extending (A.4) as follows: where is the state vector, and are the input vectors, is the sample time, is the control action, and is an additive uncertainty. In principle, can be used to model either perturbed systems or a wide class of model mismatches. In what follows, a control law is designed to compensate for the tracking errors introduced by .

Taking into account the fact that the mismatch might depend on both system states and inputs, we here consider a real plant described by . The additive uncertainty can be expressed as , where is the discrete-time nonlinear system model. If** z** and** u** are assumed to be bounded and* f* is Lipschitz, then can be modeled as a bounded uncertainty [36–38].

With the additive uncertainty of (1), the fed-batch bioreactor model of (A.3) becomesThe control action for the secreted protein concentration, , given by of [30], isThe discrete value of , , is calculated by replacing (3) into the first row of (2):where and depend on and , respectively. Thus, the tracking error of isBy combining (4) and (5), one obtainsThe Taylor interpolation of at its desired value isBy replacing (7) into (6), the tracking error for* P*_{n} becomesIf a similar procedure is followed for , , and , then the error equation becomes [30]Alternatively, in vector form,where is a nonlinearity that tends to zero when tends to zero. Equations (9a) and (9b) suggest a direct effect of the additive uncertainty on the tracking error. Therefore, the presence of an uncertainty avoids a convergence of tracking errors to zero (see Theorem 1 of [30]).

##### 2.1. Single Integral Action

In order to reduce the effect of , some integrators for the tracking errors will be introduced, depending on the time variation hypothesis of . We will assume that is unknown and each of its components is an -order polynomial.

*Remark 1. *The first-order difference of is defined as , the second-order difference is defined as , and, as a rule, the -th order difference is defined as .

*Remark 2. *The -th difference of a -order polynomial is zero.

Consider first a constant uncertainty . Then, . In this case, an integrator for each state variable will force the error to converge to zero. Call the continuous time error in the state vector. Then, the integral of the tracking error is defined asHence, the control action (A.7) will be computed by assuming a new term in (A.5); that is,where** K** and are the proportional and integral design parameters, respectively.

By adding one integrator to (9b) and using (10), one obtainsRewriting (12) for the next sample time,Combining (10) and (12) and substituting them into (13),Alternatively,Therefore,** K** and are chosen in order to ensure the stability of the linear system represented in the left-hand side of (15). To achieve this condition, the roots of such polynomial must lie inside the unit circle. Then, when ; that is, the error tends to zero as long as uncertainties are constant.

Once the integral of the error has been added, the control action is calculated following the same design procedure based on linear algebra presented in [30]. Therefore, the new control law can be obtained through the least squares procedure as

##### 2.2. Multiple Integral Actions

Assume that the uncertainty can be modeled by a function with a second-order difference equal to zero; that is, . Similar to (10), a double integrator is now introduced, and new variables and are defined:In this case, the control action (A.7) will include an additional term in (A.5) as follows:where** K**,** k**_{1}, and** k**_{2} are the proportional, integral, and double-integral design parameters, respectively. With a similar procedure to that used for deriving (15) and after adding two integrators ( and ), the error dynamics can be expressed as As suggested by (19), under the assumption of a constant or linear varying uncertainty , the uncertainty has no influence on the error dynamics. As in the case of the single integrator, the controller parameters** K**,** k**_{1}, and** k**_{2} should be chosen in order to ensure the stability of the left-hand side of (19).

The previous derivation can be generalized for uncertainties that can be approximated with a -order polynomial; and, therefore, the influence of upon would be cancelled off by addition of integrators.

The linear algebra-based procedure enables the calculation of the following control law:

#### 3. Neural State Estimator

The proposed feedback control technique requires reliable estimates of several state variables. The state estimation error, , is defined aswhere is the vector of estimated state variables and is the vector of online measured states. The following neural estimator represents the nonlinear dynamic of the bioreactor described in (7):where is the input vector to the neural estimator, is the optimal weight vector, is the neural approximation error, is the RBF that represents each neuron in the hidden layers, subindex indicates the neuron number of the RBF, and is the maximum number of neurons. Every neuron of the hidden layer is modeled aswhere and are two parameter vectors that represent the centers and widths of the RBF, respectively.

In the bioreactor under study, biomass cannot be measured online and the protein concentrations cannot be measured at all; then a state estimator function based on (22) is determined as follows:From the difference between (22) and (24), the neural identification error can be described bywhere represents the neural weight vector estimation error and it is defined asIn order to train the neural network for identification, an offline data set of will be used. This training data will be obtained after following the same procedure presented in [39] and then applied in [40, 41]. A similar procedure was also used in [42]. The learning rule utilized to train the neural network is demonstrated in the next theorem.

Theorem 3. *The model of the fed-batch bioreactor (see (A.1) and (2)) can be approximated by the neural network (24) using a neuronal adjustment law defined by*

*Proof. *The demonstration of this theorem is presented in Appendix B.

As recommended by [35], the estimator was evaluated to test its performance and avoid biased estimates. Table 1 summarizes the implemented tests, and their results were evaluated through the average values of the mean squared errors (MSE) corresponding to , , and . The obtained results show an acceptable performance of the RBF neural network for estimating the nonmeasurable states. Note that none of the MSE values exceeds the thresholds proposed by [43], . Figure 1 shows the proposed closed-loop architecture. Note that the controller requires information on both measured and estimated variables.

#### 4. Results and Discussion

All simulation examples are based on the bioprocess originally modeled by [31]. The available optimal profiles for such process are also adopted [32]. Equation (A.1) represents the fed-batch bioreactor model. The effectiveness of the new control law is tested through some simulation examples. First, the optimal controller parameters are tuned through a Monte Carlo experiment. Then, a sinusoidal perturbation is added to some model parameters. Also, the controller performance under uncertain time-varying parameters is verified through a Monte Carlo Randomized Algorithm. In other examples, the functionality of the specific growth rate is changed. Finally, some parameters are contaminated by additive white noise.

As previously discussed, the behavior of the controlled system depends on parameters , where is the number of integrators. For , the controller parameters are determined through a Monte Carlo method, where the number of simulations is determined by application of Theorem of [30]. All simulations are implemented in Matlab™ (MathWorks, Inc.), with a sample time h. For each controller (i.e., for each ), 1,000 simulations are run; and the controller parameters are randomly chosen, such that the roots of the linear systems defined in the right-hand side of (9a) and (9b) and left-hand side of (15) and (19) verify . More specifically, for , the controller parameters are directly taken from [30]. For = 1, 1,000 new simulations are performed, and each controller parameter is chosen according towhere the superscript refers to each state variable and and are the roots of the linear system in the left-hand side of (15). For = 2, 1,000 simulations are performed, and each controller parameter is chosen according towhere , , and are the roots of the linear system in (19).

Table 2 summarizes the results obtained with each controller. For comparison purposes, the performance of the proposed strategy is compared with that provided by a Parallel Two-Degree-of-Freedom PID Controller (2-DOF PID), which computes a weighted difference signal for each of the proportional, integral, and derivative actions according to the set point weights specified by the user [44]. In this case, the culture cell density has been chosen as the controlled variable. The 2-DOF PID was tuned with the automatic tuning of the Simulink™ Control Design software that provides the proportional, integral, and derivative gains, as well as the filter coefficient.

##### 4.1. Time-Varying Parameters and Input Disturbances

As in [18, 45], the following expressions are adopted for simulating uncertainties in the parameters and the feed flow rate:The tracking error is defined asIn what follows, we shall call C0, C1, and C2 the controllers that use = 0, = 1, and = 2 integrators, respectively, and C3 the 2-DOF PID controller. Particularly note that C0 corresponds to the controller designed in [30]. In spite of the assumed time-varying parameters, Figure 2(a) shows that C2 produces the minimum tracking error. According to Figure 2(b), both C2 and C1 are effective to reach the desired protein concentration profile, while C0 and C3 produce large errors.

**(a)**

**(b)**

In several research fields, probabilistic methods have been found to be useful for dealing with problems related to robustness of systems affected by uncertainties [46]. Particularly, the Monte Carlo Randomized Algorithm has been used in many applications such as radioactive decay, power system generation, and traffic on roads [47]. In the control area, Monte Carlo methods are effective tools for the analysis of probabilistically robust control schemes [30, 46]. A Monte Carlo simulation is here performed to demonstrate the effectiveness of C2 from a statistical point of view [48–50], when uncertainties are introduced by time-varying parameter. To this effect, 1,000 random values of and are chosen in a range of ±25% of their nominal values. Figures 3(a), 3(c), 3(e), and 3(g) show that only C2 and C1 were able to adequately track the reference profile. In fact, both controller responses are quite similar, thus suggesting that the time-varying parameter has not affected their performances. For comparison, the tracking error was evaluated through the following performance index :where represents the number of points of the reference profile. Figures 3(b), 3(d), 3(f), and 3(h) confirm a better performance of C2 and C1 over C0 and C3. In fact, is quite small for C2 and C1, even though 25% of uncertainty in the main bioreactor parameters was adopted. Since 1,000 simulations with random values of and were carried out and the tracking errors remained bounded, the performance of C2 and C1 will be acceptable with 99% of probability whatever the parameter values are in a range of ±25% [46].

**(a)**

**(b)**

**(c)**

**(d)**

**(e)**

**(f)**

**(g)**

**(h)**

##### 4.2. Uncertain Nonlinearities

Uncertain nonlinearities are often present in many bioprocesses. For instance, the specific growth rate, , can suffer from important changes from batch to batch, and unfortunately such variability is impossible to be accurately modeled. There are a number of models in the literature for [8], for example, the third expression of (A.2), for the bioprocess currently under study [31]. To demonstrate the controller performance, an additional term can be included in the calculation of as follows [18, 51]:Figure 4(a) compares the performances reached by C2, C1, C0, and C3 under the simulated uncertain nonlinearities in . The lower tracking errors correspond to C2. Figure 4(b) shows that only C2 and C1 are able to reach the maximum concentration of the total protein.

**(a)**

**(b)**

##### 4.3. Unmodeled Dynamics

The presence of white noise in leads to a mathematical model governed by a set of stochastic differential equations. In order to properly deal with this type of uncertainties, a kind of stochastic control approach should be used. As an alternative to the complex theory of stochastic control, a simpler deterministic strategy is here proposed. To this effect, note first that all parameters in (A.1) and (A.2) are truly affected by environmental noise, and consequently they always fluctuate at long times around some average values. For simplicity, we will consider that only is affected by randomness and changes according to the following expression [52, 53]:where is a standard Brownian motion defined on a complete probability space and *α* ≥ 0 represents the intensity of the noisy signal.

A white noise can be approximated through a band-limited white noise (BLWN), which is simply implemented through a proper block of Simulink. Theoretically, a white noise is uncorrelated; it exhibits a flat power spectral density, and therefore it is characterized by an infinite energy. The BLWN block can simulate the effect of white noise by using a random sequence with a correlation time much smaller than the shortest time constant of the system. The correlation time of the noise is the sample rate of the block [54] and can be estimated aswhere is the correlation time, is the bandwidth of the system, and is the maximum sample time for which the system is stable. For the feedback control presented in this work, = 0.0145 h. Then, from (35), = 0.0018 h.

Many researchers have utilized this methodology to prove the robustness of the bioreactors controllers [19, 22–24, 55]. In the current simulations, a BLWN with a noise power of 0.9 and *α* = 0.5 is introduced along with the feedback signal. Thus, the maximum specific growth rate affected with BLWN is calculated asFigure 5(a) indicates that C2 and C1 adequately work in the presence of unmodeled dynamics. In fact, the tracking errors are quite small in comparison with C0 and C3. Figure 5(b) shows the profile tracking for each controller. Again, C2 and C1 are able to follow the reference profile, while C0 and C3 fail to reach the maximum protein concentration.

**(a)**

**(b)**

#### 5. Conclusions

A new control law for tracking an optimal concentration profile in a fed-batch bioreactor has been presented. The proposed method allows controlling a nonlinear system in presence of unavoidable uncertainties in bioreactors: time-varying parameters, uncertain nonlinearities, and unmodeled dynamics.

In order to deal with these uncertainties, some integrators have been added to the original controller proposed in [30], and a Radial Basis Function (RBF) neural network was employed to estimate , , and . The integrators have been chosen based on the variation hypothesis of the modeling error : if a -grade polynomial can approximate , then integrators will be added in order to avoid its influence on the tracking error .

Several tests were carried out to demonstrate the effectiveness of the proposed methodology. The controllers are tuned through a Monte Carlo experiment and under stability conditions. To evaluate the performance of the designed controllers, different uncertainties were simulated. It was proven that the proposed methodology successfully compensates the uncertainties, and the tracking errors decrease as long as the number of integrators increases. The performance of the proposed controller with two integrators (C2) is remarkable, and the complexity of the control algorithm is relatively small. Moreover, the addition of integrators does not increase significantly the calculation time.

In comparison to previous strategies that deal with similar control problems (e.g., [17, 21, 22, 24, 25, 27]), the proposed controllers present the advantage of avoiding a stochastic modeling, as it is typically required to deal with systems where the parameters are contaminated by white noise. The trade-off between the response speed and the stability is solved through a simple Monte Carlo experiment. In addition, the controller design is mathematically simple. Currently, the developed methodology for the controller design is being applied to another bioprocess. Besides, the inclusion of saturation of the control signals and sliding mode control schemes in the formulation of the problem will be addressed in future contributions.

#### Appendix

#### A. Linear Algebra-Based Controller

Below is a brief description of the fed-batch bioreactor and the controller derived in [30]. The state variables in a fed-batch reactor can be described by the following set of equations proposed in [31]:withIn (A.1) and (A.2), the following variables and parameters are used: the amount of secreted protein per culture volume unit , the total protein amount per culture volume unit , the culture cell density , the culture glucose concentration , and the culture volume ; is the feed flow rate, is the glucose concentration of the feed stream, is the yield of glucose per cell mass, and , , and are the protein expression rate, the specific growth rate of the host cell, and the protein secretion rate, respectively. The optimal profiles for the state variables calculated in [32] were chosen as the reference profiles to be tracked by the control algorithm. The nominal values of the model parameters are gL^{−1} and . Initial conditions for the state variables are gL^{−1}, gL^{−1}, gL^{−1}, gL^{−1}, and L.

In order to follow the reference profiles, the required control action is calculated from the process model. First, (A.1) can be rewritten in a discrete version using the Euler approximation:Let us denote by the state vector and by and the input vectors. Then, the state vector in the sample time can be expressed asThe following replacements should be done in (A.3): where** K** is the controller parameter matrix and . The Monte Carlo Randomized Algorithm was applied to select the controller parameter matrix. This procedure does not ensure reaching the global optimum. Nevertheless, based on Theorem of [30], a large number of trials were made in order to guarantee a large confidence interval. Then, (A.3) can be rewritten asEquation (A.6) has an exact solution when the culture glucose concentration is adopted as a “sacrificed variable,” [30]. Then, the control action can be obtained through the least squares procedure:

#### B. Proof of Theorem 3

*Assumption B.1. *The optimal approximation error ** ε** is bounded by a constant: .

*Proof of Theorem 3. *Define a candidate function of Lyapunov, with being a positive definite function given byThe difference in discrete time is adopted as follows:And is defined asBy replacing (25) into (B.4), is written asBy replacing (25) into (B.4), is written asFrom (B.3) and rewriting , one obtainsThen, (B.6) is replaced into (B.5) and, after reorganizing some terms, one obtainsReplacing the adjustment laws (see (27)), , in (B.7), is represented byIn (B.8), the increment of the model error of is unknown, and it can be approximated by the following equation:The partial derivative of the model error depends only on the weights of the neural network and can be rewritten asChanging the values of the weights according to (B.6) and considering that is constant, ideal weight vectors are required only for analytical purpose. Now reorganizing , it can be rewritten asEquation (B.10) can be written as Considering the value of the partial derivatives of the neural network (see (27)) and by substitution in (B.10), it yieldsThe increase in the model error is defined aswhereConsider that is a bounded function. The value is a learning factor of the neural network and it can be arbitrarily defined.

Expressing (B.14) in vectorial form, Now, substituting the increment value of the model error in (B.16), the Lyapunov discrete difference is defined asNow replacing the learning rule in (B.17) and applying norm yieldwhere ; then (B.18) can be expressed asFrom (B.19), it follows that when , tends to zero when tends to infinity. If the error norm is , we can obtain that and the neural weights could tend to infinity. To prevent the above situation, the next update rule is used.where and are design parameters.

#### Nomenclature

*Symbol*

: | Vector of centers of each RBF function, — |

: | Standard Brownian motion, h^{−1} |

: | Tracking error, g^{2}L^{−2}h^{−1} |

: | Additive uncertainty, gL^{−1} |

: | System bandwidth, h^{−1} |

: | Performance index, gL^{−1}h^{−1} |

: | Controller parameters, — |

: | Neural design parameter, — |

: | Maximum number of neurons, — |

: | Number of points of the reference profile, — |

: | Secreted protein concentration, gL^{−1} |

: | Root of a linear system, — |

: | Glucose concentration, gL^{−1} |

: | Glucose concentration of the feed, gL^{−1} |

: | Total protein concentration, gL^{−1} |

: | Correlation time, h |

: | Sample time, h |

: | Maximum sample time, h |

: | Feed flow rate, Lh^{−1} |

: | Integral of the tracking error, g^{2}L^{−2} |

: | Bioreactor volume, L |

: | Lyapunov candidate function, — |

: | Biomass, gL^{−1} |

: | Estimated variables vector (, gL^{−1} |

: | Neural design parameter, — |

: | Yield of glucose per cell mass, — |

: | State vector, gL^{−1}. |

*Subscripts*

: | Sacrificed variable, — |

: | Actual sample time, — |

ref: | Reference profile, — |

: | Initial condition, —. |

*Greek Letters*

: | Intensity of the noisy signal, — |

: | Random number between 0 and 1, — |

: | Diagonal definite positive matrix, — |

: | RBFs vector, — |

: | Neural approximation error, — |

: | Vector of widths of each RBF function, — |

: | Neural weight matrix, — |

: | Optimal weight matrix, — |

: | Neural error weight matrix, — |

: | Auxiliary vector, — |

: | Specific growth rate, h^{−1} |

: | Maximum specific growth rate, h^{−1} |

: | Protein expression rate, — |

: | Protein secretion rate, — |

: | Input vector to the neural estimator , — |

: | Learning factor of the neural network, —. |

#### Conflicts of Interest

The authors declare that there are no conflicts of interest.

#### Acknowledgments

This work was partially funded by the following Argentine institutions: National Council of Scientific and Technological Research (CONICET), Universidad Nacional de San Juan (UNSJ), and Universidad Tecnológica Nacional.