Abstract
Multiobjective optimization is nowadays a word of order in engineering projects. Although the idea involved is simple, the implementation of any procedure to solve a general problem is not an easy task. Evolutionary algorithms are widespread as a satisfactory technique to find a candidate set for the solution. Usually they supply a discrete picture of the Pareto front even if this front is continuous. In this paper we propose three methods for solving unconstrained multiobjective optimization problems involving quadratic functions. In the first, for biobjective optimization defined in the bidimensional space, a continuous Pareto set is found analytically. In the second, applicable to multiobjective optimization, a condition test is proposed to check if a point in the decision space is Pareto optimum or not and, in the third, with functions defined in n-dimensional space, a direct noniterative algorithm is proposed to find the Pareto set. Simple problems highlight the suitability of the proposed methods.
1. Introduction
Life is about making decisions and the choice of the optimal solutions is not an exclusive subject for scientists, engineers, and economists. Decision making is present in day-to-day life. Looking for an enjoyable vacancy, everyone will formulate an optimization problem to a travel agent, a problem like with a minimum amount of money visit a maximum number of places in a minimum amount of time and with the maximum level of comfort. Usually all real design problems have more than one objective; namely, they are multiobjective. Moreover, the design objectives are often antagonistic.
Edgeworth [1] was the pioneer to define an optimum for multicriteria economic decision making problem, at King’s College, London. It was about the multiutility problem within the context of two consumers, and . “It is required to find a point such that in whatever direction we take an infinitely small step, and do not increase together but that, while one increases, the other decreases.”
Few years later, in 1896, Pareto [2], at the University of Lausanne, Switzerland, formulated his two main theories, Circulation of the Elites and the Pareto optimum. “The optimum allocation of the resources of a society is not attained so long as it is possible to make at least one individual better off in his own estimation while keeping others as well off as before in their own estimation.”
Since then, many researchers have been dedicated to developing methods to solve this kind of problem. Interestingly, solutions for problems with multiple objectives, also called multicriteria optimization or vector optimization, are treated as Pareto optimal solutions or Pareto front, although, as Stadler [3] observed, they should be treated as Edgeworth-Pareto solutions.
Extensive reviews of multiobjective optimization concepts and methods are given by Miettinen [4], for evolutionary algorithms by Goldberg [5] and for evolutionary multiobjective optimization by Deb [6]. The theoretical basis for multiobjective optimization adopted in this work was based on these references.
Thanks to the computer development, optimization of large scale problems became a common task in engineering designs. The development of high speed computers and their increasing use in several industrial branches led to significant changes in the design processes. Currently, the computers, each time faster, allow the engineer to consider a wider range of design possibilities and optimization processes allow systematic choice between alternatives, since they are based on some rational criteria. If used adequately, these procedures can, in most cases, improve or even generate the final results of a design.
Associated with computer development, many of the research done in optimization is focused on numerical methods to solve any kind of problem, but sometimes simplified problems can give important clues to the designer during the trade-off phases of a decision.
The present work aims to bring new approaches to solve multiobjective optimization problems, providing a rapid solution for the Pareto set if the objective functions involved are quadratic.
The rest of the paper is organized into 3 sections. In the first section a general multiobjective optimization problem is formulated and the nature of optimal solutions from the Pareto perspective and the necessary conditions to be met are defined. In the second section, three propositions are done to solve the unconstrained multiobjective optimization problems involving quadratic functions. In the first section the general problem comprises two bidimensional functions. In this case, the proposition permits to find the Pareto front analyticaly. In the second section, the problem considers the minimization problem with three or more functions, keeping the decision space in two dimensions. In this case the proposition helps to find the Pareto points and their boundary in the decision space. In the third section, proposition of the decision space is expanded to any dimensional size. Finally, a section with the conclusions and the proposed future work is presented.
2. Multiobjective Optimization Problem
Multiobjective optimization problems (MOOP) can be defined by the following equations:where is a vector with the values of scalar objective functions to be minimized. is the vector containing the design variables, also called decision variables, defined in design space . and are, respectively, the lower and upper bounds of the design variables. represents the th inequality constraint function and the th equality constraint function. Equations (1b) to (1d) define the region of feasible solutions, , in design space . The constraints are of type “” functions in view of the fact that “” functions may be converted to the first type if they are multiplied by −1. Similarly, the problem considers the “minimization” of , given the fact that function “maximization” can be transformed into the former by multiplying it by −1.
2.1. Pareto Optimal Solution
The notion of “optimum” in solving problems of multiobjective optimization is known as “Pareto optimal.” A solution is said to be Pareto optimal if there is no way to improve one objective without worsening at least another; that is, the feasible point is Pareto optimal if there is no other feasible point such that for all and . Due to the conflicting nature of the objective functions, the Pareto optimal solutions are usually scattered in the region , a consequence of not being able to minimize all the objective functions simultaneously. In solving the optimization problem we obtain the Pareto set or the Pareto optimal solutions defined in the design space and the Pareto front, an image of the objective functions, in the criterion space, calculated over the set of optimal solutions.
2.2. Necessary Condition for Pareto Optimality
In fact, optimizing multiobjective problems expressed by (1a)–(1d) is of general character. The equations represent the problem of single-objective optimization when . According to Miettinen [4], such as in single-objective optimization problems, the solution for the Pareto optimality must satisfy the Karush-Kuhn-Tucker (KKT) condition, expressed as follows:where is the weighting factor for the gradient of the th objective function, calculated at the point , . represents the weighting factor for the gradient of the th inequality constraint function, , and is zero when the constraint function associated is not active; that is, . represents the weighting factor for the gradient of the th equality constraint function, .
Equations (2a) to (2e) form the necessary conditions for to be a Pareto optimal as described by Miettinen [4]. They are sufficient for complete mapping of the Pareto front if the problem is convex and the objective functions are continuously differentiable in the space. Otherwise, the solution will depend on additional conditions, as shown by Marler and Arora [7].
The methods we will propose in the next sections can be classified in posteriori preference articulation and an extensive literature review of the most important methods to solve multiobjective optimization problems can be found in Augusto et al. [8].
3. Two-Dimensional Functions of Class C1
In this section we propose a simple strategy to determine the Pareto set in the decision space and the corresponding Pareto front in the function space, for MOOP involving two bidimensional differentiable functions. Consider an unconstrained multiobjective optimization problem. From (2a), the optimality condition can be interpreted by the following proposition.
Proposition 1. If there exists a Pareto front for the minimization problem with two continuous and differentiable functions defined in , say and , then the points in the decision space, where the gradients of both functions are parallel and opposite, define a continuous Pareto set that connects both functions minima.
As the gradients of each function are orthogonal to contours and point outwards from the minimum, the curve mentioned in Proposition 1 is the locus where the gradients of both functions are parallel and opposite, as shown in Figure 1.
3.1. Two Quadratic Functions Defined in Space
Proposition 1 is quite general, but as our focus is on quadratic functions let us solve an unconstrained biobjective optimization problem involving quadratic functions defined in the two-dimensional decision space; that is, . The problem is defined as follows: minimize:
Applying the optimality condition, , to (3a) and (3b) results in the following:
As the system of (4) is homogeneous, the nontrivial solution, with , requires a singularity; that is, the determinant of the coefficient matrix must be null. Consider which results in the following quadratic curve for : where
Function gradients and are parallel on the curve defined by (6), but they have to be opposite, resulting in positive weights in (4). Being the system singular, to find a relation between the weights and we can use only one of the equations as the other is its linear combination. Using the first equation, this relation can be deduced as follows: which have positive values if and only if
Therefore, (6) provides the locus where the functions gradients are parallel and (9) defines the Pareto set for the two quadratic functions minimization problem. The upper bound of (9) is reached if the first term or the second . As both terms are the first components of and , respectively, these conditions imply that the solution is over minimum or over minimum. In conclusion, the Pareto set for quadratic functions will be a quadratic curve connecting the functions minima and where the gradients are parallel and opposite.
As an example, let us consider the following biobjective problem: minimize: From (6), the Pareto set takes the form and is constrained by the following inequality:
In Figure 2(a) the contours of functions and in the two-dimensional decision space are depicted. The thicker grey continuous curve represents (12) and the thick blue coloured portion of this curve satisfies (13), being as expected the continuous Pareto set, namely, the curve along which the gradient vectors are parallel and opposite. In Figure 2(b), the continuous curve is the image of the Pareto set in the function space, that is, the Pareto front. In addition, the blue dots points are the images of the optimization functions calculated on a regular grid in the design space.
(a) Continuous Pareto set obtained by the proposed method
(b) Continuous Pareto front, the Pareto set image in the function space
| (c) Pareto set for the performance functions and obtained by the NSGA II algorithm |
| (d) Pareto front for performance functions and obtained by the NSGA II algorithm |
For comparison, it is shown in Figures 2(c) and 2(d), adapted from Augusto et al. [8], that the solution was obtained by the genetic algorithm NSGA II of Deb et al. [9]. It can be seen that the points are evenly distributed in the function space, but they are not in the decision space. That happens because the search procedure in most of GAs is focused in the function space, trying to get a well-distributed Pareto front.
3.2. Three or More Functions Defined in Space
In the previous section we found a closed form solution for the optimization of two quadratic functions in the bidimensional decision space. Unfortunately, we did not find a similar solution when we add more functions in the problem. Nevertheless, the idea behind Proposition 1 remains useful.
Consider an optimization problem involving three continuous differentiable functions , , and . If a point belongs to the Pareto set, it must satisfy (2a), (2b), (2c), (2d), and (2e). Therefore, one gradient vector, , will be a linear combination of the other two, and ; that is, there will exist positive weights such that
In Figure 3 such condition with the gradient vectors , , and associated with their weighted factors , , and , respectively, is illustrated.
An equilibrium condition exists when is oriented through the opposite angular sector defined by the two other gradient vectors, namely, and .
Based on this idea we suggest the following.
Proposition 2. Let be the unit vector defined by , with , and , the unit vector orthogonal to ; that is, . If belongs to the Pareto set resulting from the multiobjective optimization problem involving continuous and differentiable functions defined in , then there exist at least three unit vectors, say, , , and , that satisfy the following conditions:
The direction of divides the decision space in two semiplanes. If the vector points to one side, then (15a) and (15b) state that the vectors and point to the other side and (15c) states that is placed between them.
Equations (15a), (15b), and (15c) form a condition test for a point be Pareto optimum or not. This test can be useful if the problem has few optimization functions as to explore all distinguished sets with three gradient vectors in a problem with objective functions; the maximum of permutations of must be checked. Let us apply Proposition 2 to find the solution of an unconstrained MOOP with three quadratic objective functions with two of them being those defined by (11a) and (11b) and the third defined by
In Figure 4, the Pareto set found applying the Pareto test in the points of a regular grid in the design space divided in fifty points in each coordinate axis, for all , , is shown. The continuous border of the Pareto set was obtained applying Proposition 1 for each pair of objective functions.
(a) Pareto set
(b) Pareto front
| (c) Pareto front - view |
| (d) Pareto front - view |
| (e) Pareto front - view |
3.3. Quadratic Functions Defined in Space
In the former two sections we have considered unconstrained MOOP with quadratic functions defined in the two-dimensional space. To proceed to larger dimensions, let us define a quadratic function in the space, , written as follows: with and is a local coordinate system for a convenient definition of , is the position of the local coordinate system related to the global one, and is the coordinates transformation matrix, from local to global coordinates systems.
Using (18), (17) can be rewritten as follows: Calling , (19) can be rewritten as follows: As is smooth, its gradient vector is Matrix , as well as its transformed form , is the symmetric Hessian of , , containing its second partial derivatives.
With these definitions, let be the solution of an unconstrained MOOP involving quadratic functions defined in space. Accordingly, there exists , , that satisfy (2a); that is,
As is a quadratic, (21) can be used and (22) takes the form
In (23), the weights , as well as the searched solution , are unknown. Let us assume that all are known; that is, . Accordingly, (23) can be rewritten as follows: Calling then (24) can be rewritten as follows:
Let us assume that all are positive definite; that is, , for all . If is real, nonnegative and satisfies the normalization equality, that is, , then will also be positive definite and therefore its inverse will always exist.
Consequently, the Pareto optimum solution can easily be found by solving (27); that is,
In this approach, we have considered that are known. Consequently, , (25), and , (26), are promptly found. Although this is not the case for a general solution of (22), this approach is very useful to find the Pareto set and the Pareto front of unconstrained multiobjective optimization problems involving quadratic functions considering the following.
Proposition 3. Consider a MOOP involving quadratic functions, with the Hessian of each function being positive definite. To obtain Pareto optimum solutions the following steps are proposed.(1)Sort, at random over the interval , the components of , a vector containing weights .(2)Perform a normalization such as .(3)Calculate and .(4)Solve the linear system , getting the Pareto point associated with .(5)Repeat steps (1) to (4) for the number of Pareto points wanted.
Even requiring solutions of linear systems, the method is very fast depending on the order of the matrix .
Before advancing to the applications, consider an ellipsoid enclosed in a parallelepiped of sizes , , and , as shown in Figure 5. Also consider local coordinates system, with origin centered inside the ellipsoid, fixed to it, and oriented along its semiaxes.
The family of quadratic functions that represents this ellipsoid can be written as follows: with the matrix defined in Figure 5.
The ellipsoid can be rotated around the th coordinate axis; that is, . Let , , and be the rotation angles, around , , and axes, respectively. Each individual rotation matrix is depicted in Figures 9(a), 9(b), and 9(c) and the appendix. Then, the general rotation matrix is defined by
The local coordinate system can be positioned at a point , relative to a global coordinates system, . In such a case, the points on the surface of the ellipsoid can be referenced in the global system as
To get the transformation matrix of (18), we isolate in (31); that is, Being an orthogonal matrix, its inverse is equal to its transpose; that is, .
With the previous definitions, consider the following unconstrained MOOP: with , with defined in Table 1 and illustrated in Figure 6(a).
| (a) Three quadratic functions, , |
(b) Pareto set
| (c) Pareto set, - view |
(d) Pareto front
The Pareto set for this problem, illustrated in Figure 6(b), was obtained by applying Proposition 3 algorithm, with = 5000. To get all the points, an ordinary 2 GHz dual processor computer with 3 Gb RAM, running Matlab, expended 0.99 seconds of processing time.
As all ellipsoids were placed over plane and were rotated around axis, only, the Pareto set is over the plane. Bold points at the Pareto set boundary were found with the same method applied to the functions taken in pairs. According to Proposition 1, in such cases, the Pareto set is necessarily a curve.
The Pareto front is shown in Figure 6(d). It should be noticed that this front was obtained by means of a straightforward solution of the Pareto optimality conditions without using any iterative algorithm.
In the next example three ellipsoids with different orientations, as defined in Table 2 and the appendix, were distributed in the space.
The Pareto set of this optimization problem found by the proposed methodology delineates the curved surface shown in Figure 7(a). The Pareto front, in the function space, is shown in Figure 7(b).
(a) Pareto set
(b) Pareto front
Adding to the unconstrained MOOP the function , defined in Table 3 and the appendix, the proposed method generated in 1.17 seconds the three-dimensional Pareto set illustrated in Figure 8.
(a)
(b)
(c)
In the problems all functions were defined by convenience in space; nevertheless, Proposition 3 can be applied to quadratic functions defined in space.
4. Conclusions
Most of the real problems are multiobjective with their objective functions being antagonistic. To solve this problem many researchers are developing methods to solve multiobjective optimization problems without reducing them to single objective. Up to now, evolutionary algorithms are widespread as a general technique to find a candidate set of the optimal solutions. These algorithms provide a discrete picture of the Pareto front in the function space, without bringing too much information about the decision space.
In the framework of this paper, we have proposed different methods to determine the Pareto set of unconstrained multiobjective optimization problems involving quadratic objective functions. Three different procedures were proposed. One for biobjective optimization, with functions defined in space, which results in an analytical solution for the Pareto set. For three or more functions also defined in space a condition test that is able to check if a point in the decision space is Pareto optimum or not was proposed. In the third method, suitable for multiobjective optimization with functions defined in space and having Hessian positive definite, a direct algorithm was proposed which finds a Pareto optimum based in an arbitrary valid weighting vector. Some illustrative examples were used to highlight the potentiality of the methods.
It is apparent that the Pareto set for two distinct two-dimensional functions is a curve, and for three and above, the Pareto set is a surface. In three-dimensional space, for two distinct three-dimensional functions, the Pareto set will be a space curve; for three functions, a surface; and for four functions and above, a solid. Although the proposed methods are restricted to unconstrained optimization, the authors believe they can be extended to constrained problems and are working on it.
Appendix
See Figures 9(a), 9(b), and 9(c) and Tables 2 and 3.
Nomenclature
| DM: | Decision maker |
| Objective functions vector | |
| GA: | Genetic algorithm |
| : | th inequality constraint function |
| : | th equality constraint function |
| : | Number of objective functions |
| KKT: | Karush-Kuhn-Tucker |
| : | Number of equality constraint functions |
| : | Number of inequality constraint functions |
| MOOP: | Multiobjective optimization problem |
| NSGA II: | Nondominated sorting genetic algorithm, version two |
| : | Dimension of the design space |
| : | Function or criterion space |
| : | Decision variables or design space |
| : | Feasible region in the design space |
| : | th decision variable |
| : | Decision variable vector |
| : | Nondominated solution of a multiobjective optimization problem |
| : | Lower and upper bounds of the design space |
| : | Weighting factor for the th objective function gradient in KKT condition |
| : | Vector of |
| : | Weighting factor for th inequality constraint gradient in KKT condition |
| : | Vector of |
| : | Weighting factor for th equality constraint gradient in KKT condition |
| : | Vector of |
| : | Gradient operator. |
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.