Abstract

This paper presents global optimization algorithms that incorporate the idea of an interval branch and bound and the stochastic search algorithms. Two algorithms for unconstrained problems are proposed, the hybrid interval simulated annealing and the combined interval branch and bound and genetic algorithm. The numerical experiment shows better results compared to Hansen’s algorithm and simulated annealing in terms of the storage, speed, and number of function evaluations. The convergence proof is described. Moreover, the idea of both algorithms suggests a structure for an integrated interval branch and bound and genetic algorithm for constrained problems in which the algorithm is described and tested. The aim is to capture one of the solutions with higher accuracy and lower cost. The results show better quality of the solutions with less number of function evaluations compared with the traditional GA.

1. Introduction

Many problems in economics, business, sciences, and engineering are modeled as constrained optimization problems:

There are various approaches to the problems. Interval algorithms use branch and bound techniques to capture all solutions. One drawback is that they often require more memory and CPU time [1]. Stochastic search algorithms are usually easy to implement and no assumption about the continuity and differentiability is required. They are commonly used in numerous fields. Even though there is no guarantee on the solution reported when the algorithm is terminated in a finite time, the theoretical support is only on the convergence in probability. We have an interest in combining the interval branch and bound technique with stochastic search. We first study the combined algorithms for unconstrained problems and then modify them to handle the constrained problems.

The paper is organized as follows. In the next section, the related algorithms are introduced, namely, the interval branch and bound, simulated annealing, and the continuous genetic algorithms. The studies of the improvement of the described algorithms are also discussed. Our proposed algorithms are presented in Section 3. We demonstrate two algorithms for unconstrained and one for constrained problems. In Section 4, the numerical experiments and the discussion are given, followed by the conclusions in Section 5.

2. Interval Branch and Bound and Stochastic Algorithms

2.1. Interval Branch and Bound

2.1.1. Unconstrained Optimization

An interval algorithm is a tool using interval arithmetics for finding all solutions of the optimization problems. Before the discussion of the algorithms, let us first introduce the notations that we will use in this paper.

For an unconstrained problem, we study . : A search domain  : The range of over  : a set of real compact interval ,  : a set of -dimensional column vectors  an inclusion function of   will be called “box” : A number of subregions : A number of sample points from a given box listx:  A list containing the boxes and their information  : A number of boxes in listx or the length of listx : Temperature : An initial temperature for cooling schedule of simulated annealing : Maximum number of iterations : Midpoint of an interval ,  : The width of an interval ,  If , is the width of the box . : A tolerance for the width of the box : A tolerance for the width of the interval  : The best value of all the algorithm has been encountered : A vector corresponding to  : A value of at iteration  : A lower bound of interval  : An upper bound of interval .

An interval branch and bound have the procedures given in Algorithm 1.

A working box could be selected to be the one that has been on the list the longest or the one with the least lower bound of its inclusion function . The bisection direction usually is the direction with the maximum width. Box will be discarded if . The termination conditions can be set by using a prescribed maximum number of iterations, the width of the box, or the width of the interval . A detailed discussion can be found in [2]. When the algorithm ended, all optima are contained in the boxes of the list.

The algorithm shows difficulty when the dimension is high or the function is complicated. The width of the boxes in the list decreases very slowly so that the improvement of the best is found.

We describe here two versions of the interval algorithms. The first is proposed by Ichida and Fujii, in which the working box is the box with the least lower bound of . The second is Hansen’s algorithm, in which the working box is the oldest box in the list.

2.1.2. Constrained Optimization

For constrained problems, a box will be deleted from the list with the condition only if all points in are feasible. The feasibility of a box is considered by using a flag vector where . The element is assigned by the following rules.

For an inequality constraint, if then . If , . Otherwise, it is indeterminate, .

For an equality constraint, we will consider the relaxed problem and is set according to the width of and the bound of as follows: If and  if ()  else  else .

The status of a box is taken through the flag vector . If at least one element of is 2, the status is labeled as 2 (this box will be deleted from the list). If all elements of are 0, the box is feasible and the status is set to be 0. Other than that the status is 1. Usually, a box with status 1 will be bisected.

2.2. Simulated Annealing

Simulated annealing (SA) is a stochastic search technique, analogy with thermodynamics. There is a mechanism in avoiding entrapment in local optima by allowing an occasional uphill move. It also incorporates a temperature parameter into the procedure, explore more at high temperature, and restrict it when the temperature is low. The basic idea of the search is that, for a given current state with an energy level , generate a subsequent state randomly. If then accept state as the current state. Otherwise accept state with the probability where is the temperature. For the minimization problem, the solution corresponds to a state of the system and the objective function corresponds to the energy level. Prior to the process, the cooling schedule and the neighborhood structure must be defined. SA is described in Algorithm 4. A thoroughly discussion can be found in [4].

2.3. Population Based Methods

Population based methods use a population of points in each iteration. One advantage of populations is that if it has multiple optimums it will be captured in its final population. The examples of such techniques are genetic algorithm (GA), ant colony, particle swarm, and differential evolution. They are widely used in business, sciences, and engineering. The simple GA will be described next.

GA imitates the natural evolution involving three processes, creation of the offsprings, selection, and mutation. The creation of offsprings offers the diversification for the search, but the selection process is narrowing it. The mutation prevents the entrapping in the local minimum. GA is outlined is Algorithm 5.

The main idea of the selection process is to choose better individuals for the next generations by considering the fitness of each one. We are now concentrating on minimization problem; thus the better fit individual is the one with the lower value of . Good selection schemes must allow the convergence to the optimal solution without getting caught in a local minimum. There are many proposed selection techniques and the study of their convergence.

Some of the methods are, for example, elitist selection (the best individuals of each generation must be selected), the proportional selection (the better fit individuals have higher probability to be selected), the ranking selection (the individuals are ranked according to their fitness and the selection is based on this ranking), and the tournament selection (individuals are divided into subgroups and the members of each group compete against each other, and then only one is selected to be in the new generation). The mutation rate is set up such that converges to when is increased. The termination condition can be set to be the maximum number of generations or the unchanged of the best value over a given number of generations. The comparison of performance of the selection schemes including the convergence can be seen in, for example, [5–7].

The disadvantage of GA is that there is no guarantee for finding optimal solutions within a finite amount of time and the tuning of parameters such as population size or mutation rate are sometimes based on trial and error. However, there are no requirements on differentiability and continuity.

2.4. The Modifications

The previous described algorithms can be modified for an improvement. Some of them that related to our works are given next.

For the interval branch and bound algorithms, the adjustment can be made to the following aspects for better solutions:(i)an inclusion function: the kite inclusion function [8],(ii)the subdivision of domain: multisection [8–10],(iii)the selection of a box for further process: the box with the largest rejection index such as the one defined in Casado et al. [11], where represents the rejection index of a box .

The studies for the better accuracy of solutions and the speed of algorithm when using an interval branch and bound to solve constrained problems are, for example, as follows.(i)Casado et al. [11] define a rejection index of a box as in (2) to identify a good candidate box to contain a global minimum.(ii)Lagouanelle and Soubry [8] use a new inclusion function called kite enclosure.(iii)Sun and Johnson [12] introduce local sampling strategies to the working box. The convergence proof is presented along with the numerical results.(iv)Karmakar and Bhunia [13] demonstrate how to obtain one of the solutions by partitioning the accepted box (start with the search domain) into subboxes where is a number that each edge is divided. Calculate the function values of each subbox and then use interval order relations to choose a new accepted box.

There are many approaches in handling constraints for GA. The classifications can be seen in [3, 14]. We describe those that combine interval branch and bound and genetic algorithm.

Alander [15] suggests two ways in combining the two algorithms. The first is replacing, at least partly, the function to be optimized by some of its interval extensions. The other is using GA in some internal problems of the interval algorithm.

Sotiropoulos et al. [16] use an interval branch and bound technique to create subregions and choose the midpoint of each subregion to be individuals in an initial population for GA.

Zhang and Liu [17] use rejection index (2) as a fitness function for a box where is the best found. The population is taken from highest fitness boxes. Mutation is performed on 1/3 of the population with some probability. The best fit box is split into subboxes.

An attempt to combine the interval branch and bound and simulated annealing for unconstrained problem can be seen in Shary [18]. The lower bound of is used in calculating the probability of accepting a new box. If this box is accepted, it will be bisected and the half with smaller lower bound of is chosen to be the next leading box. Otherwise, choose a different box. It is tested on a six-hump camel function and Rastrigin’s function with the domain .

3. The Proposed Algorithms

We first study the performance of Ichida-Fujii (A2) and Hansen (A3) for a design of our algorithm. A set of unconstrained problems in Appendix A is used. Tables 1 and 2 show that Ichida-Fujii is more effective than Hansen in terms of the speed (number of iterations), storage (the list length), and cost (number of function evaluations). It appears that searching and bisecting the box with the least lower bound is a fastest way to reach the optimal solutions.

For unconstrained problems if we assume that is continuous, we know for certain that the box with the least lower bound of must contain a minimum. The earlier a quality can be discovered, the faster the unwanted boxes can be deleted.

To obtain a quality , we consider combining SA and GA with an interval branch and bound. SA and GA act as search engine while the interval branch and bound are responsible for keeping all solutions.

When dealing with constrained problem the deletion condition will be in effect only if the box is feasible or infeasible. For most problems, the list will contain a high percentage of the boxes with status 1 which is indeterminate. That means the width of the box must be small enough in order to split feasible from the infeasible region. The situation is even worse when the dimension of the problem is high.

There are two concerns in our algorithm as follows:(i)choosing a potential box to search: as a result, a quality can be obtained to promote the deletion of unwanted boxes,(ii)bisecting the box to isolate a feasible region.

For unconstrained problem, we can search in a box with the least lower bound of . However, for constrained problem we need a function that provides information about the value of and the lower bound of in making a decision about which box to search for a better . Let us first define a function fit to be the minimum value of among the feasible points from a box . If the box contains , fit is getting close to when the width of approaches 0. The information of and is combined by using a function called fun defined by is a nonincreasing sequence and . Note that if for all , the algorithm uses in making decision for a searched box. And when for all , only function fit which related to will be considered.

We will first present two algorithms for unconstrained problems to study their efficiency in terms of the speed, storage, and cost. Then the algorithm for constrained problems is described. Our goal is to get information of the search region from the interval branch and bound and then provide it for the continuous genetic algorithm to improve its efficiency.

3.1. Hybrid Interval Branch and Bound and Simulated Annealing for Unconstrained Problems

Our algorithm uses simulated annealing as a mechanism that encourages the search in a promising box, at the same time avoiding entrapment in a local minimum. It is described in Algorithm 6. We choose the box with maximum difference of the best value found so far and the least lower bound of , , to be bisected. Since , it might result in being able to discard half of the box.

Algorithm 6 is different from [18] in the selection of a working box. We also evaluate more than one point to update the value of the best found so far, .

The stopping criteria are or for all boxes in listx, or the maximum number of iterations has exceeded a prescribed value .

We use a linear cooling schedule by prescribing the number of iterations at each temperature. The temperature is changed by using the given cooling rate; that is, new temperature = cooling rate temperature.

For Algorithm 6, the parameters that can be adjusted are the number of the initial boxes , an initial temperature , and the annealing schedule.

Note that the value of fit can be obtained by performing some iterations of your choice of search algorithm in box . In each iteration, fit value of two boxes, and , is calculated. Thus, can also be updated.

The Convergence. The following behaviors can be observed from the mechanism of Algorithm 6.(i)At an initialization stage, the probability that the box in listx is selected to be a working box is . In each iteration every box has a probability of to be a box . Both and will be searched by randomly choosing points and recording the best found in (Algorithm 7). Thus, is a decreasing sequence. This process can be viewed as a random search in the domain that shrink over time.(ii)For each box in listx, the following inequality holds: (iii)At iteration , or will be bisected depending on whom has a higher value of .(iv)It is possible that there is a box in which the width is not small and survives through iterations. This means for some where . Since decreases with , this will be selected to be sometimes later. Therefore, a sequence of is not decreasing.(v)Every box has a nonzero probability to be bisected to make smaller, although those boxes in listx have different size. We can conclude that as for .

We can show that all solutions will be in listx after the algorithm successfully terminates.

Let us assume the properties of an inclusion function as . Denote to be a union of all boxes in listx at iteration ; that is, :

From the discarding rule the box will be discarded if . Since and , the solution is still in the list. Therefore, all boxes in the list contain solutions; that is, .

Suppose for all . There exist a box where and . The properties as and imply that as .

Now suppose that is a box containing the minimum of ; then fit. With the properties and , we can conclude that .

3.2. Integrated Interval Branch and Bound and GA for Unconstrained Problems

Algorithm 8 will combine a population technique with the interval branch and bound to give a higher probability in obtaining a better value of the best found, . Thus, more boxes will be deleted from the list. The population is selected from boxes with the least lower bound of to create a new set of candidates using the linear crossover from Michalewicz’s book [3] (Algorithm 10). The information of two points is combined and the two outputs are controlled to be in the search domain. They are not necessary in a set of working boxes. Only the best points are carried to the next generation.

The parameters that can be adjusted for the performance of Algorithm 8 are the following: the number of boxes , the number of individuals per generation , the number of working boxes , the number of individuals to include in the new generation , and the maximum number of iterations. There are also two procedures that can be changed, the process of creating a new set of points and the rule for selecting a new generation (Algorithm 9).

3.3. Integrated Interval Algorithm and GA for Constrained Problems

The major disadvantage of the interval methods is that they require more memory and CPU time than the noninterval algorithms. We propose an algorithm that integrates a known bound from an interval algorithm and the quickness of a genetic algorithm. Of course, a certainty of the solutions is lost. However, an improvement of the quality of the solution and a reduction of the cost are gained.

Let us first introduce additional functions which will be used in the algorithm. is 0 if at least one feasible point has been found from box . Otherwise, it is 1. Elements of the flag vector corresponding to box are 0 or 1. Therefore, we define nviol_box to be the sum of the elements of a flag vector . It roughly indicates the amount of constraint violation for a box . The status of a box in the list is either 0 or 1, since a box with status 2 is discarded right after becoming known. For constrained problem is modified. The term is added, taking care of feasibility. Consider In the case that a feasible point is not discovered in Step 3 of Algorithm 11, the upper bound of will be assigned to .

We use a simple GA without mutation in Algorithm 12. The mutation is omitted because GA will be invoked in every iteration. The parameters that related to GA are a number of individuals in each generation and the maximum number of generations. The difference of Algorithm 11 and GA is pointed out next.

In GA, an initial population is randomly chosen from a search domain. Then this population is evolved through the three operators crossover, selection, and mutation. The change of individuals in a population is through the creation of children and mutation process.

In Algorithm 11, GA is performed in every iteration with the assigned value of maximum generations, . An initial population consists of the best individual, , and the individuals randomly chosen from a given box considered as a potential region for a better solution. In a big picture, it is similar to performing mutation with the rate of one at every iteration. The mutation is biased because it is restricted to those in the promising region, which is listx[idsearch]. After this, the offsprings and populations are allowed to be in . For Algorithm 11, the convergence is achieved when no box can be discarded. Thus those boxes left in the list have or .

4. Numerical Results

4.1. Unconstrained Problems

Tables 3 and 6 show the value of found by Algorithms 2, 3, 4, 6, and 8 for and , respectively. The maximum number of iterations is set to be 400,000 and 600,000. In the tables, A1 stands for Algorithm 1 and similarly for other algorithms. In Algorithms 6 and 8, the maximum length of listx, the number of iterations, and the number of function evaluations are the maximum number taken over ten runs. The tolerance is set. The algorithm is successfully terminated with for all ten runs. Since is in the range of , the table presents only .