This paper considers two kinds of stochastic reentrant job shop scheduling problems (SRJSSP), i.e., the SRJSSP with the maximum tardiness criterion and the SRJSSP with the makespan criterion. Owing to the NP-complete complexity of the considered RJSSPs, an effective differential evolutionary algorithm (DEA) combined with two uncertainty handling techniques, namely, DEA_UHT, is proposed to address these problems. Firstly, to reasonably control the computation cost, the optimal computing budget allocation technique (OCBAT) is applied for allocating limited computation budgets to assure reliable evaluation and identification for excellent solutions or individuals, and the hypothesis test technique (HTT) is added to execute a statistical comparison to reduce some unnecessary repeated evaluation. Secondly, a reentrant-largest-order-value rule is designed to convert the DEA’s individual (i.e., a continuous vector) to the SRJSSP’s solution (i.e., an operation permutation). Thirdly, a conventional active decoding scheme for the job shop scheduling problem is extended to decode the solution for obtaining the criterion value. Fourthly, an Insert-based exploitation strategy and an Interchange-based exploration strategy are devised to enhance DEA’s exploitation ability and exploration ability, respectively. Finally, the test results and comparisons manifest the effectiveness and robustness of the proposed DEA_UHT.

1. Introduction

The job shop scheduling problem (JSSP) is one of the most well-known combinatorial optimization problems. As a variant of the JSSP, the reentrant JSSP (RJSSP) can be found in many real-life production and manufacturing environments, such as in semiconductor manufacturing enterprises [1, 2] and textile factories [3]. The main characteristic of the RJSSP is that one job may visit a set of machines more than once during the production process. Since the JSSP has NP-complete property [4] and it can be reduced to the RJSSP, the latter is also NP-complete. The studies on the RJSSP are still limited. Pan and Chen [5] presented two extended binary integer programming formulations for the RJSSP. Topaloglu and Kilincli [6] proposed a modified shifting bottleneck heuristic for the RJSSP with makespan criterion. Chen et al. [7] presented a hybrid estimation of distribution algorithm for the RJSSP with sequence-dependent setup times to minimize makespan.

As we all know, uncertainties are very common in many real-world manufacturing scenarios. At present, some uncertain JSSPs have been studied. Most of them are the JSSP with stochastic processing times [822], and the rest are the JSSP with probabilistic machine availability [23], the JSSP with fuzzy processing times [24], and the JSSP with fuzzy machine availability [25]. In these studies, evolutionary algorithms are the main methodology, and makespan or tardiness/earliness is the main optimization criterion. In terms of the JSSP with stochastic processing times, how to evaluate the solution’s expected objective value (i.e., criterion value) reasonably is one of the key links in the design of the algorithm. The direct method is to sample the corresponding stochastic model multiple times to obtain multiple objective values, and then use the average of these sampling values as an approximate estimation of the expected objective value. Obviously, the more the sampling times, the more stable and reliable the estimated value is. This method is the most used method in existing research. However, such a method usually has a high computation cost. Thus, several researchers have adopted three types of techniques to control and reduce the computation cost. The first is the optimal computing budget allocation technique (OCBAT) [26]. Zhang et al. [13] and Yang et al. [16] utilized it to allocate limited computation budgets to assure more reliable evaluation for better solutions. The second is the problem conversion technique. Shen and Zhu [19] used it to convert the stochastic models of the JSSPs to the deterministic ones, in which the confidence or robustness level is considered. Then, the objective value of each solution was calculated via these deterministic models. The third is the objective value prediction technique, which is mainly driven by the simulation metamodel. Ghasemi et al. [22] constructed a metamodel by using genetic programming (GP) to directly predict the solution’s objective value. However, as for the uncertain RJSSPs, only Mohammadi [27] designed a simulated annealing algorithm and a genetic algorithm for the RJSSP with fuzzy processing times to minimize makespan, and the stochastic RJSSP has not been studied until now.

From the above literature review, it can be known that it is very necessary to carry out a study on the stochastic RJSSP. Hence, the RJSSP with stochastic processing time is considered here. Recently, many effective evolutionary algorithms have been successfully designed to deal with different complex optimization problems [2837]. For example, Duan et al. [28] developed a hybrid estimation of distribution algorithm for the distributed permutation flow shop scheduling problem (FSSP) with flowline eligibility. Guo et al. [31] proposed a particle swarm optimization algorithm for the dynamic multi-objective vehicle routing problem. Sang et al. [33] presented three invasive weed optimization algorithms for the distributed assembly permutation FSSP. Guo et al. [36] devised a novel interactive preference-based multiobjective evolutionary algorithm for the optimization problem of the bolt supporting network. Zhang et al. [37] designed a matrix-cube-based estimation of distribution algorithm for the distributed assembly permutation FSSP. Computational comparisons demonstrated that these evolutionary algorithms can obtain very satisfactory solutions with a reasonable computational time. Due to the NP-completeness of the stochastic RJSSP, an effective population-based evolutionary algorithm, i.e., differential evolution algorithm (DEA), is also selected to deal with this complex problem. DEA was firstly presented for addressing optimization problems in continuous domain [38, 39]. Because of its simple concept and quick convergence, DEA has gained many applications in a variety of fields. Nevertheless, owing to its continuous nature, the studies on DEA for the JSSPs is still very limited. Thus, in this paper, an enhanced DEA integrated with uncertainty handling techniques (DEA_UHT) is designed for solving the stochastic RJSSP under the maximum tardiness and the stochastic RJSSP under the makespan .

The main features of DEA_UHT lie in two aspects: the reasonable uncertainty handling techniques and the effective DEA-based search. As for the uncertainty handling techniques, not only is the aforementioned OCBAT used to identify better solutions (i.e., individuals) with high confidence under limited computation cost but also the hypothesis test technique (HTT) is added to avoid the unnecessary repetitive evaluation for the poor solutions. That is, the solutions with high quality can be selected during the DEA’s search process, which is helpful to guide the algorithm search faster to the truly promising regions in the stochastic solution space. As for the DEA-based search, an Insert-based exploitation strategy and an Interchange-based exploration strategy are devised to enhance DEA’s exploitation ability and exploration ability, respectively. The test results and comparisons verify the effectiveness and robustness of the proposed DEA_UHT.

The remainder of the paper is organized as follows. Two kinds of stochastic RJSSPs are introduced in the next section. The basic DEA, OCBAT, and HTT are briefly described in Section 3. After presenting the solution representation, the extended decoding scheme, the revised OCBAT, and two special crossover strategies, DEA_UHT is proposed via combining DEA with OCBAT and HTT in Section 4. The test results and comparisons are given and analyzed in Section 5. Finally, concluding remarks and further study directions are summarized in Section 6.

2. Stochastic RJSSP

Consider the stochastic RJSSP that consists of n jobs and m machines. Each job should be processed on each machine L (L > 1) times. The processing time of each job on each machine is a random value under a certain distribution. Let denote the operation-based solution or sequence to be processed, and the job in , the repeat or reentrant times of job in , the noise or uncertain factors, the stochastic processing time of job on machine , the expected completion time of job , the due date of job , the excepted lateness of job , the excepted tardiness of job . Here is supposed to be subjected to a uniform distribution , ], where is the expected processing time and denotes noise magnitude.

Then, the expected maximum tardiness can be given as

The stochastic RJSSP under the maximum tardiness (SRJSSP-Tmax) can be formulated aswhere denotes the set of all possible permutations.

Similarly, the expected makespan can be given as

The stochastic RJSSP under the makespan (SRJSSP-Cmax) can be formulated aswhere denotes the set of all possible permutations.

Obviously, infinite sampling of the above problem model can make the objective values of equations (1) and (3) stable, but in fact it is practically impossible. In the actual calculation, the value of equations (1) or (3) is usually approximated by the average of finite sampling values (e.g., Monte Carlo simulation) [818]. Thus, can be estimated with

and can be estimated with

3. Introduction to DEA, OCBAT, and HTT

3.1. DEA

Differential evolution algorithm (DEA) [38, 39] is a kind of evolutionary algorithms for solving continuous optimization problems. The basic DEA aims at finding the global optimal solutions by using the differentiations among current individuals. Specifically, each individual’s search behavior is dynamically controlled via changing the differentiation’s direction and step length. At each generation, the mutation and crossover operators are applied to current individuals to generate a candidate population, and then the selection operator is utilized to determine a new population of the next generation. Nowadays, there are several variants/schemes of DEA.

3.2. OCBAT

OCBAT is a dynamic allocation technique of computing budget [26, 40]. Specifically, it can optimize the sampling times for each candidate solution so that the truly better solutions can be identified under a given computation cost. Denote the total number of potential solutions, (resp. ) the computation cost or sampling times allocated to solution (resp. the current best solution) after the kth allocation, (resp. ) the standard deviation among the sampling objective values of solution (resp. the current best solution) after the kth allocation, (resp. ) the average sampling objective value of solution (resp. the current best solution) after the kth allocation, the difference between and after the kth allocation, the total sampling times after the kth allocation, and the given total sampling times.

According to the above definitions, the standard OCBAT’s algorithm is provided as follows:Step 0. Set , and .Step 1. If , the algorithm should be stopped.Step 2. Execute samples (i.e., simulation replications) for each solution , and then set .Step 3. Calculate each via using and .Step 4. Execute samples for each solution .Step 5. Set , and return to Step 1.

Based on Reference [26], the recommended value ranges of and are [5, 20] and [, ], respectively.

3.3. HTT

HTT is a statistical inference method that can be used to judge whether the difference between two solutions and is caused by sampling error or essential difference [41]. Let the null hypothesis be “,” and then the alternative hypothesis is “.” Based on the statistical theory, the critical region of is defined as follows:where is the distribution and is the evidence level. When , it can be judged that solution is better than solutions .

4. DEA_UHT for Stochastic RJSSP

4.1. Encoding Scheme

Since DEA’s individual is a real vector, a conventional DEA cannot be directly used to deal with the discrete optimization problem. Thus, we utilize a reentrant-largest-order-value (RLOV) rule derived from the existing largest-order-value (LOV) rule [42] to transform DEA’s ith individual to the job sequence Based on the RLOV rule, the elements in are firstly ranked by descending order to obtain a temporary discrete sequence . Then, the operation-based solution can be generated viawhere is an operator to get the integral part of (e.g., ). Table 1 illustrates the encoding of vector in DEA for a small RJSSP .

4.2. Decoding Scheme

Based on the characteristics of RJSSP, an optimal solution must be an active schedule under regular criteria (e.g., the maximum tardiness and the makespan ) [43]. This means that the solution’s quality can be improved via transforming the semi-active schedule into the active one. Therefore, when decoding a solution to a feasible schedule, each operation should be filled in the first idle time interval in the current Gantt chart instead of the last idle time interval. This active decoding scheme is utilized in our proposed algorithm.

Let be the machine number of the jth operation that processes the lth reentrant job I, the processing time of the lth reentrant job j on machine i, and the possible earliest starting time of job . Notice that the value of should meet the following inequality:where , and [t1, t2] represents any idle time interval on the machine.

Based on the above definitions, the decoding scheme that generates an active schedule for a RJSSP’s solution is provided as follows:Step 1. Set and , .Step 2. If and ( mod ), then set .Step 3. Set .Step 4. Calculate , and set .Step 5. Set . If , then return to Step 2.

4.3. Application of Roulette Wheel Rule in OCBAT

For stochastic RJSSPs, the difference between and () is usually much larger than that between and . This can cause . Based on Step 4 of the standard OCBAT (see Section 3.2), the repeated times of Steps 1–5 of this standard algorithm may be much less than . That is, the number of times that OCBAT can be used to dynamically allocate computing budget is very limited. As a result, the reliable evaluation ability of OCBAT cannot be well utilized.

Under this situation, the so-called “most starving” rule [40] is usually used to handle this obstacle. However, this rule sets to 1 and needs to execute the OCBAT’s Steps 1–5 times. This may increase the running time of the OCBAT’s algorithm. Since the OCBAT decides the budget fractions instead of the number, here we utilize the “roulette wheel” rule to allocate . That is, Step 4 of the standard OCBAT should be changed to the following two substeps:Step 4.1. Calculate the allocation probability for each solution at (k+1) th allocation.Step 4.2. Apply the “roulette wheel” rule to allocate to each solution according to , and update each .

4.4. Special Exploitation and Exploration Strategies

Based on References [44, 45], the solution space of the RJSSP can be regarded as a “big valley,” where the local optima are often close to each other, and close to the global optima at the bottom of the “big valley,” and either the Insert-based operator or the Interchange-based operator can execute a more compact search. Meanwhile, DEA has been proved to be an effective algorithm to find the promising solutions or regions in the whole valley [42, 44]. As for the stochastic RJSSP, the shape and the bottom regions of the big valley are always changing. Thus, to improve our DEA_UHT’s search ability, we devise an Insert-based exploitation strategy and an Interchange-based exploration strategy to replace DEA’s original mutation and crossover, which cannot obtain enough promising solutions. The former strategy selects some reliable good individuals identified by OCBAT and HTT to replace some poor individuals, and executes an Insert-based operator on these selected individuals. The latter strategy performs an Interchange-based operator guided by DEA’s basic scheme on all individuals except those replaced poor individuals. The detailed procedures of these strategies are provided in Step 5 in the next section.

4.5. DEA_UHT

In DEA_UHT, DEA-based search is devised according to the DE/rand-to-best/1/exp strategy to perform global search or exploration, where the base vector is the current best individual. Therefore, the information of the best individual can be shared by other individuals. By incorporating OCBAT, HTT, and two special exploitation and exploration strategies into DEA, a hybrid approach, namely, DEA_UHT, is proposed for the stochastic RJSSP.

Denote a generation, a population with size at generation t, an archive set to save the individuals which need to adopt the special Insert-based crossover, the ith individual with dimension N () in , a temporary population with size in generation t, the ith individual in , the kth variable/component of individual , the kth variable of , the kth variable of , the crossover probability, the scaling factor, the ordinal number of in in descending order (see Section 4.1), the ’s corresponding job number in the job permutation (see Section 4.1), and the random value in the interval [0, 1]. Based on these definitions, the framework of DEA_UHT is given as follows:Step 1. Input , , , and let bounds be and , .Step 2. Population initializationStep 2.1. Generate for Step 2.2. Use the algorithm of OCBAT in Sections 3.2 and 4.3 to allocate T sampling budgets (i.e., evaluation times) for all individuals while estimating the objective values of all individuals and set set.Step 3. Set and select an individual from as with the minimum objective value.Step 4. Evolution phase (between Steps 4 and 9). Set .Step 5.Mutation and crossover phase.Step 5.1. If belongs to , then execute the Insert-based exploitation (see Section 4.4).Step 5.1.1. Randomly select one individual from the best five percent of and set it as .Step 5.1.2. Randomly generate u and (), and randomly set to 1 or 2.Step 5.1.3. If , then insert after .Step 5.1.4. If , then insert before .Step 5.1.5. Set and go to Step 6.Step 5.2. Perform the Interchange-based exploration (see Section 4.4). Set the trial vector, and . Randomly select , where , and randomly select .Step 5.2.1. Set , and .Step 5.2.2. If , then find satisfying and interchange with ; else randomly generate () and interchange with .Step 5.2.3. Set .Step 5.2.4. Set mod and .Step 5.2.5. If and , then return to Step 5.2.1.Step 5.2.6. Set .Step 6. Set i = i+1. If , then return to Step 5.Step 7. Use the algorithm of OCBAT in Sections 3.2 and 4.3 to estimate the objective values of all individuals of .Step 8. Selection phase. Perform HHT in Section 3.3 on with , .Step 8.1. If the null hypothesis holds and , then set a Boolean variable b_replace = true; else set b_replace = falseStep 8.2. If b_replace = true then set ; else set .Step 8.3. If b_replace = true and , then set .Step 8.4. If b_replace = false and is in the worst 20 percent of , then add to .Step 9. Set t = t+1. If (the maximum generation), then return to Step 4.Step 10. Output () and its objective value.

From the above procedure, it can be known that DEA_UHT not only utilizes OCBAT to ensure reliable evaluation for good solutions but also adopts HTT to reduce repetitive search. Furthermore, the DEA-based search combined with two special strategies is devised to effectively execute exploration and exploitation in solution space. Meanwhile, “” in Step 5.2.5 is used to control the search range of the Interchange-based exploration. In addition, it is worth noting that although Step 5.1 (resp. Step 5.2) directly performs the Insert (resp. Interchange) operation on the individual, it is equivalent to performing the same Insert (resp. Interchange) operation on the corresponding permutation of this individual (see the RLOV rule in Section 4.1).

5. Simulation Results and Comparisons

5.1. Experimental Setup

To verify DEA_UHT’s performance, ten instances of the deterministic RJSSP are created via the following steps:Step 1. Select one out of ten JSSP’s classic benchmarks [46] in turn as PP(i.e., FT06, FT10, FT20, LA01, LA06, LA11, LA16, LA21, LA26, and LA31 [46]).Step 2. Set the reentrant times .Step 3. Set each and each to the corresponding data in PP.Step 4. Set for and .Step 5. Set for .Step 6. Set for and .Step 7. Set for .

Using the above steps, ten RJSSP’s instances can be created. These instances are named FT06_R2, FT10_R2, FT20_R2, LA01_R2, LA06_R2, LA11_R2, LA16_R2, LA21_R2, LA26_R2, and LA31_R2, respectively. Each of these names is formed by adding “_R2” after the name of the corresponding JSSP’s benchmark. Obviously, these RJSSP’s instances are obtained by simply extending the classical JSSP’s instances. Since each and are randomly generated between [1, n] and [1, 100], respectively, the internal correlation within each RJSSP’s instance can be ignored. Using the above steps to obtain the RJSSP’s instances is just for the convenience of other researchers to generate the same instances.

For any created instance, each job’s due date can be obtained via the following steps:Step 1. Randomly created a permutation for each RJSSP’s instance .Step 2. Calculate each via the decoding scheme in Section 4.2.Step 3. Specify each job’s due date viawhere is a random value between . Because the ability to address tight due dates affects an enterprise’s competence and the problem under tight due dates is more difficult to solve than under loose due dates, (10) is designed to obtain a tight due date.

Next, the instances of the stochastic RJSSP can be obtained by changing each original processing time of the above RJSSP’s instances to a uniform distribution , where is the expected processing time that equals to the original and denotes noise magnitude. To test the performance of the algorithm more comprehensively, two kinds of stochastic RJSSPs are considered. The first kind is the stochastic RJSSP under . Ten instances with and them with are generated. These instances are denoted by S1_Iname_R2_Tmax, where Iname indicates that S1_Iname_R2_Tmax is derived from the deterministic instance Iname. The second kind is the stochastic RJSSP under , whose ten instances with are generated. Similarly, these instances are donated by S1_Iname_R2_Cmax. Moreover, since it is a common scenario that only a part of the machines are unstable, a variant of the first kind of problem is also considered. This variant, namely, S2_Iname_R2_Tmax, only sets the processing time of each job on machine one and machine three to a uniform distribution

The proposed DEA_UHT and the other compared algorithms were implemented in Delphi 2010 and the experiments were executed on an Intel 3.0 GHz PC with 8 GB memory. The main parameters of DEA_UHT are set as follows: the population size ,  = 0.7,  = 0.1, , , and . The maximum generation of each compared algorithm is set to 300. Each compared algorithm is executed thirty times independently on each instance.

To show the effectiveness of DEA_UHT, two variants of it are compared. The abbreviations of these variants are given as follows:(i)PDEA: DEA_UHT without OCBAT, HTT, the Insert-based exploitation strategy and the Interchange-based exploration strategy. The details can be described as follows:(1)DEA_UHT without OCBAT. At Steps 2.2 and 7 of DEA_UHT, the algorithm of OCBAT is replaced with twenty evaluation times for each solution’s performance estimation.(2)DEA_UHT without HTT. At Step 8 of DEA_UHT, the HTT-based selection is replaced with the normal DE’s selection.(3)DEA_UHT without the Insert-based exploitation strategy: Step 5.1 is deleted.(4)DEA_UHT without the Interchange-based exploitation strategy: Steps 5.2.2 and 5.2.3 are deleted, and “” in Step 5.2.5 is replaced with “.(2)DEA_UHT_O. DEA_UHT without HTT, the Insert-based exploitation strategy and the Interchange-based exploration strategy.

5.2. Simulation Results and Comparisons

The statistical results are summarized in Tables 2 and 3, where is the noise magnitude, BRE is the best expected maximum tardiness or makespan, and ARE is the average expected maximum tardiness or makespan. To make the statistical results clearer, in Tables 2 and 3, the best and the second-best values in each row are represented by using the bold and the italic fonts, respectively. For the final output solution obtained by each compared algorithm at each run, its expected objective value or is calculated with the expected processing time (i.e., the processing time without noise). Obviously, BRE and ARE can reflect the performance of these compared algorithms in a statistical sense. Moreover, to further show the statistical distributions of the results in Tables 1 and 2, the violin plots for the algorithm pairs “PDEA vs. DEA_UHT” and “DEA_UHT_O vs. DEA_UHT” are given in Figures 1 and 2.

From Tables 2 and 3, it is clear that both the BRE values and the ARE values obtained by DEA_UHT are better than those obtained by PDEA, which validates the effectiveness via incorporating OCBAT, HTT, the Insert-based exploitation strategy, and the Interchange-based exploration strategy into PDEA. This also means that DEA_UHT can achieve a better balance between exploration and exploitation than PDEA and DEA_UHT_O. Meanwhile, most of the BRE values and all of the ARE values obtained by DEA_UHT_O are better than those obtained by PDEA, from which it can be concluded that OCBAT is necessary to improve the optimization performance. It is worth noting that the above conclusions can still be drawn when considering the instances in Tables 1 and 2, where . In addition, from Figures 1 and 2, it can be seen that DEA_UHT has better statistical distributions and better median lines compared with PDEA and DEA_UHT_O. These violin plots confirm the competitiveness of DEA_UHT.

To show the role of OCBAT, the computing budget allocation of OCBAT in DEA###XXX###5F;UHT for the middle-size instance S1###XXX###5F;LA11###XXX###5F;R2###XXX###5F; is provided in Figure 3. In addition, to illustrate the effectiveness of DEA###XXX###5F;UHT, the convergence curves of the compared algorithms on the same instance are given in Figure 4, where the curves are formed by connecting the ’s value of the best individual at each generation of the corresponding algorithm. here is calculated by setting (i.e., ). The computing budget allocation and the convergence curves for other instances are similar as those in Figures 3 and 4.

From Figure 3, it is shown that OCBAT can intelligently assign more evaluation times to better individuals. That is to say, OCBAT can enhance the algorithm’s ability to find more reliable high-quality solutions or regions at each generation under the same computing budget. From Figure 4, it can be seen that the convergence speed of DEA_UHT is obviously faster than those of the other compared algorithms. This manifests that DEA_UHT can guide the whole population to move to the bottom of the expected “big valley” (i.e., the bottom of the “big valley” without noise) in solution space faster, and thus has better search performance. Moreover, the ’s values of DEA_UHT_O are less than those of PEDA in most cases, which verifies that OCBAT helps to guide the search to really promising regions in noisy environment.

6. Conclusion and Future Research

This paper proposed a novel DEA_UHT by combining DEA with OCBAT and HTT to address the stochastic RJSSP. This kind of problem has many real-world applications in modern enterprises. To the best of the authors’ knowledge, this is the first time that DEA has been applied to such an important problem. With the help of OCBAT to intelligently allocate limited sampling budgets to perform reliable identification on excellent or good individuals, and with the help of HTT to reserve good individuals with high confidence, as well as with the Insert-based and Interchange-based strategies to effectively execute enhanced exploitation and exploration, DEA_UHT is of good performance for different kinds of stochastic RJSSP. In the future, our work is to extend DEA_UHT to other kinds of stochastic scheduling problems.

Data Availability

Data were curated by the authors and are available upon request.

Conflicts of Interest

The authors declare that they have no conflicts of interest.


This research was partially supported by the National Natural Science Foundation of China (61963022 and 62173169) and the Basic Research Key Project of Yunnan Province (202101AS070097).