I Introduction
Optimisation problems exist in a variety of scientific fields, varying from medicine to agriculture. While conventional optimisation algorithms are popular, they suffer from drawbacks such as getting stuck in local optima and being sensitive in the initial state [17]. To tackle these problems, populationbased metaheuristic algorithms such as particle swarm optimisation [9] offer a powerful alternative thanks to their wellrecognised characteristics, such as selfadaptation and being derivative free [14].
Differential evolution (DE) [19] is a simple yet effective populationbased algorithm which has shown good performance in solving optimisation problems in areas including image processing [5, 12]
[15, 16, 2], and economics [1, 20]. DE is based on three primary operators: mutation, which generates new candidate solutions based on scaling differences among candidate solutions, crossover, which combines a mutant vector with the parent one, and selection, which selects a better candidate solution from a new one and its parent.The performance of DE is directly related to these operators [8]. Among them, the mutation operator plays a crucial role to generate new promising candidate solutions and significant recent work has focussed on developing effective mutation operators. [23] proposes a multipopulation DE which combines three different mutation strategies including currenttopbest/1, currenttorand/1, and rand/1. [21] employs three trial vector generation strategies and three control parameter settings and randomly selects between them to create new vectors. In [3], tournament selection is used to introduce selection pressure for selecting the base vector. [18] proposes a neighbourhoodbased mutation that is performed within each Euclidean neighbourhood. In [13], a competition scheme for generating new candidate solutions is introduced so that candidate solutions are divided into two groups, losers and winners. Winners create new candidate solutions based on standard mutation and crossover operators, while losers try to learn from winners.
In this paper, we propose a novel DE algorithm, CluDE, which employs a novel clusteringbased mutation operator. Inspired by the clustering operator in the human mental search (HMS) optimisation algorithm [11], CluDE clusters the current population into groups and selects a promising region as the cluster with the best mean objective function value. The best candidate solution in the promising region is selected as the base vector in the mutation operator. An updating strategy is then employed to include the new candidate solutions into the current population. Experimental results on CEC2017 benchmark functions with dimensionalities of 30, 50 and 100 confirm that CluDE yields improved performance compared to DE.
Ii Background
Iia Differential Evolution
Differential evolution (DE) [19] is a simple but effective populationbased optimisation algorithm based on three main operators: mutation, crossover, and selection.
The mutation operator generates a mutant vector for each candidate solution as
(1) 
where , , and are three distinct candidate solutions randomly selected from the current population and is a scaling factor.
Crossover shuffles the mutant vector with the parent vector. For this, binomial crossover, defined as
(2) 
is employed, where is called a trial vector, is the crossover rate, and is a random integer number between 1 and the number of dimensions.
Finally, the selection operator selects the better candidate solution from the new candidate solution and its parent to be passed to the new population.
IiB Clustering
Clustering is an unsupervised pattern recognition technique to partition samples into different groups so that the members of a cluster share more resemblance compared to members of different clusters. means [10] is the most popular clustering algorithm based on a similarity measure (typically Euclidean distance). It requires to define , the number of clusters, in advance and proceeds as outlined in Algorithm LABEL:Alg1.
algocf[h!]
Iii Proposed CluDE Algorithm
In this paper, we improve DE using a novel clusteringbased mutation and updating scheme. Our proposed algorithm, CluDE, is given, in the form of pseudocode in Algorithm LABEL:Alg2, while in the following we describe its main contributions.
algocf[t!]
Unimodal functions  
F1  Shifted and Rotated Bent Cigar Function  
F2  Shifted and Rotated Sum of Different Power Function  
F3  Shifted and Rotated Zakharov Function  
Multimodal functions  
F4  Shifted and Rotated Rosenbrock’s Function  
F5  Shifted and Rotated Rastrigin’s Function  
F6  Shifted and Rotated Expanded Scaffer’s Function  
F7  Shifted and Rotated Lunacek Bi_Rastrigin Function  
F8  Shifted and Rotated NonContinuous Rastrigin’s Function  
F9  Shifted and Rotated Levy Function  
F10  Shifted and Rotated Schwefel’s Function  
Hybrid multimodal functions  
F11  Hybrid Function 1 ()  
F12  Hybrid Function 2 ()  
F13  Hybrid Function 3 ()  
F14  Hybrid Function 4 ()  
F15  Hybrid Function 5 ()  
F16  Hybrid Function 6 ()  
F17  Hybrid Function 7 ()  
F18  Hybrid Function 8 ()  
F19  Hybrid Function 9 ()  
F20  Hybrid Function 10 ()  
Composite functions  
F21  Composition Function 1 ()  
F22  Composition Function 2 ()  
F23  Composition Function 3 ()  
F24  Composition Function 4 ()  
F25  Composition Function 5 ()  
F26  Composition Function 6 ()  
F27  Composition Function 7 ()  
F28  Composition Function 8 ()  
F29  Composition Function 9 ()  
F30  Composition Function 10 () 
Iiia Clusteringbased Mutation
For our improved mutation operator, CluDE first identifies a promising region in search space. This is performed, similar as in the HMS algorithm [11], using a clustering algorithm. We employ the well known means clustering algorithm to group the current population into clusters so that each cluster represents a region in search space. The number of clusters is selected randomly between 2 and [4, 16].
After clustering, the mean objective function value for each cluster is calculated, and the cluster with the best objective function value is then used to identify a promising region in search space. Fig. 1 illustrates this for a toy problem with 17 candidate solutions divided into three clusters.
Finally, our novel clusteringbased mutation is conducted as
(3) 
where and are two different randomlyselected candidate solutions, and is the best candidate solution in the promising region. It is worth noting that the best candidate solution in the winner cluster might not be the best candidate solution in the current population. Clusteringbased mutation is performed times following standard crossover and mutation.
IiiB Population Update
After generating new offsprings using clusteringbased mutation, the population is updated for which we employ a scheme based on the generic populationbased algorithm (GPBA) [6]. In particular, the population is updated in the following manner:

Selection: candidate solutions are selected randomly. This corresponds to the initial seeds for means clustering.

Generation: new candidate solutions are created as set . This is conducted by the clusteringbased mutation.

Replacement: candidate solutions are selected randomly from the current population as set B.

Update: From , the best individuals are selected as . The new population is then obtained as .
Iv Experimental results
To verify the efficacy of CluDE, we perform experiments on the CEC2017 benchmark functions [22], 30 functions with different characteristics including unimodal functions, multimodal functions, hybrid multimodal functions, and composite functions, summarised in Table I.
In all experiments, the maximum number of function evaluations is set to , where is the dimensionality of the search space.
The population size, crossover rate, and scaling factor are set to 50, 0.9, and 0.5, respectively. For CluDE,
is set to 10. Each algorithm is run 25 times independently, and we report mean and standard deviation over 25 runs.
To evaluate if there is a statistically significant difference between two algorithms, a Wilcoxon signedrank test [7]
is performed with a confidence interval of 95% on each function.
Table II gives the results of CluDE compared to standard DE for . From the table, we can see that CluDE statistically outperforms DE for 16 of the 30 functions, while obtaining equivalent performance for 12 functions. Only for two of the multimodal functions CluDE yields inferior results.
function  DE  CluDE  WSRT  
F1  avg.  8.68E+03  5.35E+03  = 
std.dev.  1.47E+04  5.74E+03  
F2  avg.  3.86E+19  1.97E+14  + 
std.dev.  1.93E+20  9.35E+14  
F3  avg.  9.43E+03  3.01E+02  + 
std.dev.  8.94E+03  2.67E+00  
F4  avg.  4.35E+02  4.49E+02  = 
std.dev.  2.08E+01  3.14E+01  
F5  avg.  6.85E+02  5.61E+02  + 
std.dev.  9.19E+00  2.62E+01  
F6  avg.  6.00E+02  6.00E+02  = 
std.dev.  1.60E04  2.25E01  
F7  avg.  9.12E+02  7.94E+02  + 
std.dev.  1.71E+01  2.88E+01  
F8  avg.  9.89E+02  8.63E+02  + 
std.dev.  1.21E+01  3.03E+01  
F9  avg.  9.00E+02  9.26E+02   
std.dev.  4.94E01  4.33E+01  
F10  avg.  8.76E+03  5.89E+03  + 
std.dev.  3.83E+02  9.66E+02  
F11  avg.  1.13E+03  1.13E+03  = 
std.dev.  2.07E+01  1.39E+01  
F12  avg.  2.55E+05  7.84E+04  + 
std.dev.  3.62E+05  7.16E+04  
F13  avg.  1.82E+04  2.16E+04  = 
std.dev.  2.15E+04  4.20E+04  
F14  avg.  1.47E+03  1.44E+03  + 
std.dev.  6.76E+00  1.12E+01  
F15  avg.  1.61E+03  1.58E+03  + 
std.dev.  8.88E+01  1.22E+02  
F16  avg.  3.09E+03  2.89E+03  + 
std.dev.  2.67E+02  3.33E+02  
F17  avg.  2.17E+03  2.22E+03  = 
std.dev.  2.01E+02  2.87E+02  
F18  avg.  1.09E+04  8.24E+03  = 
std.dev.  8.97E+03  7.25E+03  
F19  avg.  1.92E+03  1.92E+03  = 
std.dev.  5.71E+00  6.39E+00  
F20  avg.  2.32E+03  2.46E+03   
std.dev.  2.28E+02  2.20E+02  
F21  avg.  2.48E+03  2.35E+03  + 
std.dev.  7.34E+00  2.26E+01  
F22  avg.  9.99E+03  6.52E+03  + 
std.dev.  3.09E+02  2.36E+03  
F23  avg.  2.83E+03  2.72E+03  + 
std.dev.  1.09E+01  2.90E+01  
F24  avg.  3.01E+03  2.91E+03  + 
std.dev.  9.53E+00  4.32E+01  
F25  avg.  2.88E+03  2.88E+03  = 
std.dev.  1.16E+00  1.39E+00  
F26  avg.  5.26E+03  4.30E+03  + 
std.dev.  2.03E+02  2.35E+02  
F27  avg.  3.20E+03  3.20E+03  = 
std.dev.  1.32E04  2.49E04  
F28  avg.  3.30E+03  3.30E+03  = 
std.dev.  1.75E04  3.47E04  
F29  avg.  3.83E+03  3.52E+03  + 
std.dev.  2.48E+02  2.45E+02  
F30  avg.  3.22E+03  3.22E+03  = 
std.dev.  8.07E+00  1.90E+01  
wins/ties/losses for CluDE  16/12/2 
When increasing the number of dimensions to 50, for which the results are listed in Table III, CluDE retains its efficacy. As can be seen, it statistically outperforms standard DE for 12 of the 30 functions, while giving similar results for 16 functions.
function  DE  CluDE  WSRT  
F1  avg.  5.07E+03  6.25E+03  = 
std.dev.  4.66E+03  9.31E+03  
F2  avg.  1.02E+39  3.76E+42  = 
std.dev.  4.84E+39  1.88E+43  
F3  avg.  2.41E+05  8.26E+03  + 
std.dev.  5.65E+04  4.56E+03  
F4  avg.  4.49E+02  4.87E+02   
std.dev.  2.42E+01  4.00E+01  
F5  avg.  8.61E+02  6.23E+0+  + 
std.dev.  2.04E+01  3.77E+01  
F6  avg.  6.00E+02  6.00E+02  = 
std.dev.  7.71E02  1.41E+00  
F7  avg.  1.12E+03  9.40E+02  + 
std.dev.  1.63E+01  5.26E+01  
F8  avg.  1.15E+03  9.20E+02  + 
std.dev.  6.57E+01  3.22E+01  
F9  avg.  9.09E+02  1.44E+03   
std.dev.  1.04E+01  4.37E+02  
F10  avg.  1.54E+04  9.73E+03  + 
std.dev.  3.99E+02  1.57E+03  
F11  avg.  1.20E+03  1.21E+03  = 
std.dev.  6.55E+01  3.32E+01  
F12  avg.  1.96E+06  1.89E+06  = 
std.dev.  1.47E+06  1.17E+06  
F13  avg.  1.13E+04  2.88E+04  = 
std.dev.  1.39E+04  4.45E+04  
F14  avg.  7.39E+03  1.14E+04  = 
std.dev.  1.20E+04  1.55E+04  
F15  avg.  3.24E+04  1.73E+04  = 
std.dev.  4.17E+04  2.24E+04  
F16  avg.  4.69E+03  4.08E+03  + 
std.dev.  5.37E+02  7.70E+02  
F17  avg.  3.34E+03  3.30E+03  = 
std.dev.  3.78E+02  3.67E+02  
F18  avg.  1.09E+05  5.80E+04  + 
std.dev.  6.76E+04  3.75E+04  
F19  avg.  1.00E+04  1.07E+04  = 
std.dev.  1.39E+04  8.79E+03  
F20  avg.  3.47E+03  3.55E+03  = 
std.dev.  3.53E+02  4.40E+02  
F21  avg.  2.66E+03  2.41E+03  + 
std.dev.  1.70E+01  3.07E+01  
F22  avg.  1.67E+04  1.24E+04  + 
std.dev.  3.92E+02  1.79E+03  
F23  avg.  3.07E+03  2.90E+03  + 
std.dev.  3.71E+01  6.85E+01  
F24  avg.  3.28E+03  3.06E+03  + 
std.dev.  1.60E+01  5.22E+01  
F25  avg.  2.94E+03  2.97E+03  = 
std.dev.  2.32E+01  3.21E+01  
F26  avg.  7.35E+03  5.35E+03  = 
std.dev.  4.57E+02  5.20E+02  
F27  avg.  3.20E+03  3.20E+03  = 
std.dev.  1.30E04  4.01E04  
F28  avg.  3.30E+03  3.30E+03  = 
std.dev.  1.67E04  4.20E04  
F29  avg.  5.04E+03  4.23E+03  + 
std.dev.  3.66E+02  4.76E+02  
F30  avg.  7.09E+03  7.48E+03  = 
std.dev.  4.81E+03  4.70E+03  
wins/ties/losses for CluDE  12/16/2 
For , the results are given in Table IV. As we can see from there, CluDE obtains better or similar results for 24 of the 30 functions, thus clearly outperforming DE also for highdimensional problems.
function  DE  CluDE  WSRT  
F1  avg.  9.77E+03  4.00E+03  + 
std.dev.  1.21E+04  6.94E+03  
F2  avg.  7.36E+92  2.29E+105   
std.dev.  3.68E+93  1.14E+106  
F3  avg.  1.58E+06  1.54E+05  + 
std.dev.  3.59E+05  3.85E+04  
F4  avg.  5.94E+02  6.57E+02  = 
std.dev.  5.55E+01  4.92E+01  
F5  avg.  1.21E+03  8.95E+02  + 
std.dev.  3.11E+02  7.70E+01  
F6  avg.  6.01E+02  6.13E+02   
std.dev.  4.44E01  3.06E+00  
F7  avg.  1.74E+03  1.48E+03  + 
std.dev.  4.17E+01  1.52E+02  
F8  avg.  1.51E+03  1.18E+03  + 
std.dev.  3.02E+02  1.20E+02  
F9  avg.  1.91E+03  1.00E+04   
std.dev.  1.49E+03  4.80E+03  
F10  avg.  3.28E+04  2.29E+04  + 
std.dev.  5.47E+02  2.92E+03  
F11  avg.  3.50E+03  1.61E+03  + 
std.dev.  1.22E+03  2.39E+02  
F12  avg.  6.72E+06  1.20E+07   
std.dev.  4.00E+06  6.59E+06  
F13  avg.  7.94E+03  9.60E+03  = 
std.dev.  9.64E+03  1.31E+04  
F14  avg.  4.64E+05  4.33E+05  = 
std.dev.  3.51E+05  2.82E+05  
F15  avg.  6.26E+03  8.32E+03  = 
std.dev.  6.46E+03  9.75E+03  
F16  avg.  1.01E+04  7.18E+03  + 
std.dev.  3.67E+02  1.56E+03  
F17  avg.  7.00E+03  6.15E+03  + 
std.dev.  7.39E+02  8.37E+02  
F18  avg.  9.74E+05  5.34E+05  + 
std.dev.  4.26E+05  3.62E+05  
F19  avg.  3.88E+03  5.25E+03  = 
std.dev.  3.09E+03  4.40E+03  
F20  avg.  7.07E+03  6.61E+03  + 
std.dev.  7.01E+02  7.97E+02  
F21  avg.  3.08E+03  2.72E+03  + 
std.dev.  2.74E+02  6.93E+01  
F22  avg.  3.46E+04  2.57E+04  + 
std.dev.  5.29E+02  2.41E+03  
F23  avg.  3.05E+03  3.32E+03   
std.dev.  3.71E+01  6.19E+01  
F24  avg.  4.07E+03  3.97E+0=  = 
std.dev.  2.76E+02  1.05E+02  
F25  avg.  3.26E+03  3.29E+03  = 
std.dev.  7.68E+01  5.90E+01  
F26  avg.  1.13E+04  1.37E+04   
std.dev.  3.48E+03  1.12E+03  
F27  avg.  3.20E+03  3.20E+03  = 
std.dev.  1.42E04  3.31E04  
F28  avg.  3.30E+03  3.30E+03  = 
std.dev.  7.72E05  1.11E+01  
F29  avg.  8.30E+03  6.81E+03  + 
std.dev.  8.11E+02  9.69E+02  
F30  avg.  1.02E+04  1.50E+04  = 
std.dev.  9.45E+03  1.86E+04  
wins/ties/losses for CluDE  14/10/6 
Last but not least, Figure 2 shows convergence curves of our proposed algorithm compared to DE for, as representative examples, F10 and F15 and all dimensionalities. As we can observe, CluDE converges faster than standard DE.
V Conclusions
In this paper, we have proposed a novel differential evolution algorithm, CluDE, based on a novel clusteringbased mutation operator. A promising region in search space is found using means clustering and some new candidate solutions are generated using the proposed clusterbased mutation. A population update scheme is introduced to include the new candidate solutions into the current population. Extensive experiments on the CEC2017 benchmark functions and for dimensionalities of 30, 50 and 100 verify that CluDE is a competitive variant of DE. In future work, we intend to extend CluDE for multiobjective optimisation problems.
References

[1]
(2018)
Wavelets optimization method for evaluation of fractional partial differential equations: an application to financial modelling
. Advances in Difference Equations 2018 (1), pp. 8. Cited by: §I. 
[2]
(2016)
Differential evolutionbased neural network training incorporating a centroidbased strategy and dynamic oppositionbased learning
. InIEEE Congress on Evolutionary Computation
, pp. 2958–2965. Cited by: §I.  [3] (2019) Adaptive ktournament mutation scheme for differential evolution. Applied Soft Computing 85, pp. 105776. Cited by: §I.
 [4] (2011) A clusteringbased differential evolution for global optimization. Applied Soft Computing 11 (1), pp. 1363–1379. Cited by: §IIIA.
 [5] (2009) Automatic image pixel clustering with an improved differential evolution. Applied Soft Computing 9 (1), pp. 226–236. Cited by: §I.
 [6] (2005) A populationbased algorithmgenerator for realparameter optimization. Soft Computing 9 (4), pp. 236–253. Cited by: §IIIB.
 [7] (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm and Evolutionary Computation 1 (1), pp. 3–18. Cited by: §IV.
 [8] (2020) Selfadaptive collective intelligencebased mutation operator for differential evolution algorithms. The Journal of Supercomputing 76 (2), pp. 876–896. Cited by: §I.
 [9] (1995) Particle swarm optimization (PSO). In IEEE International Conference on Neural Networks, pp. 1942–1948. Cited by: §I.

[10]
(1967)
Some methods for classification and analysis of multivariate observations.
In
5th Berkeley Symposium on Mathematical Statistics and Probability
, pp. 281–297. Cited by: §IIB.  [11] (2017) Human mental search: a new populationbased metaheuristic optimization algorithm. Applied Intelligence 47 (3), pp. 850–887. Cited by: §I, §IIIA.
 [12] (2020) Manylevel image thresholding using a centerbased differential evolution algorithm. In Congress on Evolutionary Computation, Cited by: §I.
 [13] (2019) Differential evolution algorithm based on a competition scheme. In 14th International Conference on Computer Science and Education, Cited by: §I.
 [14] (2020) CenPSO: a novel centerbased particle swarm optimization algorithm for largescale optimization. In International Conference on Systems, Man, and Cybernetics, Cited by: §I.
 [15] (2020) Evolving feedforward neural networks using a quasioppositionbased differential evolution for data classification. In IEEE Symposium Series on Computational Intelligence, Cited by: §I.
 [16] (2021) RDEOP: a regionbased differential evolution algorithm incorporation oppositionbased learning for optimising the learning process of multilayer neural networks. In 24th International Conference on the Applications of Evolutionary Computation, Cited by: §I, §IIIA.
 [17] (2019) A globalbest guided human mental search algorithm with random clustering strategy. In International Conference on Systems, Man and Cybernetics, pp. 3174–3179. Cited by: §I.
 [18] (2012) Differential evolution with neighborhood mutation for multimodal optimization. IEEE Transactions on Evolutionary Computation 16 (5), pp. 601–614. Cited by: §I.

[19]
(1997)
Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces
. Journal of Global Optimization 11 (4), pp. 341–359. Cited by: §I, §IIA.  [20] (2019) A differential evolutionoriented pruning neural network model for bankruptcy prediction. Complexity 2019. Cited by: §I.
 [21] (2011) Differential evolution with composite trial vector generation strategies and control parameters. IEEE Transactions on Evolutionary Computation 15 (1), pp. 55–66. Cited by: §I.
 [22] (2016) Problem definitions and evaluation criteria for the CEC 2017 competition on constrained realparameter optimization. Technical report Nanyang Technological University, Singapore. Cited by: TABLE I, §IV.
 [23] (2016) Differential evolution with multipopulation based ensemble of mutation strategies. Information Sciences 329, pp. 329–345. Cited by: §I.
Comments
There are no comments yet.