Function Optimization using Genetic Algorithm and Particle Swarm Optimization-Results and Comparison

In this article some experiments and their results are discussed for minimization of Rastrigin function, a famous mathematical function used in Optimization Techniques evaluation. The experiments are performed on certain setup, parameters and system. This has been performed using Genetic Algorithm (GA) and Particle Swarm Optimization (PSO). The fitness function is the given function to be optimized. This article just pays focus on the way experiments are to be performed and analyzed in students lab and by mentors teaching Machine Learning results do not play a role in benchmarking or changing existing theories. It is just for elaboration for the purpose of academicians and students to learn the art of optimizing function with the help of GA and PSO.

This function is given by

The function is usually evaluated on x_i ∈ [-5.12, 5.12], for all i = 1, …, n . This has been tried on 5 different values of n between 10 and 100 as follows. Taking N as the size of the chromosome for GA and PSO implementation and population size of 10.

The Matlab implementation of fitness function is

%Fitness function function z= testfunction(x) %z= (x’x); [M,N]=size(x); z=[]; sum =0; for j=1:M for i = 1:N sum = sum + (x(j,i)x(j,i) – 10 * cos (23.14x(j,i))); end z= [z;sum + 10*N] end

The following table shows the experimental results.

N	GA Final Minimum Value after 500 iterations	PSO Global Minimum Value after 500 iterations with C1=1, C2=2	PSO Global Minimum Value after 500 iterations with C1=3, C2=3	PSO Global Minimum Value after 500 iterations with C1=3, C2=2	PSO Global Minimum Value after 500 iterations with C1=2, C2=3
10	27.8663(500 epochs)/ 9.1 in 1000 epochs	12.723	4.0002	8.4609	7.0004
20	113.9635(500 epochs)	74.584	13.001	31.879	29.679
40	399.6707 (500 epochs) / 235 in 1000 epochs	191.02	43.23	127.1	115.58
60	918.9643(500 epochs)/ 619 (1000 epochs)	372.2	96.065	193.74	189.75
80	1392.1043(500 epochs)	489.16	220.28	314.04	286.96

The general analysis of results is that PSO is performing better than GA here given the same number of iterations. Though GA solutions are gradually decreasing while PSO solutions are oscillating.  Within PSO the c1=3, c2=3 seems to perform best, c1=1 and c2=2 seem to perform the worst in achieving global minimum. More details and comments below.

Num of chromosomes : 10

One point crossover GA., with 5% mutation rate.

Maximum  number of iterations: 500.

Graphical Visualization of Results

Command for plots:

plot(ga.iters,ga.minc,’:’,pso1.iters,pso1.minc,’green’, pso2.iters, pso2.minc ,’blue’ ,pso3.iters ,pso3.minc, ‘red’,pso4.iters,pso4.minc,’black’);

Here is plot for N=80

Green: PSO1: with parameters : c1=1,c2=2 : Oscillates and convergence not as fast as other parameters.

Blue: PSO2 : with parameters : c1=3,c2=3  :  Oscillates and reaches towards minima.

Red: PSO3 : with parameters : c1=3,c2=2  : Oscillates.  Reaches towards minima

Black: PSO4 : with parameters :  c1=2,c2=3 : Oscillates.  Reaches  towards minima

Dots : GA, still decreasing and is more stable though global minimum is not less than PSO.

And comparing GA and PSO , GA seems not to oscillates and be stable though its minimum not attained as fast.

PSO1 with c1=1,c2=2, seem to not be as fast in attaining minimum as compared to other parameters which are almost equivalent in attaining minimum. Here is separate result of c1=1, c2=2.

Plot of the three competing models is as follows:

Results for N=60

Green: PSO1: with parameters: c1=1,c2=2: Oscillates and convergence not as fast as other parameters.

Blue: PSO2 : with parameters : c1=3,c2=3  :  Oscillates and reaches towards minima.

Red: PSO3 : with parameters : c1=3,c2=2  : Oscillates.  Reaches towards minima

Black: PSO4 : with parameters :  c1=2,c2=3 : Oscillates.  Reaches  towards minima

Dots: GA, still decreasing and is more stable though global minimum is not less than PSO.

And comparing GA and PSO , GA seems not to oscillates and be stable though its minimum not attained as fast.

Plot of three competing models as follows

PSO1 with c1=1,c2=2, seem to not be as fast in attaining minimum as compared to other parameters which are almost equivalent in attaining minimum. Here is separate result of c1=1, c2=2.

Results for N=40

Green: PSO1: with parameters : c1=1,c2=2 : Oscillates and convergence not as fast as other parameters.

Blue: PSO2 : with parameters : c1=3,c2=3  :  Oscillates and reaches towards minima.

Red: PSO3 : with parameters : c1=3,c2=2  : Oscillates.  Reaches towards minima

Black: PSO4 : with parameters :  c1=2,c2=3 : Oscillates.  Reaches  towards minima

Dots: GA, still decreasing and is more stable though global minimum is not less than pso.

And comparing GA and PSO , GA seems not to oscillates so much and be stable though its minimum not attained as fast.

PSO1 with c1=1,c2=2, seem to not be as fast in attaining minimum as compared to other parameters which are almost equivalent in attaining minimum. Here is separate result of c1=1, c2=2.

Results for N=20

Green: PSO1: with parameters : c1=1,c2=2 : Oscillates and convergence not as fast as other parameters.

Blue: PSO2 : with parameters : c1=3,c2=3  :  Oscillates and reaches towards minima.

Red: PSO3 : with parameters : c1=3,c2=2  : Oscillates.  Reaches towards minima

Black: PSO4 : with parameters :  c1=2,c2=3 : Oscillates.  Reaches  towards minima

Dots: GA, still decreasing and is more stable though global minimum is not less than PSO.

And comparing GA and PSO , GA seems not to oscillates and be stable though its minimum not attained as fast.

PSO1 with c1=1,c2=2, seem to not be as fast in attaining minimum as compared to other parameters which are almost equivalent in attaining minimum. Here is separate result of c1=1, c2=2.

Results for N=11

Green: PSO1: with parameters : c1=1,c2=2 : Oscillates and convergence not as fast as other parameters.

Blue: PSO2 : with parameters : c1=3,c2=3  :  Oscillates and reaches towards minima.

Red: PSO3 : with parameters : c1=3,c2=2  : Oscillates.  Reaches towards minima

Black: PSO4 : with parameters :  c1=2,c2=3 : Oscillates.  Reaches  towards minima

Dots: GA, still decreasing and is more stable though global minimum is not less than PSO

And comparing GA and PSO , GA seems not to oscillates and be stable though its minimum not attained as fast.

PSO1 with c1=1,c2=2, seem to not be as fast in attaining minimum as compared to other parameters which are almost equivalent in attaining minimum. Here is separate result of c1=1, c2=2.

The experiments are performed on certain setup, parameters and system. The general impression and analysis are given at end of each experiment conducted. Changing parameters, the mutations functions, population to be performed for creating new chromosomes in case of GA drastically effects the results obtained. In a similar manner for PSO several parameters, initializations effect the results apart from C1 and C2 experimented above. The results are not benchmarks, they are elaborated for experimental setups in labs, and to be followed by mentors teaching Machine Learning Lab Work. For benchmark results look into peer-reviewed research papers.