IGO learning rate continued

I’ve ran some more tests. Same setup as previously using 100 samples in the optimization with various learning rate values.

eta=0.25

eta=0.21875

eta=0.1875

eta=0.15625

eta=0.125

eta=0.09375

eta=0.0625

eta=0.03125

February 4, 2020February 5, 2020

Evolution of mean objective value for 2D Rastrigin function optimization. Rastrigin function is shited so that the min is at [pi,pi] and the optimization is bounded to [0,2*pi]x[0,2*pi] with various learning rate values (eta)

Left: using 100 samples Right: using 1000 samples.

Single: using 100 samples

eta = 0.3

eta = 0.6

eta = 0.9

eta = 1.2

eta = 1.5

eta = 1.8

eta = 2.1

eta = 2.4

eta = 2.7

eta = 3

eta = 3.3

eta = 3.6

eta = 3.9

eta = 4.2

eta = 4.5

eta = 4.8

eta = 5.1

eta = 5.4

eta = 5.7

eta = 6

January 15, 2020January 26, 2020

Comparison of the Gibbs sampler for the extension of the multivariate von Mises distribution and rejection sampling algorithm for the multivariate von Mises distribution

I did a comparison of the Mardia rejection sampling algorithm for the sine model of the bivariate von Mises and the trivariate von Mises distributions and the Gibbs sampling algorithm for my extension of the multivariate von Mises distribution in the case when the distribution coincide.

Left: Mardia rejection sampling algorithm. Right: Gibbs sampling algorithm.

Bivariate von Mises

BVM(0,0,0)

BVM(pi,10,2)

BVM(0,0,10)

Trivariate von Mises

TVM(0,0,10*[0 -1 1;-1 0 1;1 1 0])

projection along z axis

projection along y axis

projection along x axis

December 24, 2019December 24, 2019

Generalized von Mises distribution sampler using von Neumann’s rejection sampling algorithm.

Algorithm: Some computational aspects of the generalized von Mises distribution

100000 samples generated for every value of parameters.

The red line is the generalized von Mises density function.

mu_1 = 4.5, mu_2 = 1, kappa_1 = 0.8, kappa_2 = 2

mu_1 = 2, mu_2 = 1, kappa_1 = 3, kappa_2 = 0

mu_1 = 2, mu_2 = 1, kappa_1 = 0 kappa_2 =3

mu_1 = 2, mu_2 = 1, kappa_1 = 0 kappa_2 =0

November 17, 2019November 17, 2019

AI3SD presentation

Slides: AI3SDpresentation.pdf

November 12, 2019November 13, 2019

Heptagon space group packings using xNES

I started with plane group packings of heptagons. The heptagons are not precisely regular as I used a simple construction method. That is I placed points on a unit circle with the angle difference of 2*pi/7.

For every space group 100 experiments were performed with random initial configurations. The way the algorithm works now is that it doesn’t need feasible initial configuration.

The best know packing of heptagons is 0.892690686126509 which is also believed to be optimal. Same double lattice packing as withe pentagons. https://blogs.ams.org/visualinsight/2014/11/15/packing-regular-heptagons/

Plane group: p2

Max density: 0.892607642589804

Mean density: 0.8425

Density variance: 0.0015

Number of infeasible solutions: 0

0.4777
0.2507
0.8900
1.9010
3.7421
1.0389

Plane group: cm

Max density: 0.8414

Mean density: 0.7251

Density variance: 0.0033

Number of infeasible solutions: 0

Plane group: p2mm

Max density: 0.7365

Mean density: 0.5760

Density variance: 0.0125

Number of infeasible solutions: 0

Plane group: p2mg

Max density: 0.8238

Mean density: 0.6826

Density variance: 0.0097

Number of infeasible solutions: 0

Plane group: p2gg

Max density: 0.892690618215488

Mean density: 0.8643

Density variance: 6.5725e-04

Number of infeasible solutions: 0

Plane group: c2mm

Max density: 0.7390

Mean density: 0.5719

Density variance: 0.0040

Number of infeasible solutions: 0

Plane group: p3m1

Max density: 0.5718

Mean density: 0.5377

Density variance: 0.0022

Number of infeasible solutions: 0

0.3232
0.3333
4.4878
5.7579

October 29, 2019November 1, 2019

xNES constraint handling for pentagon packings in plane group pg.

For every constraint handling method 100 optimization procedures were performed with random initial configurations. The proved optimal packing density is (5-sqrt(5))/3 ~ 0.921310674166737

Penalty function based on feasibility

I used the penalty function from this paper: An efficient constraint handling method for genetic algorithms

The closest the algorithm got to the optimum was 0.921310674166735 in 2 instances with the constraint violation of 1.110223024625157e-16. This is also the minimum constraint violation achieved in 100 runs. This same value of constraint violation had 53 other instances with mean 0.921310674166652 and variance 2.276453802275327e-25.

The overall mean density from the 100 runs was 0.8926 and variance 0.0061.

In 99 instances the output solution violated constrains. In these cases the mean of constraint violation was 0.002671273212076 and variance 1.732259359382852e-04.

Dynamic penalty

I’ve tested the dynamic penalty function from: A survey of constraint handling techniques used with evolutionary algorithms

In 100 experiments all of the output solution violated the constraints. the minimum constrain violation in these 100 instances was 0.8888 and mean constraint violation 1.5547. This is very bad.

An interesting observation is that when I replaced the values of the objective function with the maximum of feasible solutions (as in Penalty function based on feasibility) in the penalty function the algorithm suddenly performed very well.

Annealing penalty

Same case as with dynamic penalty.

Death penalty

Not really good. Gets stuck in configurations where it can’t find any feasible samples very quickly.

Repair

Very slow and not very precise due to the way the repairing is implemented. That is by multiplying the unit cell by a small constant c>1 until there is no overlap.

Augmented lagrangian method

I’ve implemented the augmented lagrangian method for the xNES from this paper: Augmented Lagrangian Genetic Algorithm for Structural Optimization

Same as in the case of dynamic penalties. All of the 100 experiments outpu solutions violated the constraints. The mean constraint violation was 1.0899 with variance 0.0020. Not useful at all.

Augmented lagrangian method with penalty function based on feasibility

In 45 cases the algorithms output solution’s nonlinear constrains were distanced from zero less then 1.110223024625157e-16. Out of these 32 were unfeasible. From the feasible solutions the maximum density was 0.921310674166734 and the constraint violation of 0. Below is a picture of this solution.

The mean density of these 45 cases was 0.9205 and variance 2.0456e-05. Overall the mean density was 0.8777 and variance 0.0107. In 79 cases the constraints were violated. with mean constraint violation of 0.0017 and variance 6.7180e-05.

October 3, 2019October 6, 2019

CMO Talk

Slides: CMOpresentation.pdf

August 13, 2019August 24, 2019

Plane group packings with regular pentagons

First experiments with packing pentagons using plane groups. Genetic algorithm was used to optimize the packing density. It seems that the optimal packing density is approx. 0.92 ( Packings of Regular Pentagons in the Plane ).