Detail publikace

A critical problem in benchmarking and analysis of evolutionary computation methods

KŮDELA, J.

Anglický název

A critical problem in benchmarking and analysis of evolutionary computation methods

Typ

článek v časopise ve Web of Science, Jimp

Jazyk

en

Originální abstrakt

Benchmarking constitutes a cornerstone in the analysis and development of computational methods. Especially in the field of evolutionary computation, where theoretical analysis of the algorithms is almost impossible, benchmarking is at the center of attention. In this text, we show that some of the frequently used benchmark functions that have their respective optima in the center of the feasible set pose a critical problem for the analysis of evolutionary computation methods. We carry out the analysis of seven recent methods, published in respected journals, which contain a center-bias operator that lets them find these optima with ease. This makes their comparison with other methods (that do not have a center-bias) meaningless on such types of problems. We perform a computational comparison of these methods with two of the oldest methods in evolutionary computation on shifted problems and on more advanced benchmark problems. The results show a serious problem, as only one of the seven methods performed consistently better than the pair of old methods, three performed on par, two performed very badly, and the worst one performed barely better than a random search. We also give several suggestions that could help to resolve the presented issues.

Anglický abstrakt

Benchmarking constitutes a cornerstone in the analysis and development of computational methods. Especially in the field of evolutionary computation, where theoretical analysis of the algorithms is almost impossible, benchmarking is at the center of attention. In this text, we show that some of the frequently used benchmark functions that have their respective optima in the center of the feasible set pose a critical problem for the analysis of evolutionary computation methods. We carry out the analysis of seven recent methods, published in respected journals, which contain a center-bias operator that lets them find these optima with ease. This makes their comparison with other methods (that do not have a center-bias) meaningless on such types of problems. We perform a computational comparison of these methods with two of the oldest methods in evolutionary computation on shifted problems and on more advanced benchmark problems. The results show a serious problem, as only one of the seven methods performed consistently better than the pair of old methods, three performed on par, two performed very badly, and the worst one performed barely better than a random search. We also give several suggestions that could help to resolve the presented issues.

Klíčová slova anglicky

Evolutionary computation; Metaheuristics; Benchmarking; Zero-bias

Vydáno

12.12.2022

ISSN

2522-5839

Číslo

4

Strany od–do

1238–1245

Počet stran

8

BIBTEX


@article{BUT179505,
  author="Jakub {Kůdela},
  title="A critical problem in benchmarking and analysis of evolutionary computation methods",
  year="2022",
  number="4",
  month="December",
  pages="1238--1245",
  issn="2522-5839"
}