Description
Sequential parameter optimization can be described as a tuning algorithm with the following properties:
-
(i) Use the available budget (e.g., simulator runs, number of function evaluations) sequentially, i.e., use information from the exploration of the search space to guide the search by building one or several meta models. Choose new design points based on predictions from the meta model(s). Rene the meta model(s)) stepwise to improve knowledge about the search space.
-
(ii) Try to cope with noise by improving condence. Guarantee comparable confidence for search points.
-
(iii) Collect information to learn from this tuning process, e.g., apply explorative data analysis.
-
(iv) Provide mechanisms both for interactive and automated tuning.
SPOT was developed over the last years by Thomas Bartz-Beielstein, Christian Lasarczyk, and Mike Preuss. The main purpose of SPOT is to determine improved parameter settings for optimization algorithms to analyze and understand their performance.
SPOT was successfully applied to numerous optimization algorithms, especially in the field of evolutionary computation, i.e., evolution strategies, particle swarm optimization, algorithmic chemistries etc. in the following domains:
-
machine engineering: design of mold temperature control
-
aerospace industry: airfoil design optimization
-
simulation and optimization: elevator group control
-
technical thermodynamics: non sharp separation
-
economy: agri-environmental policy-switchings
-
algorithm engineering: graph drawing
-
statistics: selection under uncertainty (optimal computational budget allocation) for PSO
-
evolution strategies: threshold selection and step-size adaptation
-
genetic chromodynamics
-
computational intelligence: algorithmic chemistry
-
particle swarm optimization: analysis and application
-
numerics: comparison and analysis of classical and modern optimization algorithms
-
vehicle routing and door-assignment problems
-
bioinformatics
-
storm water management
-
differential and integral equations
-
time series analysis
An R version of this toolbox for interactive and automatic optimization of algorithms can be downloaded from CRAN.
People
Thomas Bartz-Beielstein, Christian Lasarczyk, and Mike Preuss
Publications
2012
Bartz-Beielstein, Thomas; Friese, Martina; Naujoks, Boris; Zaefferer, Martin
SPOT Applied to Non-Stochastic Optimization Problems---An Experimental Study Konferenzbeitrag
In: Rodriguez, Katya; Blum, Christian (Hrsg.): GECCO 2012 Late breaking abstracts workshop, S. 645–646, ACM, Philadelphia, Pennsylvania, USA, 2012, ISBN: 978-1-4503-1178-6.
@inproceedings{Bart12d,
title = {SPOT Applied to Non-Stochastic Optimization Problems---An Experimental Study},
author = {Thomas Bartz-Beielstein and Martina Friese and Boris Naujoks and Martin Zaefferer},
editor = {Katya Rodriguez and Christian Blum},
isbn = {978-1-4503-1178-6},
year = {2012},
date = {2012-01-01},
booktitle = {GECCO 2012 Late breaking abstracts workshop},
pages = {645--646},
publisher = {ACM},
address = {Philadelphia, Pennsylvania, USA},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Bartz-Beielstein, Thomas
Spot Seven Sonstige
Presentation---ESF Workshop Rome, 2012.
@misc{Bart12h,
title = {Spot Seven},
author = {Thomas Bartz-Beielstein},
year = {2012},
date = {2012-01-01},
howpublished = {Presentation---ESF Workshop Rome},
keywords = {},
pubstate = {published},
tppubtype = {misc}
}
2011
Bartz-Beielstein, Thomas; Zaefferer, Martin
SPOT Package Vignette Forschungsbericht
Cologne University of Applied Sciences 2011.
@techreport{Bart11m,
title = {SPOT Package Vignette},
author = {Thomas Bartz-Beielstein and Martin Zaefferer},
url = {http://cran.r-project.org/web/packages/SPOT/vignettes/SPOT.pdf},
year = {2011},
date = {2011-01-01},
institution = {Cologne University of Applied Sciences},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2010
Bartz-Beielstein, Thomas
Automatic and Interactive Tuning of Algorithms---The Sequential Parameter Optimization Toolbox (Invited talk) Sonstige
Leiden University NL, 2010.
@misc{Bart10u,
title = {Automatic and Interactive Tuning of Algorithms---The Sequential Parameter Optimization Toolbox (Invited talk)},
author = {Thomas Bartz-Beielstein},
year = {2010},
date = {2010-01-01},
howpublished = {Leiden University NL},
keywords = {},
pubstate = {published},
tppubtype = {misc}
}
Bartz-Beielstein, Thomas
SPOT: An R Package For Automatic and Interactive Tuning of Optimization Algorithms by Sequential Parameter Optimization Forschungsbericht
Research Center CIOP (Computational Intelligence, Optimization andData Mining) Cologne University of Applied Science, Faculty of Computer Scienceand Engineering Science, Nr. 05/10, 2010, ISSN: 2191-365X, (Comments: Related software can be downloaded from http://cran.r-project.org/web/packages/SPOT/index.html).
@techreport{Bart10eb,
title = {SPOT: An R Package For Automatic and Interactive Tuning of Optimization Algorithms by Sequential Parameter Optimization},
author = {Thomas Bartz-Beielstein},
url = {http://arxiv.org/abs/1006.4645},
issn = {2191-365X},
year = {2010},
date = {2010-01-01},
number = {05/10},
address = {Cologne University of Applied Science, Faculty of Computer Scienceand Engineering Science},
institution = {Research Center CIOP (Computational Intelligence, Optimization andData Mining)},
note = {Comments: Related software can be downloaded from http://cran.r-project.org/web/packages/SPOT/index.html},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
Bartz-Beielstein, Thomas; Lasarczyk, Christian; Preuss, Mike
The Sequential Parameter Optimization Toolbox Buchkapitel mit eigenem Titel
In: Bartz-Beielstein, Thomas; Chiarandini, Marco; Paquete, Luis; Preuss, Mike (Hrsg.): Experimental Methods for the Analysis of Optimization Algorithms, S. 337–360, Springer, Berlin, Heidelberg, New York, 2010.
@incollection{Bart09f,
title = {The Sequential Parameter Optimization Toolbox},
author = {Thomas Bartz-Beielstein and Christian Lasarczyk and Mike Preuss},
editor = {Thomas Bartz-Beielstein and Marco Chiarandini and Luis Paquete and Mike Preuss},
year = {2010},
date = {2010-01-01},
booktitle = {Experimental Methods for the Analysis of Optimization Algorithms},
pages = {337--360},
publisher = {Springer},
address = {Berlin, Heidelberg, New York},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
Bartz-Beielstein, Thomas
Tuning and Statistical Analysis of Optimization Algorithms - The Sequential Parameter Optimization Toolbox Unveröffentlicht
2010, (Invited Talk: The 2010 International Workshop on Nature Inspired Computation and Applications (IWNICA)).
@unpublished{Bart10s,
title = {Tuning and Statistical Analysis of Optimization Algorithms - The Sequential Parameter Optimization Toolbox},
author = {Thomas Bartz-Beielstein},
year = {2010},
date = {2010-01-01},
note = {Invited Talk: The 2010 International Workshop on Nature Inspired Computation and Applications (IWNICA)},
keywords = {},
pubstate = {published},
tppubtype = {unpublished}
}
Bartz-Beielstein, Thomas
Writing Interfaces for the Sequential Parameter Optimization ToolboxSPOT Forschungsbericht
Cologne University of Applied Sciences Cologne University of Applied Science, Faculty of Computer Science and Engineering Science, Nr. 07/10, 2010, ISSN: 2191-365X.
@techreport{Bart10nb,
title = {Writing Interfaces for the Sequential Parameter Optimization ToolboxSPOT},
author = {Thomas Bartz-Beielstein},
url = {http://maanvs03.gm.fh-koeln.de/webpub/CIOPReports.d/Bart10n.d/Bart10n.pdf},
issn = {2191-365X},
year = {2010},
date = {2010-01-01},
number = {07/10},
address = {Cologne University of Applied Science, Faculty of Computer Science and Engineering Science},
institution = {Cologne University of Applied Sciences},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
Bartz-Beielstein, Thomas
Writing Interfaces for the Sequential Parameter Optimization Toolbox SPOT Forschungsbericht
Cologne University of Applied Sciences Cologne University of Applied Science, Faculty of Computer Science and Engineering Science, Nr. 07/10, 2010, ISSN: 2191-365X.
@techreport{Bart10n,
title = {Writing Interfaces for the Sequential Parameter Optimization Toolbox SPOT},
author = {Thomas Bartz-Beielstein},
url = {http://maanvs03.gm.fh-koeln.de/webpub/CIOPReports.d/Bart10n.d/Bart10n.pdf},
issn = {2191-365X},
year = {2010},
date = {2010-01-01},
number = {07/10},
address = {Cologne University of Applied Science, Faculty of Computer Science and Engineering Science},
institution = {Cologne University of Applied Sciences},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
Bartz-Beielstein, Thomas
Performing Meta Experiments Using the Sequential Parameter Optimization Toolbox SPOT Forschungsbericht
Cologne University of Applied Sciences Cologne University of Applied Science, Faculty of Computer Science and Engineering Science, Nr. 08/10, 2010, ISSN: 2191-365X.
@techreport{Bart10o,
title = {Performing Meta Experiments Using the Sequential Parameter Optimization Toolbox SPOT},
author = {Thomas Bartz-Beielstein},
url = {http://maanvs03.gm.fh-koeln.de/webpub/CIOPReports.d/Bart10o.d/Bart10o.pdf},
issn = {2191-365X},
year = {2010},
date = {2010-01-01},
number = {08/10},
address = {Cologne University of Applied Science, Faculty of Computer Science and Engineering Science},
institution = {Cologne University of Applied Sciences},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
Bartz-Beielstein, Thomas; Flasch, Oliver; Koch, Patrick; Konen, Wolfgang
SPOT: A Toolbox for Interactive and Automatic Tuning in the R Environment Konferenzbeitrag
In: Hoffmann, Frank; Hüllermeier, Eyke (Hrsg.): Proceedings 20. Workshop Computational Intelligence, S. 264–273, Universitätsverlag Karlsruhe, 2010.
@inproceedings{Bart10p,
title = {SPOT: A Toolbox for Interactive and Automatic Tuning in the R Environment},
author = {Thomas Bartz-Beielstein and Oliver Flasch and Patrick Koch and Wolfgang Konen},
editor = {Frank Hoffmann and Eyke Hüllermeier},
year = {2010},
date = {2010-01-01},
booktitle = {Proceedings 20. Workshop Computational Intelligence},
pages = {264--273},
publisher = {Universitätsverlag Karlsruhe},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2008
Bartz-Beielstein, Thomas; Lasarczyk, Christian; Preuss, Mike
Sequential Parameter Optimization Toolbox Forschungsbericht
Universität Dortmund 2008.
@techreport{Bart06f,
title = {Sequential Parameter Optimization Toolbox},
author = {Thomas Bartz-Beielstein and Christian Lasarczyk and Mike Preuss},
url = {http://sfbci.uni-dortmund.de/Publications/Reference/Downloads/25608.pdf},
year = {2008},
date = {2008-01-01},
institution = {Universität Dortmund},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
Bartz-Beielstein, Thomas; Lasarczyk, Christian; Preuss, Mike
SPOT - Sequential Parameter Optimization Toolbox (Documentation) Forschungsbericht
Research Center CIOP (Computational Intelligence, Optimization andData Mining) Cologne University of Applied Science, Faculty of Computer Scienceand Engineering Science, Nr. 01/08, 2008, ISSN: 2191-365X.
@techreport{Bart08ib,
title = {SPOT - Sequential Parameter Optimization Toolbox (Documentation)},
author = {Thomas Bartz-Beielstein and Christian Lasarczyk and Mike Preuss},
url = {http://maanvs03.gm.fh-koeln.de/webpub/CIOPReports.d/Bart08i.d/spotdoc.pdf},
issn = {2191-365X},
year = {2008},
date = {2008-01-01},
number = {01/08},
address = {Cologne University of Applied Science, Faculty of Computer Scienceand Engineering Science},
institution = {Research Center CIOP (Computational Intelligence, Optimization andData Mining)},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2006
Bartz-Beielstein, Thomas; Preuss, Mike
Sequential parameter optimization (SPO) and the role of tuning in experimental analysis Konferenzbeitrag
In: Workshop On Empirical Methods for the Analysis of Algorithms (EMAA). Held in conjunction with the International Conference on Parallel Problem Solving From Nature (PPSN IX), S. 5–6, 2006, (Invited talk).
@inproceedings{BaPreu06b,
title = {Sequential parameter optimization (SPO) and the role of tuning in experimental analysis},
author = {Thomas Bartz-Beielstein and Mike Preuss},
year = {2006},
date = {2006-01-01},
booktitle = {Workshop On Empirical Methods for the Analysis of Algorithms (EMAA). Held in conjunction with the International Conference on Parallel Problem Solving From Nature (PPSN IX)},
pages = {5--6},
note = {Invited talk},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2005
Bartz-Beielstein, Thomas; Lasarczyk, Christian; Preuss, Mike
Sequential Parameter Optimization Konferenzbeitrag
In: McKay, B.; others, (Hrsg.): Proceedings 2005 Congress on Evolutionary Computation (CEC'05), Edinburgh, Scotland, S. 773–780, IEEE Press, Piscataway NJ, 2005.
@inproceedings{BLP05,
title = {Sequential Parameter Optimization},
author = {Thomas Bartz-Beielstein and Christian Lasarczyk and Mike Preuss},
editor = {B.~McKay and others},
year = {2005},
date = {2005-01-01},
booktitle = {Proceedings 2005 Congress on Evolutionary Computation (CEC'05), Edinburgh, Scotland},
volume = {1},
pages = {773--780},
publisher = {IEEE Press},
address = {Piscataway NJ},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
