Confidence Bound Minimization for Bayesian optimization with Student's-t Processes

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Bayesian optimization seeks the global optimum of a black-box, objective function f (x), in the fewest possible iterations. Recent work applied knowledge of the true value of the optimum to the Gaussian Process probabilistic model typically used in Bayesian optimization. This, together with a new acquisition function called Confidence Bound Minimization, resulted in a Gaussian probabilistic posterior in which the predictions were no greater than the known maximum (and no less than for minimum). Our novel work applies Confidence Bound Minimization to Bayesian optimization with Student's-t Processes, a probabilistic alternative which addresses known weaknesses in Gaussian Processes - outliers' probability and the calculation of posterior covariance. The new model is applied to the problem of hyperparameter tuning for an XGBoost classifier. Experiments show superior regret minimization and predictive accuracy, versus the popular Expected Improvement acquisition function. Combining Confidence Bound Minimization with a transformed Student's-t Process probabilistic model and known optima produces superior training regret minimization and posterior predictions for the Six-Hump Camel(2D) and Levy(4D) benchmark problems, which do not fall below true minima.
Original languageEnglish
Title of host publicationAPPIS 2020: Proceedings of the 3rd International Conference on Applications of Intelligent Systems
EditorsNicolai Petkov, Nicola Strisciuglio
PublisherAssociation for Computing Machinery
Pages1-5
ISBN (Print)978-1-4503-7630-3
DOIs
Publication statusPublished - 7 Jan 2020
EventAPPIS 2020: 3rd International Conference on Applications of Intelligent Systems - Las Palmas de Gran Canaria, Spain
Duration: 7 Jan 20209 Jan 2020

Conference

ConferenceAPPIS 2020: 3rd International Conference on Applications of Intelligent Systems
CountrySpain
CityLas Palmas de Gran Canaria
Period7/01/209/01/20

Fingerprint

Dive into the research topics of 'Confidence Bound Minimization for Bayesian optimization with Student's-t Processes'. Together they form a unique fingerprint.

Cite this