Description
Book SynopsisThis book covers the essential theory and implementation of popular Bayesian optimization techniques in an intuitive and well-illustrated manner. The techniques covered in this book will enable you to better tune the hyperparemeters of your machine learning models and learn sample-efficient approaches to global optimization.
The book begins by introducing different Bayesian Optimization (BO) techniques, covering both commonly used tools and advanced topics. It follows a develop from scratch method using Python, and gradually builds up to more advanced libraries such as BoTorch, an open-source project introduced by Facebook recently. Along the way, you''ll see practical implementations of this important discipline along with thorough coverage and straightforward explanations of essential theories. This book intends to bridge the gap between researchers and practitioners, providing both with a comprehensive, easy-to-digest, and useful reference guide.
After completing
Table of Contents
● Chapter 1: Bayesian Optimization in a Nutshell
o Chapter goal: introducing Bayesian Optimization workflow and key conceptso Estimate number of pages: 30o Sub topics:▪ What and why of Bayesian Optimization ▪ Key components in Bayesian Optimization process▪ Common Bayesian Optimization applications
● Chapter 2: Bayesian Optimization in Hyperparameter Tuningo Chapter goal: Showcase using Bayesian Optimization for hyperparameter tuning in training better ML modelso Estimate number of pages: 35o Sub topics:▪ ML workflow▪ Common hyperparameter tuning techniques▪ Advantage of Bayesian Optimization in tuning hyperparameters for ML models through practical examples
● Chapter 3 : Gaussian Processo Chapter goal: Introduce Gaussian process and its role in Bayesian Optimization workflowo Estimate number of pages: 30o Sub topics:▪ Gaussian process breakdown▪ Theory illustration on Gaussian process ▪ Coding Gaussian process as surrogate model in Bayesian Optimization
● Chapter 4 : Common Acquisition Functiono Chapter goal: Introduce popular acquisition functions including EI, PI and otherso Estimate number of pages: 35o Sub topics:▪ The role of acquisition function in Bayesian Optimization ▪ Theoretical basics for each common AF▪ Coding examples
● Chapter 5: Advanced Acquisition Functiono Chapter goal: Introduce advanced acquisition functions including KG and PE and parallel variantso Estimate number of pages: 35o Sub topics:▪ Theoretical basics for advanced AF▪ Coding examples ● Chapter 6 : Introducing BoTorcho Chapter goal: Introduce the recent GPU based package for running Bayesian Optimization o Estimate number of pages: 40o Sub topics:▪ Introduction of the package and key components▪ Starting examples▪ Advanced examples
● Chapter 7 : Case studyo Chapter goal: Demonstrate full working examples using Bayesian Optimization and BoTorcho Estimate number of pages: 30o Sub topics:▪ Two full coding examples TBD
● Chapter 8 : Exotic Bayesian Optimization Problemso Chapter goal: Introduce additional Bayesian Optimization variants such as adding constraints and getting noisy observationso Estimate number of pages: 30o Sub topics:▪ Constrained Bayesian Optimization ▪ Parallel Bayesian Optimization ▪ BO with noisy observations▪ Look ahead Bayesian Optimization
