Bayesian Optimization Book

This is a (draft) monograph on Bayesian optimization, which will be published in early 2022 by Cambridge University Press.

The book aims to provide a self-contained and comprehensive introduction to Bayesian optimization, starting “from scratch” and carefully developing all the key ideas along the way. The intended audience is graduate students and researchers in machine learning, statistics, and related fields. However, I also hope that practitioners and researchers from more distant fields will find some utility here.

The book is divided into three main parts, covering:

• theoretical and practical aspects of Gaussian process modeling,
• the Bayesian approach to sequential decision making, and
• the realization of practical and effective optimization policies.

A few additional topics are also covered:

• an overview of theoretical convergence results,
• a survey of notable extensions,
• a comprehensive history of Bayesian optimization, and
• an annotated bibliography of applications.

The book is in the final stages of preparation. I am making the draft available for initial commentary before publication. Once published, the book will remain freely available on this website.

Please note that the draft is subject to changes.

Table of Contents

1. Introduction
2. Gaussian Processes
3. Modeling with Gaussian Processes
4. Model Assessment, Selection, and Averaging
5. Decision Theory for Optimization
6. Utility Functions for Optimization
7. Common Bayesian Optimization Policies
8. Computing Policies with Gaussian Processes
9. Implementation
10. Theoretical Analysis
11. Extensions and Related Settings
12. A Brief History of Bayesian Optimization

Download

You may download the manuscript below. The 8”x10” version shares the same page size as what will appear in print and is best for online viewing. US letter and A4 versions are also provided for ease of printing.

• Download 8"x10"
• Download US Letter
• Download A4

Feedback

I welcome feedback on the manuscript! Please feel free to file an issue on the associated GitHub repository.

Cite

@book{garnett_bayesoptbook_2022,
  author    = {Garnett, Roman},
  title     = {{Bayesian Optimization}},
  year      = {2022},
  publisher = {Cambridge University Press},
  note      = {in preparation}
}