Kroese D. The Cross-Entropy Method. A Unified Approach...Machine Learning 2004

Textbook in PDF format

The cross-entropy (CE) method is one of the most significant developments in stochastic optimization and simulation in recent years. This book explains in detail how and why the CE method works. The CE method involves an iterative procedure where each iteration can be broken down into two phases: (a) generate a random data sample (trajectories, vectors, etc.) according to a specified mechanism; (b) update the parameters of the random mechanism based on this data in order to produce a ``better'' sample in the next iteration. The simplicity and versatility of the method is illustrated via a diverse collection of optimization and estimation problems.
The book is aimed at a broad audience of engineers, computer scientists, mathematicians, statisticians and in general anyone, theorist or practitioner, who is interested in fast simulation, including rare-event probability estimation, efficient combinatorial and continuous multi-extremal optimization, and machine learning algorithms.
Preliminaries
A Tutorial Introduction to the Cross-Entropy Method
Efficient Simulation via Cross-Entropy
Combinatorial Optimization via Cross-Entropy
Continuous Optimization and Modifications
Noisy Optimization with CE
Applications of CE to COPs
Applications of CE to Machine Learning
Appendix: Example Programs