Gaussian Process Models for Robust Regression, Classification, and Reinforcement Learning.
Technische Universität, Darmstadt
[Ph.D. Thesis], (2006)
Available under Creative Commons Attribution Non-commercial No Derivatives, 2.5.
Download (3MB) | Preview
|Item Type:||Ph.D. Thesis|
|Title:||Gaussian Process Models for Robust Regression, Classification, and Reinforcement Learning|
Gaussian process models constitute a class of probabilistic statistical models in which a Gaussian process (GP) is used to describe the Bayesian a priori uncertainty about a latent function. After a brief introduction of Bayesian analysis, Chapter 3 describes the general construction of GP models with the conjugate model for regression as a special case (OHagan 1978). Furthermore, it will be discussed how GP can be interpreted as priors over functions and what beliefs are implicitly represented by this. The conceptual clearness of the Bayesian approach is often in contrast with the practical difficulties that result from its analytically intractable computations. Therefore approximation techniques are of central importance for applied Bayesian analysis. Chapter 4 describes Laplace's method, the Expectation Propagation approximation, and Markov chain Monte Carlo sampling for approximate inference in GP models. The most common and successful application of GP models is in regression problems where the noise is assumed to be homoscedastic and distributed according to a normal distribution. In practical data analysis this assumption is often inappropriate and inference is sensitive to the occurrence of more extreme errors (so called outliers). Chapter 5 proposes several variants of GP models for robust regression and describes how Bayesian inference can be approximated in each. Experiments on several data sets are presented in which the proposed models are compared with respect to their predictive performance and practical applicability. Gaussian process priors can also be used to define flexible, probabilistic classification models. Again, exact Bayesian inference is analytically intractable and various approximation techniques have been proposed, but no clear picture has yet emerged, as to when and why which algorithm should be preferred. Chapter 6 presents a detailed examination of the model, focusing on the question which approximation technique is most appropriate by investigating the structure of the posterior distribution. An experimental study is presented which corroborates the theoretical insights. Reinforcement learning deals with the problem of how an agent can optimise its behaviour in a sequential decision process such that its utility over time is maximised. Chapter 7 addresses applications of GPs for model-based reinforcement learning in continuous domains. If the environment's response to the agent's actions can be predicted using GP regression models, probabilistic planning and an approximate policy iteration algorithm can be implemented. A core concept in reinforcement learning is the value function, which describes the long-term strategic value of a state. Using GP models we are able to solve an approximate continuous equivalent of the Bellman equations, and it will be shown how this can be used to estimate value functions.
|Place of Publication:||Darmstadt|
|Classification DDC:||300 Sozialwissenschaften > 310 Statistik|
|Divisions:||20 Fachbereich Informatik|
|Date Deposited:||17 Oct 2008 09:22|
|Last Modified:||10 Dec 2012 10:07|
|Referees:||Rasmussen, PhD Carl Edward and Schiele, Prof. Dr. Bernt|
|Advisors:||Hofmann, Prof. Dr. Thomas|
|Refereed:||21 March 2006|