Artificial Intelligence (AI) has made a huge impact on our everyday lives. As a dominant branch of AI since the 1990s, Machine Learning (ML) has been applied to a wide range of scenarios, including image recognition, speech recognition, fraud detection, recommendation systems, time series prediction and self-driving cars. Deep learning, backed up by Deep Neural Networks (DNNs), is a major subfield of machine learning. DNNs are good at approximating smooth functions, i.e., learning a mapping from inputs to outputs, which is also known as the predictive or supervised learning approach. Sometimes, one is not interested in a specific predictive task, but rather in finding interesting patterns in the data. In this case, a descriptive or unsupervised learning approach is needed, and the task can be formalized as density estimation. Deep probabilistic models have gained popularity for density estimation because they maintain a good balance between expressivity and tractability, whereas classical probabilistic models face an inherent trade-off.

Deep neural networks and deep probabilistic models are both deep models in the sense that they are composed of multiple layers of computation units. They are essentially computation graphs and consequently, it is hard for humans to understand the underlying decision logic behind their behavior. Despite the representational and predictive power deep models have demonstrated in many complex problems, their opaqueness is a common reason for concern. In this thesis, we provide insights into deep models using high-level interpretations and explanations of why particular decisions are made.

Explanations that contradict our intuitions or prior knowledge on the underlying domain can expose a potential concern, which may imply some desiderata of ML systems are not met. For example, a deep model may obtain high predictive accuracy by exploiting a spurious correlation in the dataset, which can lead to a lack of robustness, or unfairness if the spurious correlation is linked to a protected attribute. Built on the framework of Explanatory Interactive Machine Learning (XIL), we propose to interactively improve deep models based on the explanations we get. This way, we put users in the training loop and take user feedback on explanations as additional training signals. As an effect, the model can learn the rules that align with our intuitions or prior knowledge.