Abstract
Deep neural networks (DNN) have boosted artificial intelligence and achieved significant progress across various fields and tasks such as computer vision and natural language processing. However, it is difficult to explain the decisions of DNNs due to the lack of transparency, which also makes it risky to trust DNNs. Furthermore, without an in-depth understanding of DNNs, it is challenging to debug and improve the DNN models. To this end, explainable AI aims at explaining and interpreting the decisions of DNN models in a way that can be understood by humans. One way to tackle the interpretability issue is to develop more transparent models, such as some relatively shallow machine learning models (support vector machines and decision trees), which is unrealistic for DNN models that are intrinsically opaque. Another branch of explainable AI is to develop explanation methods. Different from designing models with more transparency, explanation methods directly tackle the interpretability issue of opaque models by analyzing the contribution of the input to a particular prediction. Furthermore, they allow to employ non-linear design patterns in neural networks which have proven to perform well rather than attempting to compromise between prediction and explanation. A well-performing explanation method is capable of finding more related features/evidence for the model decisions. Thus, with explanation methods, we can check the reliability of DNN models and analyze the reasons for wrong predictions made by the model. In this work, we first study the explanation methods for recurrent neural network architectures based on two well-performing explanation methods for convolutional neural networks. We further derive explanation methods for multi-modal models such as the image captioning models. We show that explanation methods can disentangle the contributions from multiple inputs. We then explore the use-cases of explanation results to debug and improve DNN models. Specifically, we start from analyzing the reasons for object hallucination problems of image captioning models and effectively alleviate this issue with an explanation-inference fine-tuning strategy. Besides, we effectively improve the performance of few-shot classification models by designing an explanation-guided training strategy, especially under cross-domain settings. With extensive experiments, we demonstrate that explanation methods can not only provide plausible and intuitive explanations in the inference phase but can be used to debug and improve DNN models during the training phase as well.