  Although AI systems have been applied in various fields and achieved
impressive performance, their safety and reliability are still a big concern.
This is especially important for safety-critical tasks. One shared
characteristic of these critical tasks is their risk sensitivity, where small
mistakes can cause big consequences and even endanger life. There are several
factors that could be guidelines for the successful deployment of AI systems in
sensitive tasks: (i) failure detection and out-of-distribution (OOD) detection;
(ii) overfitting identification; (iii) uncertainty quantification for
predictions; (iv) robustness to data perturbations. These factors are also
challenges of current AI systems, which are major blocks for building safe and
reliable AI. Specifically, the current AI algorithms are unable to identify
common causes for failure detection. Furthermore, additional techniques are
required to quantify the quality of predictions. All these contribute to
inaccurate uncertainty quantification, which lowers trust in predictions. Hence
obtaining accurate model uncertainty quantification and its further improvement
are challenging. To address these issues, many techniques have been proposed,
such as regularization methods and learning strategies. As vision and language
are the most typical data type and have many open source benchmark datasets,
this thesis will focus on vision-language data processing for tasks like
classification, image captioning, and vision question answering. In this
thesis, we aim to build a safeguard by further developing current techniques to
ensure the accurate model uncertainty for safety-critical tasks.
