Introduction to the Special Issue on Explainable AI on Multimedia Computing

Machine learning, especially deep neural networks, and artificial intelligence (ML/AI), have contributed to the recent big progresses in a wide range of research and applications in computer vision, intelligence systems, and natural language processing tasks, etc. As machine learning and AI related technologies become more ubiquitous, researchers in the whole ML/AI community find that the need to trust these AI based systems with all manner of decision making is paramount. Despite the unprecedented practical success of AI in the field of multimedia, the inability to “explain” AI-based models' decisions in a human understandable way still limits their effectiveness. To explain why an AI model achieves a certain result is so far still extremely difficult, since it is complicated and often impossible to get insights into the internal workings of a model. Therefore, explainable artificial intelligence (XAI) has started to catch people's attention. Efforts in explainable AI not only attempt to open the “black box” of AI models, but also attempt to develop models that can provide intuitive explanations of the results for the users, and system designers, which can help to ameliorate the model transparency and effectiveness.

This special issue is a compilation of cutting-edge research on explainable AI on multimedia computing. First of all, biometric authentication has been used in a wide variety of multimedia applications as part of a process validating a user for access to a system, in the form of fingerprints, facial recognition, voice recognition, etc. The paper “xCos: An Explainable Cosine Metric for Face Verification Task” by Y.-S. Lin et al. focuses on developing a novel similarity metric, called explainable cosine , that comes with a learnable module that can be plugged into most of the face verification models to provide meaningful explanations. The paper “Learning to Fool the Speaker Recognition” by J. Li et al. studies the adversarial attack to speech-based systems and presents an effective method by leveraging a pretrained phoneme recognition model to optimize the speaker recognition attacker to obtain a trade-off between the attack success rate and the perceptual quality. The paper “Explainable AI: A Multispectral Palm Vein Identification System with New Augmentation Features” by Y.-Y. Chen et al. proposes a palm-vein identification system based on lightweight neural networks with explainability.

The remaining papers that form the special issue covers other key aspects of explainable AI to multimedia computing. The paper “Semantic Explanation for Networks using Feature Interac-tions” by B. Xia et al. considers the feature engineering not only from the perspective of individual features but also their interactions to enable a more derailed interpretation of the decisions made by neural networks. The paper “Leveraging Deep Statistics for Underwater Image Enhancement” by Y. Wang et al. develops an explainable framework based on neural networks to learning hierarchical statistical features related to color cast and contrast degradation, and to leverage them for underwater image enhancement. The paper “An Explainable Deep Learning Ensemble Model for Robust Diagnosis of Diabetic Retinopathy Grading” by M. Shorfuzzaman et al. proposes an ex-plainable deep learning ensemble framework using the transfer learning concept where weights from different models are combined into a single model to extract salient features and the effective-ness is demonstrated via the application of diagnosis of diabetic retinopathy grading. The paper “Black-Box Diagnosis and Calibration on GAN Intra-Mode Collapse: A Pilot Study” by Z. Wu et al. explores diagnosing GAN intra-mode collapse in a novel black-box setting through the proposed systematic approach. The paper “Precise No-Reference Image Quality Evaluation Based on Distor-tion Identification” by C. Yan et al. studies no-reference image quality assessment to address the challenge of lacking of knowledge about the distortion in image.

All these papers show the diversity of methods and algorithms for developing advanced models and techniques to explainable AI on multimedia computing. We hope that these novel papers will inspire more research efforts on a variety of explainable AI topics including novel perspectives of network interpretability, new neural network architectures and protocols to strengthen the network trustworthiness, and critical applications related to emerging multimedia technologies.