Bone X-Ray analysis with explainable AI and multimodal self-supervised learning

Englebert, Alexandre
(2024)

Files

thesis24_11_12.pdf
  • Open Access
  • Adobe PDF
  • 31.6 MB
Download

Details

Authors
  • Englebert, AlexandreUCLouvain
    author
Supervisors
De Vleeschouwer , Christophe
 ;
Cornu, Olivier
Abstract
The rapid development of artificial intelligence (AI) in medicine promises to transform diagnosis, treatment, research, and medical education. AI-powered systems have demonstrated remarkable capabilities in image recognition, natural language processing, and predictive analytics, improving the accuracy and efficiency of various applications. However, deploying AI models in medicine faces significant challenges, including the need for more model transparency and high-quality annotated datasets. Explainability: To address the “black-box” nature of AI models, we developed explainability methods to improve transparency and trust in AI diagnoses. We made Poly-CAM, a method for generating high-resolution class activation maps (CAMs) for Convolutional Neural Networks (CNNs) without relying on gradient backpropagation. We demonstrated Poly-CAM on bone radiographs to identify potential biases in model predictions. Additionally, we introduced Transformer Input Sampling (TIS), enhancing explainability for vision transformers by sampling tokens. Self-Supervision and Vision-Language Models: Given the scarcity of annotated medical data, we decided to explore self-supervised learning techniques to reduce the need for manual annotation while maintaining robust model performance. Due to the lack of available datasets, we also created a dataset from raw bone radiographic images and French reports at Cliniques Universitaires Saint Luc, which enabled us to explore self-supervised multimodal techniques. We demonstrated the effectiveness of self-supervised techniques and pseudo-labels for enhancing downstream tasks. Additionally, we preprocessed the dataset to produce training data for future vision-language models aimed at automating medical report generation and visual question answering (VQA). Overall, this research contributes to a more transparent and reliable healthcare system where AI supports medical professionals and opens doors for future research.
Affiliations

Citations

  • APA
  • Chicago
  • FWB

Englebert, A. (2024). Bone X-Ray analysis with explainable AI and multimodal self-supervised learning. https://hdl.handle.net/2078.5/235546