About me:
I am a PhD candidate at TU Munich. I currently work on the integration of image and tabular data into explainable deep learning models for medical image analysis. Previously, I was a researcher in the biomedical image analysis group at VRVis and the Laboratory for Artificial Intelligence in Medical Imaging at LMU. I completed my master’s degree in Biomedical Computing at TU Munich.
E-mail: tom_nuno.wolf@tum.de
Research interests:
- Medical Image Analysis
- Explainable AI
- Deep Learning on Heterogeneous Data
- Bias-Mitigation for Neural Networks
- Self-Supervised Learning
Publications
Li, Yitong; Wolf, Tom Nuno; Pölsterl, Sebastian; Yakushev, Igor; Hedderich, Dennis M.; Wachinger, Christian From Barlow Twins to Triplet Training: Differentiating Dementia with Limited Data Conference Medical Imaging with Deep Learning (MIDL), 2024. @conference{Li2024, |
Wolf, Tom Nuno; Bongratz, Fabian; Rickmann, Anne-Marie; Pölsterl, Sebastian; Wachinger, Christian Keep the Faith: Faithful Explanations in Convolutional Neural Networks for Case-Based Reasoning Proceedings Article Forthcoming In: AAAI Conference on Artificial Intelligence, 2024, Forthcoming. @inproceedings{nokey,Explaining predictions of black-box neural networks is crucial when applied to decision-critical tasks. Thus, attribution maps are commonly used to identify important image regions, despite prior work showing that humans prefer explanations based on similar examples. To this end, ProtoPNet learns a set of class-representative feature vectors (prototypes) for case-based reasoning. During inference, similarities of latent features to prototypes are linearly classified to form predictions and attribution maps are provided to explain the similarity. In this work, we evaluate whether architectures for case-based reasoning fulfill established axioms required for faithful explanations using the example of ProtoPNet. We show that such architectures allow the extraction of faithful explanations. However, we prove that the attribution maps used to explain the similarities violate the axioms. We propose a new procedure to extract explanations for trained ProtoPNets, named ProtoPFaith. Conceptually, these explanations are Shapley values, calculated on the similarity scores of each prototype. They allow to faithfully answer which prototypes are present in an unseen image and quantify each pixel's contribution to that presence, thereby complying with all axioms. The theoretical violations of ProtoPNet manifest in our experiments on three datasets (CUB-200-2011, Stanford Dogs, RSNA) and five architectures (ConvNet, ResNet, ResNet50, WideResNet50, ResNeXt50). Our experiments show a qualitative difference between the explanations given by ProtoPNet and ProtoPFaith. Additionally, we quantify the explanations with the Area Over the Perturbation Curve, on which ProtoPFaith outperforms ProtoPNet on all experiments by a factor >10**3. |
Wachinger, Christian; Wolf, Tom Nuno; Pölsterl, Sebastian Deep learning for the prediction of type 2 diabetes mellitus from neck-to-knee Dixon MRI in the UK biobank Journal Article In: Heliyon, vol. 9, no. 11, 2023. @article{nokey, |
Wolf, Tom Nuno; Pölsterl, Sebastian; Wachinger, Christian Don't PANIC: Prototypical Additive Neural Network for Interpretable Classification of Alzheimer's Disease Proceedings Article In: Information Processing in Medical Imaging (IPMI), 2023. @inproceedings{nokey,Alzheimer's disease (AD) has a complex and multifactorial etiology, which requires integrating information about neuroanatomy, genetics, and cerebrospinal fluid biomarkers for accurate diagnosis. Hence, recent deep learning approaches combined image and tabular information to improve diagnostic performance. However, the black-box nature of such neural networks is still a barrier for clinical applications, in which understanding the decision of a heterogeneous model is integral. We propose PANIC, a prototypical additive neural network for interpretable AD classification that integrates 3D image and tabular data. It is interpretable by design and, thus, avoids the need for post-hoc explanations that try to approximate the decision of a network. Our results demonstrate that PANIC achieves state-of-the-art performance in AD classification, while directly providing local and global explanations. Finally, we show that PANIC extracts biologically meaningful signatures of AD, and satisfies a set of desirable desiderata for trustworthy machine learning. Our implementation is available at https://github.com/ai-med/PANIC. |
Rickmann, Anne-Marie; Xu, Murong; Wolf, Tom Nuno; Kovalenko, Oksana; Wachinger, Christian HALOS: Hallucination-free Organ Segmentation after Organ Resection Surgery Proceedings Article In: Information Processing in Medical Imaging (IPMI), 2023. @inproceedings{nokey,The wide range of research in deep learning-based medical image segmentation pushed the boundaries in a multitude of applications. A clinically relevant problem that received less attention is the handling of scans with irregular anatomy, e.g., after organ resection. State-of-the-art segmentation models often lead to organ hallucinations, i.e., false-positive predictions of organs, which cannot be alleviated by oversampling or post-processing. Motivated by the increasing need to develop robust deep learning models, we propose HALOS for abdominal organ segmentation in MR images that handles cases after organ resection surgery. To this end, we combine missing organ classification and multi-organ segmentation tasks into a multi-task model, yielding a classification-assisted segmentation pipeline. The segmentation network learns to incorporate knowledge about organ existence via feature fusion modules. Extensive experiments on a small labeled test set and large-scale UK Biobank data demonstrate the effectiveness of our approach in terms of higher segmentation Dice scores and near-to-zero false positive prediction rate. |
Wolf, Tom Nuno; Pölsterl, Sebastian; Wachinger, Christian DAFT: A Universal Module to Interweave Tabular Data and 3D Images in CNNs Journal Article In: NeuroImage, pp. 119505, 2022. @article{WOLF2022119505,Prior work on Alzheimer’s Disease (AD) has demonstrated that convolutional neural networks (CNNs) can leverage the high-dimensional image information for diagnosing patients. Beside such data-driven approaches, many established biomarkers exist and are typically represented as tabular data, such as demographics, genetic alterations, or laboratory measurements from cerebrospinal fluid. However, little research has focused on the effective integration of tabular data into existing CNN architectures to improve patient diagnosis. We introduce the Dynamic Affine Feature Map Transform (DAFT), a general-purpose module for CNNs that incites or represses high-level concepts learned from a 3D image by conditioning feature maps of a convolutional layer on both a patient’s image and tabular clinical information. This is achieved by using an auxiliary neural network that outputs a scaling factor and offset to dynamically apply an affine transformation to the feature maps of a convolutional layer. In our experiments on AD diagnosis and time-to-dementia prediction, we show that the DAFT is highly effective in combining 3D image and tabular information by achieving a mean balanced accuracy of 0.622 for diagnosis, and mean c-index of 0.748 for time-to-dementia prediction, thus outperforming all baseline methods. Finally, our extensive ablation study and empirical experiments reveal that the performance improvement due to the DAFT is robust with respect to many design choices. |
Pölsterl, Sebastian; Wolf, Tom Nuno; Wachinger, Christian Combining 3D Image and Tabular Data via the Dynamic Affine Feature Map Transform Conference Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2021. @conference{Poelsterl2021-daft,Prior work on diagnosing Alzheimer’s disease from magnetic resonance images of the brain established that convolutional neural networks (CNNs) can leverage the high-dimensional image information for classifying patients. However, little research focused on how these models can utilize the usually low-dimensional tabular information, such as patient demographics or laboratory measurements. We introduce the Dynamic Affine Feature Map Transform (DAFT), a general-purpose module for CNNs that dynamically rescales and shifts the feature maps of a convolutional layer, conditional on a patient’s tabular clinical information. We show that DAFT is highly effective in combining 3D image and tabular information for diagnosis and time-to-dementia prediction, where it outperforms competing CNNs with a mean balanced accuracy of 0.622 and mean c-index of 0.748, respectively. Our extensive ablation study provides valuable insights into the architectural properties of DAFT. Our implementation is available at https://github.com/ai-med/DAFT. |
You must be logged in to post a comment.