Vision-language models inherit and amplify societal biases at every stage of their lifecycle: in the representations they learn, the behaviors they exhibit at inference time, and even in how they respond to requests to forget data. This talk presents three complementary perspectives on tackling bias in CLIP and multimodal LLM systems. First, we look inside the model: SITH performs data-free, training-free interpretability of the vision transformer of CLIP, decomposing attention-head weights via singular value decomposition and mapping them to human-interpretable concepts. This enables targeted interventions to suppress bias and spurious correlations without any data or retraining. Second, we act on what we find: SEM performs post-hoc debiasing of vision-language models in a sparse, disentangled embedding space, modulating bias-related neurons while preserving task-relevant semantics, improving fairness in retrieval and zero-shot classification without retraining. Third, we focus on fair unlearning: when deletion requests are demographically skewed, naive unlearning can introduce new bias. We present FairGet, a VQA benchmark for fairness-aware unlearning built on 4,000 fictitious identities, and FAUN, a method combining activation steering with bias-aware PCA to forget data while preserving demographic parity.

Abstract: --

AI models for medical imaging can exhibit systematic differences in performance across demographic groups, for example by sex or ethnicity. In healthcare, where AI is increasingly being integrated into clinical decision-making, such disparities risk perpetuating or exacerbating existing health inequalities. Although a wide range of generic bias mitigation methods have been proposed, their effectiveness in medical imaging has been mixed. Medical imaging datasets are often relatively small compared with those used in computer vision, and models must contend with complex domain shifts arising from differences in scanners, acquisition protocols, clinical settings, and patient populations. In this talk, I will describe a research programme that takes a different approach: rather than attempting to mitigate bias without first understanding its causes, we seek to identify what the model is learning that leads to disparities and use this understanding to develop targeted mitigation strategies. I will show how this process can now be largely automated, enabling effective identification and mitigation of sources of bias across domain shifts. This targeted approach can substantially outperform generic bias mitigation methods and can even mitigate bias without the commonly observed accuracy-fairness trade-off. The talk will highlight the importance of understanding model behaviour, not simply measuring its outcomes, as a route towards developing more reliable and equitable medical imaging AI.