Context Learning disentangled generative factors in an unsupervised way has gathered much attention lately since it is of interest in many domains, such as medical imaging. Most approaches look for factors that capture distinct, noticeable and semantically meaningful variations in one dataset (e.g., presence of hat or glasses in CelebA). Authors usually propose well adapted regularizations, which may promote, for instance, ”uncorrelatedness” (e.g., FactorVAE) or ”informativeness” (e.g., InfoGAN). In this project, we focus on a related but different problem, that has been named Contrastive Analysis (CA). We wish to discover in an unsupervised way what is added or modified on a target dataset compared to a control (or background) dataset, as well as what is common between the two domains. For example, in medical imaging, one would like to discover the salient variations characterizing a pathology that are only present in a population of patients (tumors or glasses in the figure below) and not in a population of healthy controls. Both the target (patients) and the background (healthy) datasets are supposed to share uninteresting (healthy) variations.
Two examples of datasets for Contrastive Analysis. First dataset: Brain MRI images with/without tumors. Top: MRI images of healthy brains (control dataset). Bottom: MRI images of brains with tumor (target dataset). Second dataset: CelebA dataset. Top: control dataset with regular faces (no smile, no glasses). Bottom: target dataset that contains smiling faces with glasses.
Goal The goal is to identify and separate the generative factors common to both populations from the ones distinctive (i.e., specific) only of the target dataset
Contributions Lately, we have proposed three new CA methods based on:
Results Let \(X={x_i}\) and \(Y={y_j}\) be the background (or control) and target datasets of images repsectively. Both \({x_i}\) and \({y_j}\) are assumed to be i.i.d. from two different and unknown distributions (\(P(x)\) and \(P(y)\)) that depend on a pair of latent variables \((c, s)\). Here, \(s\) is assumed to capture the salient generative factors proper only to \(Y\) whereas \(c\) should describe the common generative factors between \(X\) and \(Y\). At inference, each one of the proposed CA method can estimate the common \(c_t\) and salient \(s_t\) factors specific to a test image \(t\). We can thus test the performance of the algorithm by evalating its reconstruction quality and by swapping the salient factors between a background and target image, as shown in the figure below (each row presents a different result).
Image reconstruction and swap with the CelebA with accessories dataset.
References
2024
MIDL
SepVAE: a contrastive VAE to separate pathological patterns from healthy ones
Robin Louiset, Edouard Duchesnay, Antoine Grigis, Benoit Dufumier, and Pietro Gori
In Medical Imaging with Deep Learning (MIDL), 2024
Contrastive Analysis VAEs (CA-VAEs) are a family of Variational auto-encoders (VAEs) that aims at separating the common factors of variation between a background dataset (BG) (i.e., healthy subjects) and a target dataset (TG) (i.e., patients) from the ones that only exist in the target dataset. To do so, these methods separate the latent space into a set of salient features (i.e., proper to the target dataset) and a set of common features (i.e., exist in both datasets). Currently, all models fail to prevent the sharing of information between latent spaces effectively and to capture all salient factors of variation. To this end, we introduce two crucial regularization losses: a disentangling term between common and salient representations and a classification term between background and target samples in the salient space. We show a better performance than previous CA-VAEs methods on three medical applications and a natural images dataset (CelebA).
AISTATS
Double InfoGAN for Contrastive Analysis
Florence Carton, Robin Louiset, and Pietro Gori
In International Conference on Artificial Intelligence and Statistics (AISTATS), 2024
Contrastive Analysis (CA) deals with the discovery of what is common and what is distinctive of a target domain compared to a background one. This is of great interest in many applications, such as medical imaging. Current state-of-the-art (SOTA) methods are latent variable models based on VAE (CAVAEs). However, they all either ignore important constraints or they don’t enforce fundamental assumptions. This may lead to suboptimal solutions where distinctive factors are mistaken for common ones (or viceversa). Furthermore, the generated images have a rather poor quality, typical of VAEs, decreasing their interpretability and usefulness. Here, we propose Double InfoGAN, the first GAN based method for CA that leverages the high-quality synthesis of GAN and the separation power of InfoGAN. Experimental results on four visual datasets, from simple synthetic examples to complex medical images, show that the proposed method outperforms SOTA CA-VAEs in terms of latent separation and image quality. Datasets and code are available online1.
ICLR
Separating common from salient patterns with Contrastive Representation Learning
Robin Louiset, Edouard Duchesnay, Antoine Grigis, and Pietro Gori
In International Conference on Learning Representations (ICLR), 2024
Contrastive Analysis is a sub-field of Representation Learning that aims at separating common factors of variation between two datasets, a background (i.e., healthy subjects) and a target (i.e., diseased subjects), from the salient factors of variation, only present in the target dataset. Despite their relevance, current models based on Variational Auto-Encoders have shown poor performance in learning semantically-expressive representations. On the other hand, Contrastive Representation Learning has shown tremendous performance leaps in various applications (classification, clustering, etc.). In this work, we propose to leverage the ability of Contrastive Learning to learn semantically expressive representations well adapted for Contrastive Analysis. We reformulate it under the lens of the InfoMax Principle and identify two Mutual Information terms to maximize and one to minimize. We decompose the first two terms into an Alignment and a Uniformity term, as commonly done in Contrastive Learning. Then, we motivate a novel Mutual Information minimization strategy to prevent information leakage between common and salient distributions. We validate our method, called SepCLR, on three visual datasets and three medical datasets, specifically conceived to assess the pattern separation capability in Contrastive Analysis.
