BLIND SOURCE SEPARATION
and
INDEPENDENT COMPONENT ANALYSIS

If this was not permanently under construction , it would be useless...


Breaking news (2013)

In March 2013, the European Spatial Agency released the new refrence map of the Cosmic Microwave Background (CMB), a product of the Planck collaboration.

The above movie does not really explain that the CMB map is obtained by linearly combining the full-sky maps observed by Planck in nine frequency channels from 30 GHz to 857 GHz, thusly:

Yes, this is a component separation problem! The Planck component separation effort is summarized in this paper.

The Planck CMB map is obtained by a blind component separation method dubbed SMICA (for Spectral Matching ICA) which I hand-taylored for Planck. The main ideas are described here. Some limited details are in the Planck paper cited above.

For some background (!) about signal processing for the CMB, you may have a look at this introductory paper: Precision cosmology with the cosmic microwave background ("IEEE Signal Processing Magazine", vol.27, no.1, pp.55,66, Jan. 2010), which does not focus on component separation but rather on the fun of working on the sphere...


About this page

This page is an attempt to organize some of the material that I make available regarding source separation and ICA.

This is not very much of tutorial value and you need prior exposition to source separation ideas to figure out what I am talking about below. An introduction to the statistical principles at work in ICA/BSS is given in the tutorial paper listed below.

Tutorial paper

I have written a tutorial paper Blind signal separation: statistical principles, for the Proceedings of IEEE which explains the statistical principles behind source separation and ICA. Reprints are available, formatted for A4 paper or for US letter paper.

Source separation and equivariance

There is an underlying multiplicative structure to the source separation problem for the simple reason that the source separation model is a transformation model: the observations are obtained via multiplication of the source signals by the unknown mixing matrix.

It is very rewarding to explore the consequences of this simple fact. It leads in particular to the notion of `serial updating' by following the relative gradient by which efficient adaptive algorithms can be derived. A summary of these ideas can be found in The invariant approach to source separation published in the proceedings of NOLTA'95 and in Performance and implementation of invariant source separation algorithms published in the proceedings of ISCAS'96.

Note: the idea of `relative gradient' has been independently introduced by Pr. Amari who defines a `natural gradient' (based on the Riemannian structure of the probability model) which, after some simplification is identical to our relative gradient (based on the group structure) in the case of ICA. David MacKay also arrived at a similar idea by what he calls a `covariant' approach. If you are really worried about the differences between natural gradient and relative gradient, have a look at this short paper from the proceedings of SSAP'98.

The notion of `estimating function' is helpful to unify many approaches to the source separation problem: maximum likelihood, infomax, contrast optimization, cumulant matching... This recent conference paper about Estimating equations for source separation, published in the proceedings of ICASSP '97 summarizes a good part of these ideas.

An efficient batch algorithm: JADE

For off-line ICA, we have developed with Antoine Souloumiac an algorithm based on the (joint) diagonalization of cumulant matrices. `Good' statistical performance is achieved by involving all the cumulants of order 2 and 4 while a fast optimization is obtained by the device of joint diagonalization.

See this page about joint diagonalization.

JADE has been successfully applied to the processing of real data sets, such as found in mobile telephony and in airport radar as well as to bio-medical signals (ECG, EEG, multi-electrode neural recordings).

What we think is the strongest point of JADE for applications of ICA is that it works off-the-shelf (no parameter tuning). Actually, we advocate using the code provided below as a plug-in replacement for PCA (whenever one is willing to investigate if such a replacement is appropriate). The weakest point of the current implementation is that the number of sources (but not of sensors) is limited in practice (by the available memory) to something like 40 or 50 depending on your computer.

Adaptive algorithms: relative gradient algorithms

For adaptive source separation, we have developed with Beate Laheld a class of equivariant algorithms. This means that their performance is independent of the mixing matrix. They are obtained as stochastic relative gradient algorithms.

Multi-dimensional independent component analysis.

Performing ICA on ECG signals with the JADE algorithm, I realized that an interesting extension of the notion of independent component analysis would be to consider an analysis into linear components that would be `as independent as possible' as in ICA, but would be `living' in subspaces of dimension greater than 1. This could be called `MICA' for Multi-dimensional Independent Component Analysis. This is explained in an ICASSP paper and is illustrated therein as well as on this page.

My travels on the matrix manifold

Can you find the straight square ? Can you find it blindly ?


Return to the top (I mean: top page).
Note concerning IEEE publications: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder. (see IEEE copyright policies)