BLIND SOURCE SEPARATION
INDEPENDENT COMPONENT ANALYSIS
If this was not permanently under construction
, it would be useless...
What is this page about?
This page is an attempt to organize some of the material that I make
available regarding source separation and ICA.
This is not very much of tutorial value and you need prior exposition
to source separation ideas to figure out what I am talking about
below. An introduction to the statistical principles at work in
ICA/BSS is given in the tutorial paper listed below.
I have written a tutorial paper Blind signal separation:
statistical principles, for the Proceedings of IEEE which explains
the statistical principles behind source separation and ICA. Reprints
are available, formatted for A4 paper
or for US
Source separation and equivariance
There is an underlying multiplicative structure to the source
separation problem for the simple reason that the source separation
model is a transformation model: the observations are
obtained via multiplication of the source signals by the
unknown mixing matrix.
It is very rewarding to explore the consequences of this simple
fact. It leads in particular to the notion of `serial updating' by
following the relative gradient by which efficient adaptive
algorithms can be derived. A summary of these ideas can be found in
approach to source separation published in the proceedings of
NOLTA'95 and in Performance
and implementation of invariant source separation algorithms
published in the proceedings of ISCAS'96.
Note: the idea of `relative gradient' has been independently
introduced by Pr. Amari who
defines a `natural gradient' (based on the Riemannian structure of the
probability model) which, after some simplification is identical to
our relative gradient (based on the group structure) in the case of
ICA. David MacKay
also arrived at a similar idea by what he calls a `covariant'
If you are really worried about the differences between natural
gradient and relative gradient, have a look at this short paper
from the proceedings of SSAP'98.
The notion of `estimating function' is helpful to unify many
approaches to the source separation problem: maximum likelihood,
infomax, contrast optimization, cumulant matching... This recent
conference paper about Estimating
equations for source separation, published in the proceedings of
ICASSP '97 summarizes a good part of these ideas.
An efficient batch algorithm: JADE
For off-line ICA, we have developed with Antoine Souloumiac an
algorithm based on the (joint) diagonalization of cumulant matrices.
`Good' statistical performance is achieved by involving all the
cumulants of order 2 and 4 while a fast optimization is obtained by
the device of joint diagonalization.
page about joint diagonalization.
JADE has been successfully applied to the processing of real data sets,
such as found in mobile telephony and in airport radar as well as to
bio-medical signals (ECG, EEG, multi-electrode neural recordings).
These experiments will be reported here in the (near?) future.
What we think is the strongest point of JADE for applications of ICA
is that it works off-the-shelf (no parameter tuning). Actually, we
advocate using the code provided below as a plug-in replacement for
PCA (whenever one is willing to investigate if such a replacement is
appropriate). The weakest point of the current implementation is that
the number of sources (but not of sensors) is limited in practice (by
the available memory) to something like 40 or 50 depending on your
The Jade algorithm is described or discussed in these papers.
- The first journal paper describing JADE is Blind beamforming
for non-Gaussian signals. This is a reprint from IEE Proc-F., a
journal with a strong focus on array processing. This paper
includes some performance analysis (inclusing the noisy case) and
comparisons to more classic approches to signal separation than ICA.
Readers unfamiliar with beamforming and the processing of complex
signals may prefer to read the paper published in Neural
- The conference paper: On the
performance of orthogonal algorithms (reprint from EUSIPCO'94)
also adresses the issue of asymptotic performance analysis. It
elaborates on the performance of algorithms which, like JADE and
many others, are based on pre-whitening followed by contrast
optimization. This is a rather technical paper.
- The more recent paper High-order
contrasts for independent component analysis, published in
`Neural Computation' 11(1):157--192, 1999, gives another try at
explaining why JADE and related approaches are statistically and
comptutationally attractive. The paper considers only the
noise-free real case, but includes more explanations about JADE as
well as comparisons to gradient-based methods.
The JADE algorithm was originally developed to process complex
signals, motivated by applications to digital communications. I now
make available another implementation which is tuned to process more
efficiently real-valued signals.
- Implementation in Matlab/octave for the ICA of real-valued data.
- Here is an implementation of JADE for real-valued
data (latest release: version 1.8. May 2005).
- You may want to test the code on this toy demo calling program.
- This stand-alone program
reads an ascii file of input signals, runs jade on it, and produces an ascii file of
separated components. It can be called from the shell if octave (the free Matlab clone) is
installed on your system. No need for programming at all, just read the
header of the file...
- Implementation in Matlab/octave for complex-valued signals.
You can download the JADE algorithm for
complex-valued signals as a Matlab function (new, faster, cleaner
version: nov. 97).
To run a Matlab demo of it, you can use this calling program.
- Implementation in python for real-valued signals.
This is a port of the matlab/octave code to Python+NumPy by GabriŽl J.L. Beckers.
The first release is here.
but you will find more details at the
dedicated page of GabriŽl.
- Implementation in C for real-valued signals.
You can download an implementation of JADE in C
for real-valued signals.
- Matlab MEX-files for real-valued signals.
available from the same archive as above, i.e. here.
- Implementation in R for real-valued signals.
JADE has been ported to R by Klaus Nordhausen. The package can be
found on CRAN (Comprehensive
R archive network) .
- Other implementations.
Have you ported JADE to some other language? Please, let me know.
- Joint diagonalization only
Some code for joint
diagonalization in matlab/octave.
Adaptive algorithms: relative gradient algorithms
For adaptive source separation, we have developed with Beate Laheld a
class of equivariant algorithms. This means that their
performance is independent of the mixing matrix. They are obtained as
stochastic relative gradient algorithms.
- Code. By the way, this algorithm works pretty well...
Check it out! Get a Matlab demo of
it. Beware: this is a demo with complex-valued signals, of the type
encountered in digital communications.
Multi-dimensional independent component analysis.
Performing ICA on ECG signals with the JADE algorithm, I realized that
an interesting extension of the notion of independent component
analysis would be to consider an analysis into linear components that
would be `as independent as possible' as in ICA, but would be `living'
in subspaces of dimension greater than 1. This could be called `MICA'
for Multi-dimensional Independent Component Analysis. This is
explained in an ICASSP paper and is illustrated therein as well as on
My travels on the matrix manifold
Can you find the straight square ? Can you find it blindly ?
Return to the top (I mean: top page).
Note concerning IEEE publications: This material is presented to
ensure timely dissemination of scholarly and technical work. Copyright
and all rights therein are retained by authors or by other copyright
holders. All persons copying this information are expected to adhere
to the terms and constraints invoked by each author's copyright. In
most cases, these works may not be reposted without the explicit
permission of the copyright holder. (see IEEE