2011 - 2012
- Jeudi 14 Juin [Amphi Hermite]
- Jeudi 10 Mai [Amphi Hermite] Attention, changement d'horaire
(exceptionnellement)
- [15h30] Orateur : Meili Baragatti (MISTEA,
SupAgro)
- Titre : Parallel Tempering with
Equi-Energy Moves, and Likelihood Free Parallel tempering
-
Résumé : We will consider the
common problem of generating samples from a target density $\pi$, in
particular using Population-based Monte Carlo markov Chain approaches. In conventional MCMC methods
(Metropolis-Hastings, Gibbs Sampler), a Markov process is built to
sample the target probability distribution. But in practice this process
can be easily trapped into a local mode from where it cannot escape in
reasonable time. Many
techniques have been proposed to address this waiting time problem,
including among others Parallel Tempering (PT) (see Geyer [1991] or
Geyer and Thompson [1995]), and the Equi-Energy Sampler (EES) (Kou et
al. [2006]). The EES is based on a population of chains which are
updated by local moves and global moves, also called Equi-Energy jumps.
The state space is partitioned into energy rings, and the current state
of a chain can jump to a past state of an adjacent chain that has an
energy level close to its level. This algorithm has been developed to
facilitate global moves between different chains, resulting in a good
exploration of the state space by the target chain. This method seems to be more
efficient than the classical PT algorithm. However the process obtained
is non Markovian, it is difficult to use in combination with a Gibbs
sampler and it necessitates increased storage. We will present an adaptation
of this EES that combines PT with the principle of swapping between
chains with same levels of energy. This adaptation, that we shall call
Parallel Tempering with Equi-Energy Moves (PTEEM), keeps the original
idea of the EES method while generating a Markovian process and
requiring less storage. Performances of the PTEEM algorithm will be
compared with those of the EES and of the standard PT algorithms in the
context of mixture models, and in a problem of identification of
transcription factor binding motifs.
In a second part of the
presentation, we will present an other Population-based Monte Carlo
markov Chain approach which can be use in an Approximate Bayesian
Computational (ABC) framework (or likelihood-free framework). ABC
methods have appeared in the past fifteen years as useful methods to
perform Bayesian analysis when the likelihood is analytically or
computationally intractable. Several approaches have been proposed:
Monte Carlo Markov Chains (MCMC) methods have been developed by
Marjoram et al. [2003] and by Bortot et l. [2007] for instance, and
sequential methods have been proposed among others by Sisson et al.
[2007], Beaumont et al. [2009] and Del Moral et al. [2012]. Until
recently, while ABC-MCMC methods were the reference, sequential ABC
methods have appeared to outperform them (see for example McKinley et
al. [2009] or Sisson et al. [2007]). We will present a new algorithm
combining population-based MCMC methods with ABC requirements, using an
analogy with the Parallel Tempering algorithm.
Joint work with A. Grimaud, D. Pommeret.
- Références :
- Baragatti M, Grimaud A, Pommeret D. Likelihood-Free Parallel
Tempering, Statistics and Computing, in press, 2012.
- Baragatti M, Grimaud A, Pommeret D. Parallel tempering with
Equi-Energy moves, Statistics and Computing, in press, 2012.
- [16h40] Orateur
: Kerrie
Mengersen (Queensland Univ. of Technology , Australia)
- Titre : Understanding
images: from inferential aims to models to algorithms
- Résumé : In
the excitement of working with algorithms, it is sometimes salutory to remind ourselves of their
purpose. In this presentation, we consider the analysis of image data and
try to match inferential aims, models and computational methods. We
describe and compare the approaches in the context of some real case
studies in agriculture and environmental monitoring.
- Jeudi 12 Avril [Amphi Darboux]
- [15h] Orateur : Pierre Jacob
(ENSAE) et Robin
Ryder (CEREMADE)
- Titre : Some
aspects of the Wang-Landau algorithm.
- Résumé : The
Wang-Landau algorithm is an adaptive MCMC algorithm which generates a
Markov chain designed to move efficiently in the state space, by
constantly penalizing already-visited regions. It hence falls into the
class of exploratory algorithms, especially when the chosen regions
correspond to different levels of density values. We
explore two novel aspects of the Wang-Landau algorithm. First, we show
that the algorithm reaches the so-called Flat Histogram criterion in
finite time, which ensures convergence properties. Second, we examine
the effect of using multiple chains, interacting through a common
component. That component essentially represents the history of
already-visited regions, computed on all the chains. We show
numerically the benefit of using parallel chains even if a single
processing unit is available, in terms of stabilization of
the schedule used in the adaptation process. If time permits, we shall
present an ongoing attempt to study theoretically the effect of
parallelization using Feynman-Kac semigroups.
- Références http://arxiv.org/abs/1110.4025
et http://arxiv.org/abs/1109.3829
- [16h] Orateur : Nick Whiteley ( Univ.
Bristol, UK)
- Titre :
A particle method for approximating principal eigen-functions and
related quantities
- Résumé :
Perron-Frobenius theory
treats the existence of a positive eigen-vector associated with the
principal eigen-value \lambda_{\star} of a non-negative matrix, say Q.
A simple method for approximating this eigen-vector involves computing
the iterate \lambda_{\star}^{-n}Q^{(n)}, for large n. In the more
general case that Q is a non-negative integral kernel, an extended
Perron-Frobenius theory applies, but it is typical that neither the
principal eigen-function nor the iterate \lambda_{\star}^{-n}Q^{(n)}
can be computed exactly. In this setting we introduce an interacting
particle algorithm which yields a numerical approximation of the
principal eigen-function and the associated twisted Markov kernel. Some
of its theoretical properties will be discussed and applications will
be outlined. In particular, the algorithm allows approximation of an
optimal importance sampling method for Markov chain rare event
estimation.
Joint work with Nikolas Kantas.
- Référence : http://arxiv.org/abs/1202.6678
- Jeudi 15 Mars [salle 314]
- Jeudi 9 Février
[Amphi Darboux]
- [15h] Orateur : A. Kebaier
(LAGA, Paris 13)
- Titre :
Central
limit theorem for the multi-level algorithm
- Résumé : This
paper deals with the problem of the multi-level Monte Carlo method, introduced by Giles
(Multilevel Monte Carlo path simulation Operations Research, 2008; 56:607-617)
as an extended method of the statistical Romberg one of Kebaier
(Romberg Extrapolation: A New Variance Reduction Method and Applications to
Option Pricing. Ann. Appl. Probab. 15 (2005), no. 4, 2681-2705). When
approximating the expected value of a function of a stochastic differential
equation solution, these methods improve efficiently the computational
complexity of standard Monte Carlo. In this work, we analyze the asymptotic error of this
algorithm and establish a central limit theorem based on a new stable
functional central limit
theorem on the error in the Euler scheme for a given level. This allows us to
obtain the optimal choice of the parameters method. Then,
we investigate the application of this method to the pricing of Asian options. In this setting, the
approximation relies on the discretization of the integral of the price
process over a time interval.
We also analyze the error process and prove a stable functional central limit theorem.
Finally, We use our result in order to optimize the choice of the parameters, which are
different from the ones in the Euler scheme. Numerical simulations were
processed.
- [16h15] Orateur : N. Chopin (CREST)
- Titre : SMC2: A
sequential Monte Carlo algorithm with particle Markov chain Monte Carlo
updates
-
Résumé : We consider
the generic
problem of performing sequential Bayesian
inference in a state-space model with observation process y, state
process x and fixed parameter theta. An idealized approach would be to
apply the iterated batch importance sampling (IBIS) algorithm of Chopin
(2002). This is a sequential Monte Carlo algorithm in the
theta-dimension, that samples values of theta, reweights iteratively
these values using the likelihood increments p(y_t|y_1:t-1, theta), and
rejuvenates the theta-particles through a resampling step and a MCMC
update step. In state-space models these likelihood increments are
intractable in most cases, but they may be unbiasedly estimated by a
particle filter in the x-dimension, for any fixed theta. This motivates
the SMC^2 algorithm proposed in this article: a sequential Monte Carlo
algorithm, defined in the theta-dimension, which propagates and
resamples many particle filters in the x-dimension. The filters in the
x-dimension are an example of the random weight particle filter as in
Fearnhead et al. (2010). On the other hand, the particle Markov chain
Monte Carlo (PMCMC) framework developed in Andrieu et al. (2010) allows
us to design appropriate MCMC rejuvenation steps. Thus, the
theta-particles target the correct posterior distribution at each
iteration t, despite the intractability of the likelihood increments.
We explore the applicability of our algorithm in both sequential and
non-sequential applications and consider various degrees of freedom, as
for example increasing dynamically the number of x-particles. We
contrast our approach to various competing methods, both conceptually
and empirically through a detailed simulation study, included here and
in a supplement, and based on particularly challenging examples.
Joint work with Pierre E. Jacob, and Omiros
Papaspiliopoulos.
-
- Jeudi 12 Janvier [Amphi
Darboux]
- [15h] Orateur : S.
Allassonnière (CMAP, Ecole Polytechnique)
- Titre : Anisotropic
Metropolis
Adjusted Langevin Algorithm: convergence and utility in
Stochastic EM algorithm
- Résumé : Sampling
a random variable in a high dimensional framework becomes a crucial
issue regarding the large range of applications which require such
samples, such as image analysis. The Langevin simulation method is well
adapted in this context. However, the use of an isotropic covariance
matrix usually prevents from a numerical convergence which suffers from
the low acceptation rate. In this work, we propose to adapt the
Langevin simulation taking into account the anisotropy of the
distribution to sample from. We prove that this new algorithm leads to
a uniform ergodic Markov chain. Moreover,
we apply this new Monte Carlo Markov Chain method into a stochastic
Expectation-Maximization algorithm in order to estimate some model
parameter by maximizing the likelihood. We prove that under mild
conditions, the estimated parameters converge almost surely and are
asymptotically Gaussian distributed. All this pipeline is numerically
tested on handwritten digits and some medical images for the deformable
template estimation.
- [16h15] Orateur
: A. Fulop
(Essec)
- Titre : Bayesian Learning of
Impacts of Self-Exciting Jumps in Returns and Volatility
- Résumé :
The
paper proposes a new class of continuous-time asset pricing models where negative jumps play a
crucial role. Whenever there is a negative jump in asset returns, it is
simultaneously passed on to diffusion variance and the jump
intensity, generating self-exciting co-jumps of prices and volatility and
jump clustering. To properly deal with parameter uncertainty and
in-sample over-fitting, a Bayesian learning approach combined with an
efficient particle filter is employed. It not only allows for
comparison of both nested and non-nested models, but also generates all
quantities necessary for sequential model analysis. Empirical
investigation using S&P 500 index returns shows that volatility jumps at the
same time as negative jumps in asset returns mainly through jumps
in diffusion volatility. We find substantial
evidence for jump clustering, in particular, after the recent financial crisis in
2008, even though parameters driving dynamics of the jump
intensity remain difficult to identify.
- Référence
: http://www.andrasfulop.com/home/Self_Exciting_Dec14.pdf?attredirects=0
- Jeudi 8 décembre
[salle
314]
- [15h] Orateur
: S. Le Corff
(LTCI)
- Titre : Block online EM for hidden Markov models
with application to parameter estimation in general state-spaces
-
Résumé : The EM algorithm has
been successfully applied for ML inference in HMM. However, when processing
large data sets or data streams, the EM algorithm might become
computationally very intensive and different online variants have been recently
proposed. Despite the encouraging first results, the convergence of
these algorithms remain an open question in the common framework of general
state-spaces. In this contribution, we propose a new online EM algorithm,
the block online EM algorithm. In this block online EM algorithm, the
M-step (and thus, the update of the parameter) occurs at some deterministic
times known in advance.The k-th (block) E-step consists in the computation
of a mean value of the sufficient statistics associated to the
observations in the k-th block. This algorithm relies on the ability to compute this
mean value sequentially, i.e. without storing all the observations. At the
end of the block, the parameter is updated based on a weighted average
of all the mean statistics (over the successive blocks). The convergence
properties of this algorithm are addressed for HMM models for which
expectations under the joint smoothing distribution can be computed explicitly (e.g.
finite state space or linear Gaussian model) and for a stochastic variant
based on Sequential Monte Carlo methods in the case of more general
models. The behavior of the proposed algorithms is numerically
evaluated
for different models.
- [16h15] Orateur : T. Lelièvre (CERMICS)
- Titre :
Generating efficiently metastable dynamics
- Résumé :
We will
present two methods to sample metastable trajectories of solutions to
some stochastic differential equations which are used in molecular
dynamics. Naive approaches are typically inefficient since transitions
between two given metastable states are very rare. The interest of such
sampling techniques is to obtain typical transition paths between two
metastable regions, to compute some reaction rates, to determine the
transition states, etc.
The first method is an adaptation of the Adaptive Multilevel Splitting
approach proposed by F. Cérou and A. Guyader in 2007. The
idea
is to select trajectories which go the most far in some given
direction. We have recently used this technique on simple test cases,
and most of the mathematical analysis remains to be done [1].
The second method (the parallel replica method) is an algorithm which
has been proposed by A. Voter in 1997. It is in particular used in
molecular dynamics simulations in material sciences. We have recently
analyzed mathematically this algorithm [2]. The quasi stationary
distribution plays a crucial role in the analysis.
[1] F. Cérou, A. Guyader, T. Lelièvre and D.
Pommier, A multiple replica approach
to simulate
reactive trajectories, Journal of Chemical Physics, 134, 054108, (2011).
[2] C. Le Bris, T. Lelièvre, M. Luskin and D. Perez, A mathematical formalization of the
parallel replica dynamics, http://hal.archives-ouvertes.fr/hal-00596161/fr/
.
- Mardi 15 Novembre
- Exceptionnellement, le
séminaire
est
remplacé par une journée
scientifique qui aura lieu à Telecom ParisTech. Voir site
web
- Jeudi 13 Octobre [Amphi Hermite]
- [15h] Orateur
: B. Miasojedow (LTCI,
Paris)
- Titre : Nonasymptotic bounds
on the estimation error of MCMC algorithms
- Résumé :
MCMC methods
are used not only to sample from posterior distributions but also
to
estimate
expectations. We derive bounds on the mean square error of
MCMC
estimators.
Our analysis is non-asymptotic. We first establish a general
result
valid
for essentially all ergodic Markov chains encountered in
Bayesian
computation
and a possibly unbounded target function f. Assuming
minorization
condition
we cut the trajectory by regeneration technique
into
iid blocks. In further analysis we use tools of statistical sequential
analysis
and
renewal theory, such as Wald's identities and Lorden's
theorem.
The
bound is sharp in the sense that the leading term is
equal
to
the well known aspymptotic results. Next,
we proceed to specific assumptions (drift and minorization conditions)
and
give
explicit computable bounds for geometrically and polynomially
ergodic
Markov
chains. In both cases final results are in terms
of
computable quantities which appear in the drift and minorization conditions. The
talk
is
based on a joint paper with K. Latuszynski and W. Niemiro.
- [16h15] : Pas de séminaire /
Réunion
du projet BigMC
- Jeudi 1er Septembre [salle 201]
- Orateur : X-L.
Meng (Univ. Harvard, USA)
- Titre : Statistical Inception for the MCMC Dream:
The kick is in the
residual (augmentation)!
- Résumé
: The development of MCMC algorithms via
data augmentation (DA) or
equivalently auxiliary variables has some resemblance to the theme plot of the recent Hollywood
hit Inception. We MCMC designers all share essentially the same ?3S?
dream, that is, to create algorithms that are simple, stable, and speedy.
Within that grand dream, however, we have created a rather complex web
of tools, with some of them producing very similar algorithms but for
unclear reasons, or others that were thought to be of different origins
but actually are layered when viewed from a suitable distance. These
include conditional augmentation, marginal augmentation, PX-DA,
partially non-centering parameterization, sandwiched algorithms,
interweaving strategies, ASIS, etc. It turns out that there is a simple
statistical insight that can unify essentially all these methods
conceptually, and it also provides practical guidelines for their DA
constructions. It is the simple concept of regression residuals, which
are constructed to be orthogonal to the regression functions. All
these methods in one form or another effectively build a residual
augmentation. Given a DA distribution f(T,A), where T is our targeted
variable (i.e., f(T) is our targeted distribution) and A is the
augmented variable, there are two broad classes of residuals
depending on whether we regress T on A or A on T. In this talk we will
demonstrate how methods like conditional augmentation and partially
non-centering parameterization build their residual augmentations by
regressing A on T, whereas methods such as marginal augmentation and
ASIS effectively use residual augmentations from regressing T on A. For
either class, the attempted orthogonality helps to reduce the
dependence among MCMC draws, and when the orthogonality leads to true
independence as occurring in some special cases, we reach the dream of
producing i.i.d. draws. (The talk is based on an upcoming discussion
article, especially its rejoinder, Yu and Meng (2011, JCGS) )
2010 - 2011
- Jeudi 9 Juin
[Salle 314]
- [15
h] Orateur :
H. F. Lopes (Univ. of Chicago, US)
- Titre : Parsimonious
Bayesian Factor Analysis when the Number of
Factors is Unknown
-
Résumé : We
introduce
a new and general set of identifiability conditions for factor models
which handles the ordering problem associated with current common
practice. In addition, the new class of parsimonious Bayesian factor
analysis leads to a factor loading matrix representation which is an
intuitive and easy to implement factor selection scheme. We argue that
the structuring the factor loadings matrix is in concordance with
recent trends in applied factor analysis. Our MCMC scheme for posterior
inference makes several improvements over the existing alternatives
while outlining various strategies for conditional posterior inference
in a factor selection scenario. Four applications, two based on
synthetic data and two based on well known real data, are introduced to
illustrate the applicability and generality of the new class of
parsimonious factor models, as well as to highlight features of the
proposed sampling schemes. (Joint
work with Sylvia Fruhwirth-Schnatter, Univ. of Linz - Austria).
- [16h15] Orateur
: A. Eberle (Univ.
Bonn,
Germany)
- Titre : Mixing times of Metropolis-adjusted
Langevin algorithms for
log-concave probability measures in high dimensions
-
Résumé :
The
Metropolis-adjusted Langevin algorithm (MALA) is a
Metropolis-Hastings algorithm for approximate sampling
from continuous distributions. We derive upper bounds for
the distance from equilibrium after a finite number of steps
for MALA with semi-implicit Euler proposals applied to
log-concave perturbations of Gaussian measures. For
sufficiently ``regular´´ perturbations, the
estimates are
dimension-independent in a sense to be specified.
- Jeudi 5 Mai [Salle
314]
- [15 h] Orateur
: M. Vihola (Univ.
Jyvaslyla, Finlande)
- Titre : Stability of adaptive random-walk
Metropolis algorithms
- Résumé : Most
results
on adaptive MCMC in the literature are based on assumptions that
require the adaptation process to be `stable' (in a certain sense).
Such stability can rarely be established, unless the process is
modified by introducing specific stabilisation structures. This talk
outlines recent stability and ergodicity results for adaptive MCMC
algorithms without such stabilisation. The key idea behind the results
is that the ergodic averages can converge even if the Markov kernels
gradually `lose' their ergodic properties.
- [16h15] Orateur:
B. Bouzy
(Univ. Descartes, Paris)
- Titre : Monte-Carlo Tree Search for the game of
Go
- Résumé :
This talk
describes Monte-Carlo Tree Search (MCTS) the successful technique now
used in many complex games such as the game of Go. In a first part, the
talk reminds the results achieved by old go programs until 2003, i.e.
medium on the human scale. This average result was firstly caused by
the game tree size that forbids global tree search. Building an
accurate evaluation functions for non terminal positions was the second
difficulty. These difficulties lead to erratic behaviour of go programs
until 2003. In a second part, the talk shows how the Monte-Carlo (MC)
approach has simplified the computer go between 2003 and 2006. To
evaluate a non terminal position, the MC method consists in launching
random games starting from this position, and playing them out until
terminal positions, easy to score, are reached. The MC evaluation of a
position is the mean of the scores encountered. MC evaluations cost a
lot of time, but they are robust, and their precision improves with the
time available. In a third part, the talk shows the success of MCTS
since 2006. MCTS cleverly integrates tree search and simulations. MCTS
progressively builds a tree whose nodes contain statistics about
simulations. While thinking time is available, MCTS repeatedly executes
four stages: the selection stage (browsing the tree from the root to a
leaf), the expansion stage (adding a new node), the simulation stage
(performing the simulation until the end of the game et getting the
outcome) and the update stage (updating node values). MCTS resulted in
a big improvement of Go programs. On 9x9 boards the best playing
programs reached human professional level. On 19x19 boards, the best
playing programs reached human strong amator level. In a last part, the
talk shows how MCTS can be refined. First, using domain-dependent
knowledge is advised in the selection and simulation stages.
Reinforcement learning or supervised learning can be used to
automatically learn this knowledge. Second, the selection stage can be
very efficient with the RAVE (Rapid Action Value Estimation) heuristic.
Parallelization is a third mandatory tool to demonstrate the power of
MCTS in go. Before conclusion, the talk shows the remaining obstacles
and future works.
- Jeudi 7 Avril
[Salle 314]
- [15 h] Orateur
: O. Papaspiliopoulos
(Univ. P. Fabra, Barcelone)
- Titre :
Bayesian inference of coefficients of
diffusion processes using data augmentation
-
Résumé :
We consider estimation of scalar functions which
determine the dynamics of diffusion processes. It is known that
nonparametric maximum likelihood is ill-posed in this context. We
adopt a probabilistic approach to regularize the problem by the
adoption of a prior distribution for the unknown functional. Our first
result is that a Bayesian Gaussian
conjugate analysis for the drift of one-dimensional non-linear
diffusions is
feasible given
high-frequency data. This is achieved by expressing the
log-likelihood
as a quadratic function of
the drift, with sufficient statistics given by the so-called
local time process and
the end points of the observed path. We show how to
implement such analysis using flexible and amenable to
efficient computations Gaussian Markov process priors. We provide a
simple estimator of the local time and a finite element method for the
numerical derivation of the posterior mean and precision, as well as
the
posterior simulation of the unknown functional. We
embed this technology in partially observed
situations and adopt a data augmentation approach whereby we
iteratively generate missing data paths and draws from the
(conditionally) Gaussian posterior distribution of the functional
given complete data. Our methodology is applied to estimate the drift
of models used in molecular dynamics and financial econometrics using
high and low frequency, real and simulated, data. We discuss
extensions to other partially observed schemes, connections to
other types of non/semi-parametric inference, and to estimation of
vector-valued functions.
- Joint work with
Pokern, Roberts and Stuart.
- [16h15] Orateur
:
P. Bui Quang (ONERA Palaiseau)
- Titre :
High-dimensional importance sampling: An analysis via
the Laplace method
- Résumé : Several
authors have underlined that importance sampling for Bayesian inference
often behaves poorly when the dimension of the parameter of interest is
large. We consider here the asymptotic variance of the importance
weights to quantify the impact of dimensionality on the inference. To
calculate this variance, we propose to use the Laplace method, which
allows us to approximate accurately multidimensional integrals. When
applied in a Bayesian framework, the approximation converges with
respect to the data size. We use this method to calculate the
asymptotic variance of the importance weights, and in particular to
analyze its dependence on the dimension.
- Travail en
collaboration avec C. Musso (ONERA Palaiseau) et F. Le Gland
(INRIA Rennes).
- Jeudi 3 Mars
[Salle 201]
- [15 h] Orateur :
O. Feron
(EDF)
- Titre :
Echantillonnage de champs gaussiens de grande dimension : le cas non
creux / non circulant
- Résumé :
Ces travaux
ont été menés par
François Orieux, Post-doc
à l'institut Pasteur,
Jean-François
Giovannelli,
professeur au laboratoire de l'intégration
du
matériau au système de
l'université de Bordeaux, et moi-même. Dans un premier temps, nous
proposons une nouvelle approche pour l'échantillonnage
de
champs gaussiens corrélés dans le cas
où les approches
classiques ne sont
pas utilisables : lorsque la dimension du problème est
très grande et lorsque la matrice inverse de covariance (ou matrice de
précision)
n'est pas creuse et n'est pas circulante. Cette approche est valide dans le
cas où une structure particulière de la matrice de
précision
est disponible. Cette structure apparaît dans la résolution de
problèmes inverses par des méthodes
d'estimation
bayésienne. L'algorithme
proposé trouve
une
application directe pour les méthodes d'inversion myopes
et/ou non
supervisées,
fondées sur des méthodes
d'échantillonnage de type
MCMC. L'efficacité
de cette approche est illustrée sur l'inversion non supervisée d'un
problème de super résolution d'images. Aujourd'hui, nous
étudions des extensions à cet algorithme dans
le but d'accélérer
les
algorithmes MCMC mis en oeuvre. Dans ce but, nous énoncerons les
différentes pistes envisagées et les
difficultés rencontrées
pour
assurer la convergence des algorithmes MCMC.
- [16h15] Orateur : O. Cappé
(LTCI)
- Titre
: L'algorithme EM en ligne et ses
possibles extensions Monte Carlo
- Résumé
: Dans cet exposé, je
décrirai le
principe de l'algorithme EM en ligne (ou adaptatif) qui a pour but
d'estimer les paramètres d'un modèle
Ã
données latentes (a priori à partir
d'observations
indépendantes) en incorporant les observations une
à une.
Je présenterai également une
façon
légèrement différente d'utiliser
l'algorithme
ayant pour but de déterminer l'estimateur du maximum de
vraisemblance correspondant à un ensemble
fixé
d'observations (ce qui en fait un concurrent direct de l'algorithme EM
classique). Après une discussion des
propriétés de
convergence de l'algorithme, je présenterai des travaux en
cours
sur des extensions de l'approche en particulier à des cas
où l'étape E de l'algorithme EM en ligne
nécessite
l'utilisation de simulations Monte Carlo.
- Réf. O.
Cappé and E.
Moulines. On-line Expectation-Maximization Algorithm for Latent Data
Models. J. Royal Statist. Soc. B, 71(3):593-613, 2009.
- Jeudi 3
Février
[Salle 314]
- [15h] Orateur :
F.
Perron (Univ. de Montréal, Canada)
- Titre :
Estimation de copules, une approche bayésienne
- Résumé :
La copule
associée à un couple aléatoire
continu (X,Y) est
une mesure de dépendance qui est invariante pour les
transformations monotones sur X et Y. Cette mesure a une forme
particulière dans le cas des valeurs extrêmes
qui fait
intervenir la fonction de dépendance de Pickands ( fonction convex
sur [0,1] avec conditions aux bornes ). On cherche Ã
estimer
la fonction de Pickands. Pour cela nous allons constuire des
modèles
bayésiens. Ces modèles seront
motivés par une
construction
géométrique, un lien avec la mesure spectrale et une utilisation des
polynômes. L`aspect computationnel va prendre une place très
importante. Nous allons discuter du comment faire les calculs en faisant appel
à des méthodes de simulation
basées sur les chaînes
de Markov.
Finalement, on traitera le problème standard où les variables ne sont pas
à valeurs extrêmes.
- [16h15] GdT : J. Rousseau
(Univ. Dauphine / CREST)
- Référence : Y. Pokern, O.
Papaspiliopoulos, G.O. Roberts ans A. Stuart. Nonparametric Bayesian
drift estimation for one-dimensional diffusion processes, submitted to the Ann.Statist. Available in pdf
- Jeudi 6 Janvier
[Salle 314]
- [15h] Orateur : J.L.
Foulley
(INRA)
- Titre :
Calcul
de la vraisemblance marginale par la
méthode des posteriors de puissance : rappels et
applications
- Résumé :
Cet exposé se place dans le cadre des
méthodes de calcul
de la
vraisemblance marginale après intégration des
paramètres par
rapport à leur densité a priori.
Après un bref
rappel des
principales méthodes standard (Importance Sampling, Newton
&
Raftery, Gelfand & Dey, Chib, « Bridge
sampling »
notamment), l’exposé se focalisera sur la
méthode des
posteriors
de puissance de température due à Friel &
Pettitt
(2008). Nous
rappellerons les fondements théoriques de la
méthode et
discuterons
de sa mise en œuvre calculatoire. Nous évoquerons
ses liens avec
l’approche du facteur de Bayes fractionnaire d’O Hagan
et avec le
critère DIC. Nous montrerons comment cette
méthode peut
être
implantée sous Winbugs. Nous illustrerons le propos par
deux
exemples : l’un relatif à l’analyse
de données
longitudinales par des modèles mixtes, et l’autre,
ayant trait
Ã
l’analyse de différenciation
génétique de
marqueurs moléculaires
SNP par des modèles bayésiens
hiérarchiques
simples.
- Référence :
Friel N, Pettitt AN (2008) Marginal likelihood estimation via power
posteriors, JRSS, B, 70, 589-607
- [16h15] Orateur
: G.
Celeux (INRIA, Futur)
- Titre :
Méthodes d'estimation pour le modèle des blocs
latents.
- Résumé :
Le
modèle des blocs latents est un modèle de
mélange proposé
par Govaert et
Nadif (2007) qui permet la classification conjointe des lignes et des
colonnes d'un tableau de données. Après sa
présentation, nous comparerons différentes
méthodes d'estimation
des
paramètres de ce modèle pour lequel
l'inférence pose
des difficultés.
Nous évoquerons une approximation variationnelle de l'algorithme EM (Govert et
Nadif 2007), une version stochastique de EM de type EM Ã
la
Gibbs et une approche bayésienne proposée par
Wyse et Friel (2010).
Pour finir
le problème du choix d'un nombre de blocs sera
évoqué et
ses particularités souligné.
- Govaert and Nadif (2007) Block Clustering with
Bernoulli miture models: Comparison
of
different
approaches. Computational Statistics and Data Analysis 52, 3233-3245.
- Wyse
and Friel (2010) Block clustering with collapsed latent block model. In revision for Statistics
and Computing
- Jeudi 2
Décembre
[Salle 314]
- Jeudi 18 Novembre
- [15h] Orateur : D. Woodard
(Cornell Univ., USA)
- Titre : Convergence rate of Markov chain methods
for genomic motif discovery
- Résumé
: We analyze the convergence rate of a popular
Gibbs sampling method used for statistical discovery of gene regulatory
binding motifs in DNA sequences. This sampler satisfies a very strong
form of ergodicity (uniform ergodicity). However, we show that, due to
multimodality of the posterior distribution, the rate of convergence
often decreases exponentially in the length of the DNA sequence.
Specifically, we show exponential or polynomial decay for several
cases, which support the conjecture that the decay is exponential if
and only if more than one true repeating pattern exists in the data.
Since there are typically multiple, even numerous, such patterns in
real data (the goal being to detect well-conserved and
frequently-occurring patterns), we argue that the Gibbs sampler rate of
convergence decays exponentially in practice.
This matches empirical results, which find such poor mixing of the
motif-discovery Gibbs sampler that it is used only for discovering
modes of the posterior distribution (candidate motifs). This is one of
the first examples of a Markov chain method that provably fails to
obtain samples from the posterior distribution of a statistical model
within polynomial time.
- [16h15] GdT :
C.
Schäfer (CREST, Paris)
- Titre : Adaptive Monte Carlo on
multivariate binary sampling spaces
- Résumé : A
Monte Carlo
algorithm is said to be adaptive if it can adjust
automatically its current proposal distribution using past simulations.
The choice of the parametric family that defines the set of proposal
distributions is critical for good performance. In this paper, we
discuss such parametric families for adaptive sampling on multivariate
binary spaces. A practical motivation for this problem is variable
selection in a linear regression context. We want to sample from a
Bayesian posterior distribution on the model space using Sequential
Monte Carlo for sampling and the Cross-Entropy method for optimization.
Raw versions of the algorithm are easily implemented using binary
vectors with independent components. For high-dimensional variable
selection problems, however, these straightforward proposals do not
yield satisfactory results. The key to efficient adaptive algorithms
are binary parametric families which take correlations into account,
analogously to the multivariate normal distribution on continuous
spaces.
- Article : arXiv
- Jeudi 7 Octobre
[Salle 314]
- [15h] Orateur
: A. Samson (Paris V)
- Titre :
Estimation pour des modeles mixtes définis
par
équations différentielles
stochastiques
via
l'algorithme SAEM et un filtre particulaire
- Résumé :
Les
données longitudinales biologiques peuvent
être
analysées par des modèles
mixtes
définis
par une équation différentielle
stochastique (EDS).
L'estimation par maximum de vraisemblance dans ces modeles est complexe, la vraisemblance
n'étant pas explicite. L'algorithme SAEM (Delyon et al, 1999) a
été proposé pour l'estimation
des modeles mixtes mais
n'est pas
adapté au cas des modeles définis par EDS.
Dans ce travail, nous
proposons de
coupler l'algorithme SAEM avec un
algorithme PMCMC (Andrieu et al, 2010), ce qui permet de
realiser de facon
efficace l'etape E de l'algorithme EM. Nous montrons la convergence de notre
algorithme vers le maximum de vraisemblance. Nous illustrons notre approche sur
deux exemples, un processus d'Ornstein-Ulhenbeck
et
un
modele à volatilité stochastique non homogène en temps
très utilisé pour modéliser des
courbes de
croissance.
- [16h15] Orateur
: G. Fort
(LTCI, CNRS)
- Groupe de Travail
: Approximation stochastique et Algorithmes MCMC adaptatifs
- Résumé :
Nous
présenterons quelques résultats
récents sur la
convergence d'échantillonneurs MCMC adaptatifs lorsque le
mécanisme d'adaptation se fait selon une dynamique
d'approximation stochastique. Nous verrons que la convergence du
processus d'adaptation n'est pas nécessairement requise,
ni
même sa stabilité. Nous illustrerons nos
propos en
considérant l'algorithme "adaptive Metropolis"
proposé
par Haario et al. (1999); et l'algorithme Equi-Energy
proposé
par Kou et al. (2006). Discussion à partir des
résultats énoncés dans
- M. Vihola (2010) On
the
Stability
and Ergodicity of an Adaptive Scaling Metropolis
Algorithm, Preprint [format pdf]
- E. Saksman and M. Vihola (2010) On the Ergodicity of the Adaptive
Metropolis Algorithm on Unbounded Domains , Ann.Appl. Probab. [format pdf]
- G. Fort, E. Moulines et P. Priouret (2010) Convergence of
adaptive MCMC algorithms: ergodicity and law of large numbers, Preprint
[format pdf]
