The rise in prominence of Machine Learning (ML) has been undoubtedly driven by kernel machines and the revival of deep neural networks. The cornerstone of these machines is the pre-preprocessing of the data with a (cascade of) nonlinear transformation(s), which embeds them into a feature/latent space where data-processing techniques can be easily carried out. Apart from developing nonlinear models that outperform linear ones in supervised classification tasks, the nonlinear embedding is inevitable in numerous application areas. This is the case when dealing with discrete structured spaces, such as in chemioinformatics and bioinformatics where molecules are represented by strings and graphs, and in signal processing where data are time series with irregular sampling. In all these settings, since the Euclidean metric is inappropriate, data need to be embedded into a suitable space to carry out Euclidean-defined techniques. While the data embedding is essential in ML, the inverse embedding is of great importance in many pattern recognition and data mining problems. Indeed, one often needs to extract patterns in the data space, not in the implicit feature one, such as for instance the estimation of the barycenter of a set of graph data needs to be done in the graph space, not in the embedding space. The challenging inverse embedding is the preimage problem. Estimating the preimage is a hard ill-posed optimization problem, due to the nonlinear, often implicit, embedding. Moreover, it is even harder when dealing with discrete structured spaces, such as in all the aforementioned domains (time series, strings and graph data). This project aims to unlock the potential of ML for unsupervised learning, by addressing the fundamental issue of the preimage in all its forms in kernel machines and deep learning. To this end, four clearly identified attack points will be investigated: - Establish a novel class of ML algorithms that do not suffer from the curse of the preimage, by investigating a paradigm shift in nonlinear ML thanks to closed-form solutions using probabilistic models or joint optimization strategies. - Set up pattern recognition with compact representation and metric learning for time series, and more generally temporal data, by examining recurrent and hierarchical neural networks, temporal kernel machines, and Siamese networks for temporal metric learning, in the light of the preimage problem. - Explore pattern recognition on discrete structured spaces, namely string and graph data, with an emphasis on the task of synthesizing a molecule from a set of molecules represented by graphs, as in bioinformatics and chemioinformatics (e.g. drug design). - Associate the preimage problem in kernel machines with two classes of neural networks, autoencoders and the emerging generative adversarial networks (GANs), in order to provide deeper insights of their underlying functioning, and stimulate the development and exchange of ideas to design new architectures. While the major contributions will essentially be theoretical in ML, the project will tackle several applicative domains concerning other scientific communities, with an interest in the signal processing, bioinformatics and chemioinformatics. Moreover, since the preimage problem is pervasive in the multi-disciplinary ML field, this project also seeks to broaden the scope of application to new areas, such as multimodal/heterogeneous data fusion. Each of these working directions will investigate both kernel machines and deep learning. By providing a cross-fertilization of ideas, this project is expected to bridge the gap in unsupervised learning between these classes of ML algorithms. In order to address all these challenging points, it will create a synergy between renowned research teams. It is worth noting that the consortium members have recently carried out, separately, some encouraging preliminary studies showing the relevance, and importance of these working directions.

In the last decade, deep neural networks became state-of-the-art in many computer vision tasks. Nevertheless, their performances are affected when test data are acquired in environments visually different from the data used at training time. Recent domain adaptation techniques are efficient to mitigate this problem but they assume that target data distributions are fixed and available in a batch setting. These limitations severely constrain potential applications. In ODACE, we consider the scenario of an autonomous device, such as a car or a robot, navigating in a continuously changing environment. In this scenario, the different vision tasks are performed using a deep neural network. We focus more specifically on structured prediction tasks such as depth estimation and instance segmentation. In this project, we propose to develop new types of deep learning algorithms where the neural network parameters are continuously adapted to handle the current visual environment. The goal is to design dynamic mechanisms that can online adapt the network representations without human supervision using only the video frames from the current environment. In this scenario, we identify four main challenges. First, because of the sequential nature of data collection in the current environment, the model disposes of a few samples only with limited visual variability. This partial knowledge about the current environment affects the performance of standard domain adaptation methods. This problem is referred to as partial domain adaptation. Second, in ODACE, we will specifically address structured prediction problems. In this case, we claim that adaptation should be constrained in such a way that it leads to predictions with coherent structures. Third, in the case of deep neural networks, parameters are updated via Stochastic Gradient Descent (SGD). In our online setting, SGD is problematic since it is computationally costly and it assumes that training samples are independent and identically distributed while video frames are usually correlated. Consequently, new online optimization methods must be designed to obtain a fast online adaptation. Forth, we assume that the device may encounter environments similar to previously seen environments. Therefore, we propose models capable of benefiting from past adaptation experiences in order to adapt faster to the current environment. This ability to continually learn over time by retaining previously learned experiences is referred to as continual learning. Unfortunately, neural networks tend to forget previously learned information. Solutions have been proposed to alleviate this catastrophic forgetting problem but they are generally limited to classification settings. Therefore, we aim at designing new continual learning methods for structured prediction tasks that can address the online setting. To illustrate the large range of potential applications of our approaches, the proposed methods will be evaluated on four different use-cases corresponding to two tasks (depth estimation and instance segmentation) performed in two different scenarios (autonomous driving and robot navigation). To validate the methods developed in this project, we plan to record a new dataset for depth estimation in the case of a continuously changing environment.

Forecast is a major task of statistics in many domains of application. It often takes the form of a probabilistic forecast where the so-called predictive istribution represents the uncertainty of the future outcome given the information available today. Of particular interest is the distributional forecast of rare and extreme events, for instance environmental hazards such as flooding or heat waves, that can have major socio-economic consequences but for which current prediction are often inaccurate and unsatisfactory. In meteorology and weather forecast, the current numerical weather prediction (NWP) models are rather skillful for non-extreme weather events but often fail to provide accurate predictions on extreme weather events. One objective is to derive new statistical post-processing methods that are tailored to the output of existing NWP models in order to improve the forecast of extreme weather. Tree-based method such as generalized random forest for extremes will be developed. Data sets required to develop and validate the new proposed methods will be provided by the Centre National de Recherches Météorologiques (CNRM). In energy and electricity consumption forecast, balancing production and demand is a major concern and forecasting the demand and especially its peaks is crucial. Electricit\'e de France (EDF) has developed a strong expertise where one key tool is the possibility to combine different models to improve prediction. Aggregation of experts is another main direction of the T-REX project with an emphasis on specialized/sleeping experts that focus on extreme regimes. EDF will provide relevant data sets for electricity consumption forecast. Extreme value theory (EVT) provides a theoretical framework for risk assessment and mathematically justified estimation of rare event probabilities. The T-REX project is rooted in EVT and will bridge different research fields to improve probability forecasts of extremes from complex and high dimensional systems.

Large random matrices have been proved to be of fundamental importance in mathematics (high dimensional probability and statistics, operator algebras, combinatorics, number theory,...) and in physics (nuclear physics, quantum fields theory, quantum chaos,..) for a long time. The introduction of large random matrix theory in electrical engineering is more recent. It was introduced at the end of the nineties in the context of digital communications in order to analyse the performance of large CDMA and MIMO systems. Except some pionneering works of Girko, the use of large random matrices is even more recent in statistical signal processing. The corresponding tools turn out to be useful when the observation is a large dimension (say M) multivariate time series (y(n)), n=1,...N, and that the sample size N is of the same order of magnitude than M. This context poses a number of new difficult statistical problems that are intensively studied by the high-dimensional statistics community. The most significant example is related to the fundamental problem of estimating of the covariance matrix of the observation because the standard empirical covariance matrix defined as the empirical mean of the y(n)y(n)* is known to perform poorly if N is not significantly larger than M. In the context of this project, the dimension of the observation corresponds to the number of elements of a large sensor network, and the components of vector y(n) represent the signal received at time n on the various sensors. It turns out that a number of fundamental statistical signal processing schemes such as source detection, source localisation, independent source separation, estimation of linear prediction filters...fail in the case where M and N are large and of the same order of magnitude, a context modelled in the following by the asymptotic regime M and N both converge to infinity in such a way that the ratio M/N converges to a non zero constant c. The purpose of this project is to develop new mathematical tools and algorithms which allow to enhance the performance of classical methods in the above regime. In Task 1, the literature of large random matrices and of high-dimensional statistics is reviewed in order to identify the situations in which new mathematical results are needed. Task 2 is devoted to the study of fundamental statistical inference problems related to the so-called narrow-band sources antenna array model. Although a few topics related to this model were addressed quite recently in the above asymptotic regime, important problems still deserve to be studied. Task 2-1 first revisits classical detection and estimation problems related to narrowband sources in the case where M and N are of the same order of magnitude. The goal of Task 2-2 is to adapt certain blind source separation algorithms to the above asymptotic regime. For this, it is necessary to study, among others, the behaviour of large random matrices such as the empirical covariance matrix associated to vector z(n) defined as the Kronecker product of y(n) with itself. Task 3 addresses the case of wide band sources for which the contribution of each source to the observation is the output of an unknown a 1 input / M outputs filter driven by the source signal. In this context, it is crucial to estimate the covariance matrix of the extended vector obtained by stacking L consecutive observations. The goal of Task 3-1 is to develop the corresponding new mathematical results. The behaviour of the spectrum of the empirical covariance matrix of the augmented vector is first studied when ML and N are of the same order of magnitude. Then, consistent operator norm estimators based on banding, thresholding or tapering are developed when these schemes are relevant. The new results are used to address the detection of wideband sources (Task 3-2) and the trained and blind spatio-temporal equalization (Task 3-3).

The goal of this project is to detect "online" noticeable events in video sequences. By "noticeable event" we mean any event that draws attention by its spatial and/or temporal behavior. In this project, we wish to develop a general-purpose detection framework. This rules out supervised learning-based algorithms which require specific training data. Rather, we propose to characterize noticeable events as break points w.r.t. their context, which is more suited for our goal. Actually, we are not only interested in studying only the actors of the scene (i.e., any entity whose dynamics or visual aspect draws attention) and their individual behaviors, but also in studying their behaviors w.r.t. their environment. Each actor will be observed in a so-called "observation" region that will be determined using local or semi-local low-level generic primitives (points of interest, edgels, areas, etc.) still to be defined, as well as clustering methods applied on these primitives. The relationships between different actors or between the actors and the context will be modeled by spatial, temporal and spatio-temporal relations. Those will be defined over observation regions or over sets of descriptors to result in compact descriptions. In addition, they will be compared and their evolution over time studied. Modeling relations between scattered sets of visual primitives (including points of interest) as well as spatio-temporal relations, and defining their comparison measures is one of the novel contributions of this project. For this purpose, we propose to rely on both mathematical morphology and fuzzy sets. In addition, observation regions will be accurately tracked by an advanced tracking module based on a particle filter relying on a dynamic Bayesian network. The advantage of exploiting this module is twofold: first, the optimal filter will be guided in its prediction and estimation processes between consecutive time slices by the spatial relations computed before, hence resulting in better trackings; second, these estimations will be exploited to improve the determination of the descriptors and the observation regions in the next time slice. As observation regions evolve over time (their number may for instance vary) as well as their relations, we propose to make the structure of the dynamic Bayesian network evolve accordingly (in this case, such a network is called non-stationary). Thus, all the features of the studied scene can be integrated "online" into the dynamic Bayesian network. This is another originality of the project. Noticeable event detection will be performed either on a temporal or a spatial basis. For the former case, the non-stationary dynamic Bayesian network will allow us to detect break points due to its structure shifts over time. In addition, measuring the quality of the estimations provided by the particle filter will allow us to detect inaccurate trackings, which may be an indicator of contextual break points highlighted by a misguidance of the prediction process due to the spatial relations. Finally, measuring the evolution over time of the spatial relations will also provide evidence of local break points. Our approach will be tested against different kinds of scenarios (somebody leaving an object, somebody moving very differently from the surrounding crowd, etc.). It will then be validated on databases of sequences dedicated to event detection whose ground truth we will construct.