The objective is to create a complete three-dimensional digital talking head including the vocal tract from the vocal folds to the lips, the face and integrating the digital simulation of aeroacoustic phenomena. Our project is particularly aimed at learning articulatory gestures from corpora of data from the vocal tract (real-time MRI), the face (motion capture) and subglottic pressure, highlighting latent articulatory variables that are relevant from the point of view of speech production control, and aeroacoustic simulations that allow exploring speech production and learning control of a replica simulating the vocal tract. The project will make extensive use of deep learning techniques in interaction with physical simulations, which is an important innovation. The consortium is made up of 4 remarkably complementary research teams with internationally leading theoretical and practical experience in the fields of AI (particularly deep learning techniques in automatic speech processing), acoustics, experimental phonetics, MRI imaging and automatic speech processing. The project is organized into 5 main tasks: 1) acquisition of a corpus of data covering 3 hours of speech (with several expressions) for one male and one female speaker (plus two speakers with less complete data) for dynamic MRI, facial deformation and subglottic pressure data. 2) corpus pre-processing to track the contour of articulators in MRI films, align modalities, denoise speech data, and reconstruct the vocal tract in 3D from dynamic 2D data and static 3D MRI data. 3) development of the control of the temporal evolution of the vocal tract shape, the face and the glottis opening based on the sequence of phonemes to be articulated and supra-segmental information. The approach will be based on in-depth learning using the corpus of the project and will aim in particular to bring out latent variables allowing the speaking head to be controlled and expressions to be rendered. 4) learning how to control a physical model of the simplified vocal tract using a large number of measurements. Deep learning will allow the development production strategies for plosives involving phenomena that are too rapid to be imaged with sufficient precision. 5) Adaptation of the talking head to other speakers based on anatomical landmarks and study of the acoustic impact of articulatory perturbation using the talking head. The talking head will generate the temporal evolution of the complete shape of the vocal tract and face and the signal produced by acoustic simulation from a sentence to be pronounced. It will also be possible to produce the audio-visual signal without the acoustic simulation but losing the possibility of introducing perturbations into production and thus to study in depth the production of speech which is the main interest of this project. The first result is the development of a radically new approach to the modelling of speech production. Until now, production models, and in particular those used for articulatory synthesis, exploit numerical models whose formal framework limits the possibility of accounting for real data such as real-time MRI. The fields of application concern the exploitation of dynamic MRI data, the diagnosis of speech pathologies, real-time feedback inside the MRI machine, the rehabilitation of articulation gestures, the deployment of realistic talking heads for the entire vocal tract and the improvement of the rendering of lips in talking heads.

The CONFLUENCE project aims to develop artificial intelligence (AI) technologies for sound semantic segmentation of acoustic signals that can recognize sound events and separate/isolate the signals of the sound sources forming semantic entities. This can be seen as an audio extension of the image semantic segmentation task that has been receiving much attention in the AI field. The objective is to design an embedded AI system that can implement these technologies for two telecom application services: immersive communication and home monitoring/assistance devices. Specifically, by analyzing, separating, and augmenting the separated signals with metadata such as location information, it will be possible to transmit only the necessary sounds from a sound scene and spatially recreate that scene in a remote location, or understand what is happening remotely. This could allow advanced communication services, such as privacy-conscious web conferencing systems and home monitoring/assistance devices/systems, that filter out the sounds of daily life and allow only transmit adult speech and important sounds, or new inclusive communication systems that would allow local and remote participants in international conferences to feel in the same location. In this project, we will mainly address the following issues: (1) the development of sound source separation and target sound extraction technologies as a basis for semantic separation, (2) open source implementation of the developed technology and hardware implementation in embedded devices, (3) dissemination of the technology through a proposal to the International Standardization of Mobile Communication Systems (3GPP) and deployment to privacy-preserving home monitoring devices, (4) Contribution to the acceleration of the academic field by disclosing the data acquired in this project and holding international technology evaluation challenges. Through the combination of scientific and industrial expertise from Japan and France, the CONFLUENCE project will develop innovative sound segmentation solutions for embedded devices.

Formal Concept Analysis (FCA) is a mathematical framework based on lattice theory and aimed at data analysis and classification. FCA, which is closely related to pattern mining in knowledge discovery (KD), can be used for data mining purposes in many application domains, e.g. life sciences and linked data. Moreover, FCA is human-centered and provides means for visualization and interaction with data and patterns. Actually it is now possible to deal with complex data such as intervals, sequences, trajectories, trees, and graphs. Research in FCA is dynamic, but there is still room for extensions of the original formalism. Many theoretical and practical challenges remain. Actually there does not exist any consensual platform offering the necessary components for analyzing real-life data. This is precisely the objective of the SmartFCA project to develop the theory and practice of FCA and its extensions, to make the related components inter-operable, and to implement a usable and consensual platform offering the necessary services and workflows for KD. In particular, for satisfying in the best way the needs of experts in many application domains, SmartFCa will offer a ``Knowledge as a Service'' (KaaS) component for making domain knowledge operable and reusable on demand.

SEVERITAS advances information socio-technical security for Electronic Test and Assessment Systems (e-TAS). These systems measure skills and performances in education and training. They improve management, reduce time-to-assessment, reach larger audiences, but they do not always provide security by design. This project recognizes that the security aspects for e-TAS are still mostly unexplored. We fill these gaps by studying current and other to-be-defined security properties. We develop automated tools to advance the formal verification of security and show how to validating e- TAS security. rigorously. We also develop new secure, transparent, verifiable and lawful e-TAS procedures and protocols. We also deploy novel run-time monitoring strategies to reduce frauds and study the user experience about processes to foster e-TAS usable security. And thanks to connections with players in the business of e-TAS, such as OASYS, this project will contribute to the development of secure e-TAS.

Noise pollution has a significant impact on quality of life. In the office, noise exposure creates stress that leads to reduced performance, provokes annoyance responses and changes in social behaviour. Headphones with excellent noise-cancelling processors can now be acquired in order to protect oneself from the noise exposure. While these techniques have reached a high performance level, residual noises still remain that can be important sources of distraction and annoyance. We propose to study two augmented reality approaches, mostly targeted towards disturbance in open offices. We target additional sound source levels that are below or equal to the one of the noise source. The first approach is to conceal the presence of an unpleasant source by adding some spectrotemporal cues which will seemingly convert it into a more pleasant one. Adversarial machine learning techniques will be considered to learn correspondences between noise and pleasing sounds and to train a deep audio synthesiser that is able to generate an effective concealing sound of moderate loudness. The second approach is to tackle a common issue encountered in open offices, where the ability to concentrate on the task at hand is made harder when people are speaking nearby. We propose to reduce the intelligibility of nearby speech by the addition of sound sources whose spectro-temporal properties are specifically designed or synthesised with a generative model to conceal important aspects of the nearby speech. The expected outcomes of the project are: 1) advances in the recent field of deep neural audio and speech synthesis and 2) lead to innovative applications for the engineering of the mitigation of noise in our daily life.