Much clinical and biomedical knowledge is contained in the text of published articles, Electronic Health Records (EHRs) or online patient forums and is not directly accessible for automatic computation. Natural Language Processing (NLP) techniques have been successfully developed to extract information from text and convert it to machine-readable representations. The most advanced applications have focused on identifying clinically relevant entities and concepts from English text. However, for many biomedical informatics tasks it is necessary to go beyond the identification of isolated instances in single documents – the context of concept occurrences and the nature of the relationships between co-occurring concepts are often crucial for a specific understanding of the analyzed text. Furthermore, while most of the literature is available in English, EHRs in French hospitals are written in French. Therefore, it is important to develop advanced methods for French that will provide structured representations of clinical text compatible with existing representations for English. This research project will focus on the following aims: 1. Providing material for text analysis in a specialized domain (i.e. the biomedical domain) in French 2. Adaptation to a specialized domain of NLP tools developed for the general language 3. Application to the automatic detection of links between clinical characteristics and medical history of patients described in EHRs, predictive biomarkers identified by immunologic or genetic studies and evidence of such associations reported in the literature The proposed research is innovative and will provide an in-depth study of multiple biomedical texts in French (EHRs) and in English (literature). It will be guided by linguistic principles and by the application to personalized medicine. A global approach should ensure that the methods used can be generalized to other biomedical applications.

Abstract: The new uses of affective social robots, conversational agents, and the so-called “intelligent” systems, in fields as diverse as health, education or transport reflect a phase of significant change in human-machine relations which should receive great attention. How will Human co-learn, co-create and co-adapt with the Machine? Notably, how will vulnerable people will be protected against potential threats from the machine? The first results from an original pre-experiment, conducted by the proposed Chair’s team in June 2019 in partnership with an elementary school, shows that an AI machine (Pepper robot or Google Home) is more efficient at nudging than adults. HUMAAINE’s aim is to study these interactions and relationships, in order to audit and measure the potential influence of affective systems on humans, and finally to go towards a conception of "ethical systems by design" and to propose evaluation measures. The planned scientific work focuses on the detection of social emotions in human voice, and on the study of audio and spoken language manipulations (nudges), intended to induce changes in the behavior of the human interlocutor. This work also uses the contributions of behavioral economics highlighted by the recent 2017 Nobel Laureate Richard Thaler, in our case, applied to human-machine interactions. The roap map for the Chair’s work includes experimental studies to evaluate ethical aspects and confidence in the Human-Machine couple, as well as by demystification of these technologies among the general public which naturally tends towards anthropomorphism. This project combines the scientific research in artificial intelligence with the implementation of an innovative methodology to evaluate and improve the ethics of the HM affective interaction, despite the current opacity of the AI systems. This project pushes forward a strong interdisciplinary collaboration already existing between affective computing, behavioral economics, linguistics, and natural language processing. The researchers will disseminate the results of the chair through their Master courses and workshops with Collège des Bernardins. HUMAAINE is supported by CNRS-LIMSI, Cognition Institute (Institut Carnot), and by the future of society (Harvard Initiative for Learning and Teaching), ENSC (Bordeaux) and presents great synergy with DATAIA Institute. The Domicile foundation, IRCEM and the Collège des Bernardins are the first to support HUMAAINE, followed by industials (CareCever, Renault, etc.) and other foundations (MAIF, Anne de Gaulle). We are also building cooperation with international research teams with Japan (Osaka Univ.), Germany (DFKI), and also Canada (Observatory/MILA).

From the original idea to the actual distribution on TV, VOD and DVD, the production process and subsequent “life” of a TV program are divided into numerous stages, involving numerous actors (e.g. distribution, production, direction, screenwriting, casting, post-production or dubbing) and thus leading to the generation of a huge amount of heterogeneous metadata. However, only a few metadata eventually survive the tortuous production and distribution processes, making their integration difficult into novel TV-centric products. Even for TV productions where one actor manages the whole production pipeline (e.g. Canal+ in France or BBC in the UK), most of the metadata do get lost at one point or another. The MetaDaTV network proposal aims at initiating a European research community around metadata associated with TV productions (such as dramas, documentaries or TV films) and at gathering interested partners from all over Europe toward a joint European project submission (Horizon 2020).

Speaker diarization is an unsupervised process that aims at identifying each speaker within an audio stream and determining when each speaker is active. It considers that the number of speakers, their identities and their speech turns are all unknown. Speaker diarization has become an important key technology in many domains such as content-based information retrieval, voice biometrics, forensics or social-behavioural analysis. Examples of applications of speaker diarization include speech and speaker indexing, speaker recognition (in the presence of multiple speakers), speaker role detection, speech-to-text transcription, speech-to-speech translation and document content structuring. Although speaker diarization has been studied for almost two decades, current state-of-the-art systems suffer from many limitations. Such systems are extremely domain-dependent: for instance, a speaker diarization system trained on radio/TV broadcast news experiences drastically degraded performance when tested on a different type of recordings such as radio/TV debates, meetings, lectures, conversational telephone speech or conversational voice-over-ip speech. Overlap speech, spontaneous speaking style, background noise, music and other non-speech sources (laugh, applause, etc.) are all nuisance factors that badly affect the quality of speaker diarization. Furthermore, most existing work addresses the problem of offline speaker diarization, that is, the system has full access to the entire audio recording beforehand and no real time processing is required. Therefore, the multi-pass processing over the same data is feasible and a bunch of elegant machine learning tools can be used. Nevertheless, these compromises are not admissible in real-time applications mainly when it comes to public security and fight against terrorism and cyber-criminality. Moreover, after an initial step of segmentation into speech turns, most approaches address speaker diarization as a bag-of-speech-turns clustering problem and do not take into account the inherent temporal structure of interactions between speakers. One goal of the project is to integrate this information and rely on structured prediction techniques to improve over standard hierarchical clustering methods. Since our main application is related to the fight against cyber-criminality and public security, designing an online speaker diarization system is necessary. Therefore, the focus on industrial research will be supplemented by addressing more fundamental research issues related to structured prediction and methods such as conditional random fields. Speaker diarization is inherently related to speaker recognition. In the recent years, state-of-the-art speaker recognition systems have shown good improvement, thanks to the emergence of new recognition paradigms such as i-vectors and deep learning, new session compensation techniques such as probabilistic linear discriminant analysis, and new score normalization techniques such as adaptive symmetric score normalization. However, existing speaker diarization systems did not take full advantages of those new techniques. Therefore, one goal of the project is to adapt those techniques for speaker diarization, and thus fill the research gap in the current literature. To evaluate the proposed algorithms and to ensure their genericness, different existing databases will be considered such as NIST SRE 2008 summed-channel telephone data, NIST RT 2003-2004 conversational telephone data, REPERE TV broadcast data and AMI meeting corpus. Furthermore, we are aiming to collect a medium-size database that suits our main application of fight against cyber-criminality.

The aim of this project is to augment and amplify human intelligence by artificial intelligence. In particular, we propose and develop Learning Cyclotron (LeCycl), a framework which monitors and actuates their learning processes. The cyclotron of LeCycl is a framework in which learners acquire knowledge with the following three steps: perceive (e.g., reading a textbook), master (e.g., solving an exercise), and transfer (e.g., explaining to other learners). Behaviors of learners on each step are measured by sensors. From the sensor signals, we estimate cognitive/affective states including also attention, interest, concentration, mental workload, and self-confidence. The estimated states will be utilized to select how to nudge learners. As the same as a cyclotron accelerates charged particles by a magnetic field, LeCycl circulates knowledge between learners.