Many complex autonomous systems (e.g., electrical distribution networks) repeatedly select actions with the aim of achieving a given objective. Reinforcement learning (RL) offers a powerful framework for acquiring adaptive behaviour in this setting, associating a scalar reward with each action and learning from experience which action to select to maximise long-term reward. Although RL has produced impressive results recently (e.g., achieving human-level play in Atari games and beating the human world champion in the board game Go), most existing solutions only work under strong assumptions: the environment model is stationary, the objective is fixed, and trials end once the objective is met.The aim of this project is to advance the state of the art of fundamental research in lifelong RL by developing several novel RL algorithms that relax the above assumptions. The new algorithms should be robust to environmental changes, both in terms of the observations that the system can make and the actions that the system can perform. Moreover, the algorithms should be able to operate over long periods of time while achieving different objectives. The proposed algorithms will address three key problems related to lifelong RL: planning, exploration, and task decomposition. Planning is the problem of computing an action selection strategy given a (possibly partial) model of the task at hand. Exploration is the problem of selecting actions with the aim of mapping out the environment rather than achieving a particular objective. Task decomposition is the problem of defining different objectives and assigning a separate action selection strategy to each. The algorithms will be evaluated in two realistic scenarios: active network management for electrical distribution networks, and microgrid management. A test protocol will be developed to evaluate each individual algorithm, as well as their combinations.

Advocated by the European Commission’s Green Paper Towards a new culture for urban mobility, the “city of short distances” is a sustainable development pattern, which should be extended to the metropolitan scale (CEC, 2007). Based on this pattern, the main objectives of the SIEMM project are: (1) to scale-up the impact of innovative urban mobility models (e.g. our French Ministries projects NEST-TERR/ADEME-MODEVALURB 2015-2 and Tram-train/IMR-MCC 2013-2015) and to support modal shift towards more efficient modes (e.g. high-speed, tram-train, tram, cycling and walking networks) for the development of metropolitan areas such “short distances and easy access cities”; and (2) to work closely together and share latest research results with local authorities in order to strengthen their knowledge and to support governmental decision making. The framework for the cooperation between the universities and local stakeholders will be the concept of European Metropolitan Mobility Modeling Centers. Moreover, the project will create a network of Metropolitan Mobility Modeling Centers to share knowledge and experiences across European cities. This synergy-rich framework, based on the Urban Modeling and Visualization technology (sort of BIM at the territorial scale), allows universities and local authorities to draw full benefit from the existing mobility projects in each city (e.g. NEST-TERR, IMR, CATS, CIVITAS) to optimize their expertise, and to apply new socio-economical and financing models related to the future of their metropolitan territories. The project will analyze differences, scale-up typologies and identify best mobility solutions that provide a potential added value in terms of efficiency, sustainability, livability and integration (e.g. morphological, social, economic) for a certain type of metropolitan area. The results will be brought together in Interactive Cartography, Maps and Tools on Innovative Urban and Metropolitan Mobility, and published at a major International Symposium in Strasbourg. The project will be contributing to enhancing stakeholder knowledge and capacities in this field, and to redirecting local policy in relation to the challenges of energy transition and sustainable metropolitan development. It will reinforce light-rail solutions and modal shift approaches (e.g. CATS-electric cars, tramway and tram-train projects) from a small number of locations in Europe to new locations in Europe and in China.