The HAL4SDV proposal aligns with the EU Strategic Research and Innovation Agenda 2022 on Electronic Components and Systems. It aims to pioneer methods, technologies, and processes for series vehicle development beyond 2030, driven by anticipated advancements in microelectronics, communication technology, software engineering, and AI. HAL4SDV envisions a future where vehicles are fully integrated into smart cities, intelligent highways, and cyberspace, blurring the lines between inside and outside the vehicle. Assumptions include data-centricity, code portability, efficient data fusion, unlimited scalability, real-time capabilities, and robust cybersecurity. The objectives encompass unifying software interfaces, creating a hardware abstraction framework, enabling Over-The-Air (OTA) updates, designing platform architectures, ensuring hardware abstraction and virtualization, offering hardware support, automating integration, supporting safety features, harnessing edge computing, implementing security measures, and providing essential development tools. By focusing on these objectives, HAL4SDV aims to establish a unified ecosystem for software-defined vehicles, positioning Europe's automotive industry for continued leadership post-2030 while leveraging existing results and technologies to accelerate progress.

Automated Road Transport (ART) is seen as one of the key technologies and major technological advancements influencing and shaping our future mobility and quality of life. The ART technology encompasses passenger cars, public transport vehicles, and urban and interurban freight transport and also extends to the road, IT and telecommunication infrastructure needed to guarantee safe and efficient operations of the vehicles. In this framework, CARTRE is accelerating development and deployment of automated road transport by increasing market and policy certainties. CARTRE supports the development of clearer and more consistent policies of EU Member States in collaboration with industry players ensuring that ART systems and services are compatible on a EU level and are deployed in a coherent way across Europe. CARTRE includes a joint stakeholder’s forum in order to coordinate and harmonise ART approaches at European and international level. CARTRE creates a solid knowledge base of all European activities, supports current activities and structures research outcomes by enablers and thematic areas. CARTRE involves more than 60 organisations to consolidate the current industry and policy fragmentation surrounding the development of ART.

The development of algorithms for Autonomous Vehicles (AVs) faces important challenges throughout the design and implementation pipeline. The high cost and complex operation of real-world test-beds limits the experience an embedded Artificial Intelligence (AI) can gather, as it originates from a few vehicles that cannot be kept online extensively. For this reason, development often goes through a simulation stage or a testing step on a simplified system (e.g., smaller vehicles, standalone sensors or robotic models). In MultiTrans the focus is on the perception stage of AVs, which needs to provide a very accurate representation of the driving environment(s), that is used as an input for the following decision and control steps, while allowing a clear discrimination between similar but different contexts. The project takes the perspective of vision-based embedded systems (i.e., relying on cameras or similar sensors) that are among the most promising perception solutions. Their underlying sensing technologies however make them sensitive to an important research challenge: facing adverse conditions (such as bad weather or sun glare). In addition, knowledge transfer between different (real or virtual) environment suffers from two additional issues: reality gap, when a simulation/model fails to capture all the particularities of a real system, and the extended development time caused by the inherent repeated iterative process of adapting an algorithm from a system/domain to a different one. In MultiTrans, we propose to address these research issues by tackling autonomous driving algorithms development and deployment jointly. The idea is to enable data, experience and knowledge to be transferable across the different systems (simulation, robotic models, and real-word cars), thus potentially accelerating the rate an embedded intelligent system can gradually learn to operate at each deployment stage. The research hypothesis acting as a starting point of MultiTrans corresponds to the current state of deployment of autonomous driving technologies: AVs can be programmed (or are able to learn) to react and operate in controlled (or restricted) environments autonomously. The focus of our proposal is on the AI-side : research is needed to help these systems during the perception stage, enabling AVs to be operational and safer in a wider range of situations. The project is expected to contribute to substantial advances with respect to state of the art, by resulting in (i) A novel theoretical framework and new algorithms on transfer and frugal learning in virtual and real environments; (ii) Advances in multi-domain and multi-source computer vision for semantic segmentation and scene recognition applied to safe autonomous driving and (iii) The development of a robotic autonomous vehicle model demonstrator combined with a virtual world model. The novelty in this project is to develop an intermediate environment that allows to deploy algorithms in a physical world model. This additional step will allow to re-create more realistic use cases that would contribute to a better, faster and more frugal transfer of perception algorithms to and from real autonomous vehicle test-beds. This robotic platform will also enable to lead research focusing on multi-domain and multi-actor transfer by reducing the time and efforts required to build relevant use cases and multiple variants of these scenarios, thus allowing to achieve domain generalization. We will also explore frugal learning techniques such as few-shot learning would reduce the amount of samples require for the recognition/segmentation tasks to converge before transferring them. Thanks to the platform, we will be able to evaluate solutions for complex configurations in the virtual environment and then transfer them on the platform, bridging the gap between behaviour cloning (through imitation learning) and simulation.

For improving the capabilities of general-purpose AI models and for extending their applicability to domains where the temporal dimension – among several others – is of importance, we will target the development of the next generation of Multimodal Space-Time Foundation Models (MSTFMs). These will combine spatio-temporal understanding, which is important even for modalities such as the visual one that have already been introduced in large generative models, with the effective management of new time-relevant modalities that are yet to be supported in foundation models, such as industrial time series data, remote sensing data and health-related measurements. Real and synthetic data, to mitigate data scarcity, will be leveraged for training general-purpose MSTFMs and for further adapting them for specific downstream tasks. Real data used for training will include data directly provided by members of the consortium as well as data from relevant European Data Spaces, while complementary synthetic data will be generated by exploiting existing generative AI capabilities as well as new ones developed in the project. European HPC infrastructure is directly included in the consortium to ensure the availability of the necessary computing resources.