Coupling Data and HPC resources together: the EUDAT - PRACE Collaboration activity

Overview:

This proposal presents the progress status of the collaboration activity between two major European Infrastructures, EUDAT, the European Collaborative Data Infrastructure, and PRACE, the Partnership for Advanced Computing in Europe. 
The EUDAT infrastructure initiative is a consortium of several major European data & compute centers and research community centers and organizations that are working towards the development and realisation of the Collaborative Data Infrastructure (CDI) which provides an interoperable layer of common data services and a common model for managing data spanning all European research data centres and data repositories to create a single European data infrastructure.
The scope of the CDI covers data management functions and policies for upload and retrieval, identification and description, movement, replication and data integrity.

PRACE – the Partnership for Advanced Computing in Europe – research infrastructure enables high impact European scientific discovery and engineering research and development across all disciplines to enhance European competitiveness for the benefit of society. PRACE seeks to realize this mission through world class computing and data management resources and services open to all European public research through a peer review process. The broad participation of European governments through representative organizations allows PRACE to provide a diversity of resources throughout Europe including expertise for the effective use of these resources. PRACE operates a dedicated 10 Gb/s network among Tier-0 and Tier-1 partners. To aid users and potential users as well as preparing the next generation scientists and engineers PRACE has an extensive pan-European education and training effort. Twenty-five PRACE members collaborate in the Fourth Implementation Phase (PRACE-4IP) project co-funded by the European Commission’s Framework Programme H2020. 

Research communities need to gain access to high quality managed e-Infrastructures whose resources are designed to be connected together thus enabling cross-utilization use cases. The capability to couple data and compute resources together is fundamental to accelerate scientific innovation and advance research frontiers. In ever growing scientific and industrial domains, the use of large-scale instruments (synchrotron, telescopes, satellites, sequencers, network of sensors, scanners), supercomputers and open data archives is leading towards the convergence among HPC, High throughput Computing, Networks and Data Management facilities. The number and rate of data produced in any particular discipline is starting to exceed our ability to treat them individually, leading to the development of a new field called High Performance Data Analytics. A large variety of services already exist but e-Infrastructures, for many reasons, have evolved along different dimensions creating separate offerings for computing and data. Scientific communities do not access low level computing and data services directly, but rather work with portals and workflows to perform complex tasks which leverage on the composition of different services typically offered by different providers.
The aim of this presentation is to report about the status of the collaboration between PRACE and EUDAT, two leading European e-Infrastructures in the field of High Performance Computing and Data Management, and on how they are collaborating to implement the Open Science vision where resources of any kind and size are accessible without any technical barrier. This collaboration connects experts, and representatives from scientific user communities, exploring ways in which major European e-Infrastructures can develop synergic and compound services. This joint activity covers different aspects, including standardization of service interfaces, harmonisation of access policies, lowering of technical barriers, joint support of users, and coordination of joint training activities and calls for proposal. Users participate actively and closely through the definition of concrete pilots, ensuring that the results of this effort meet their needs. As result of a first joint call for proposal, five pilots, covering different scientific domains, have been granted to access compute and data resources, and collaborate to the definition of new services and tools.

Target Audience:

By attending this session, research communities and infrastructures will learn about the opportunities and benefits of this collaboration and how they can contribute.

This PRACE – EUDAT collaboration contributes to the achievement of the European Commission’s Digital Single Market, and particularly the ambition that every research centre, every research project and every researcher has access to world-class supercomputing, storage and data facilities on the European territory.

Benefits for Audience:

The PRACE - EUDAT collaboration will be presented through real life use cases. This will give attendees an opportunity to identify potential usage and view how the implementation can deliver solutions.
They will also understand how some processes and tools can help defining their data workflow, as well as the way to handle the non technical aspects of such a project. Training opportunities will be presented also.

Topic 2: Services enabling research