Storage Federation

A national effort is underway to connect HPC and storage providers to the needs of the NFDI consortia to build higher level RDM services. Among others, national service providers like the NHR alliance and de.NBI aim to contribute to this build-up. This work package prototypes the foundational infrastructure for a resilient and interoperable storage layer that integrates the high-capacity hardware assets, e.g. funded by the DFG/NFDI-storage call (and potentially further storage systems) and aligns them with the requirements towards generic EOSC Node capabilities and serves as a blueprint that can then be adopted by other storage providers. This integrated approach backs EOSC Node Germany with a storage platform for FAIR-compliant and data-intensive research. 

A federated system based on S3-compatible interfaces and geo-distributed object storage abstractions enables seamless data mobility and lifecycle management across local, national, and European tiers. Additionally, FAIR Digital Object based interoperability layers as provided by the RDM platform Coscine can be further integrated to ensure data is machine-actionable and linked with its relevant metadata. This ensures that research data is hosted on sovereign, high-performance infrastructure while remaining fully discoverable and accessible through the EOSC Federation, thereby bridging the technical gap between provision and service delivery to build further node capabilities. The work package focuses on the development of storage management capabilities that allow service providers to provision and manage S3 buckets, and aims to connect the national activities to Géant activities on the pan-European object storage.

S3 based object storage is available as a federated storage capability into services in the national node. It provides a scalable, secure, and durable interface that is ideal for managing large volumes of unstructured data. 

Secure access to federated storage is managed through the IAM4NFDI provided Community-AAIs. This connects the storage platform to digital identities and federated access across all NFDI consortia and within the EOSC Federation. 

Federated interoperability of data on top of storage protocols using machine-actionable self-descriptions. Using the FAIR Digital Object (FDO) concept, platforms like Coscine or or TUD Data Hub can add an interoperability layer to access data and metadata on the storage platforms.