The EO4EU Multi-Cloud Computing infrastructure

As Earth Observation (EO) data continues to grow in volume, velocity, and variety, the EO4EU platform rises to the challenge with a robust, scalable, and federated multi-cloud computing infrastructure. At the heart of this infrastructure are two major cloud providers: CINECA and WEkEO, the latter through the European Centre for Medium-Range Weather Forecasts (ECMWF). These platforms power the computational backbone of EO4EU, supporting diverse workloads from EO data fusion to AI/ML inference, while embracing open standards and cloud-native technologies for resilience and flexibility. 

CINECA and the Ada Cloud 

CINECA is a non-profit consortium comprising Italian universities, research institutions, and the Italian Ministry of Education. It is one of the largest computing centres in Europe and plays a pivotal role in supporting scientific research through its advanced computing facilities. 

EO4EU taps into the Ada Cloud, CINECA’s OpenStack-based cloud infrastructure, which provides elastic and high-performance virtualized resources tailored to the needs of modern research projects. Designed to complement traditional HPC workloads, Ada Cloud offers an ideal platform for deploying scalable cloud-native applications and services. 

ECMWF and the WEkEO Platform 

The European Centre for Medium-Range Weather Forecasts (ECMWF) is an independent intergovernmental organization that provides global numerical weather predictions and supports climate monitoring, atmospheric composition, and environmental services. Its advanced computational infrastructure and scientific expertise make it a key player not only in forecasting but also in large-scale Earth system data processing and analysis. 

ECMWF is also a core partner in the WEkEO DIAS (Data and Information Access Service), which is operated jointly by EUMETSAT, ECMWF, EEA, and Mercator Ocean International, and which combines Copernicus EO data accessibility with cloud-based processing capabilities.  

A Federated Multi-Cloud Architecture 

EO4EU's computing infrastructure spans both CINECA Ada Cloud and WEkEO, effectively forming a federated multi-cloud environment. This approach ensures that resources are efficiently allocated across providers, enabling flexible deployment strategies and reducing the risk of vendor lock-in. 

Both infrastructures are built on OpenStack, a broadly deployed open-source cloud operating system that allows for the creation and management of large pools of compute, storage, and networking resources. 

Resource Overview 

The EO4EU platform is currently leveraging the following compute and network resources: 

As described below, these resources underpin a series of Kubernetes clusters, orchestrated through Rancher, a widely adopted multi-cluster management platform. 

Cloud-Native Orchestration: Kubernetes and Rancher 

Kubernetes is an open-source container orchestration system designed to automate the deployment, scaling, and management of containerized applications. By decoupling applications from underlying infrastructure, Kubernetes facilitates seamless portability, fault tolerance, and continuous deployment. 

To manage the Kubernetes environment across multiple clouds, EO4EU uses Rancher, which simplifies operations by providing a centralized interface for cluster provisioning, policy enforcement, and workload monitoring. 

EO4EU Kubernetes Cluster Landscape 

1. The EO4EU architecture currently includes several dedicated Kubernetes clusters: 
2. Main Production Cluster – Hosts the EO4EU platform's core components and services. 
3. Management Cluster – Provides Authentication and Authorization Infrastructure (AAI) and supports DevOps practices through CI/CD and GitOps. 
4. AI/ML Inference Cluster – Handles model deployment and execution for EO-based machine learning tasks. 
5. Fusion Cluster – Supports data integration, harmonization, and semantic fusion activities. 
6. Observability Cluster – Enables monitoring, logging, and tracing to ensure system health and performance. 
7. Object Storage Cluster – Offers S3-compatible storage services for the components of the EO4EU platform. 
8. Staging Cluster – Serves as a testing ground for components before moving into production. 

Challenges in Federated Cloud Operations 

Operating across a federated, multi-cloud environment brings several technical challenges: 

• Network Reliability: Ensuring consistent and secure communication between components distributed across cloud boundaries. 
• Large Data Transfers: Minimising (and managing if needed) the movement of massive EO datasets efficiently between clouds. 
• Observability and Testing: Proactive monitoring and testing pipelines (CI/CD) are essential to detect and resolve failures early. 
• Debugging Complexity: Diagnosing issues in a layered system with distributed services can be non-trivial. 
• Security Surface: A broader infrastructure footprint increases the potential attack surface, demanding robust security hardening. 

Architectural Advantages 

Despite these challenges, the EO4EU multi-cloud architecture offers compelling benefits: 

• Federated Flexibility: Resources can be deployed strategically across clouds based on workload needs, data locality, or performance. 
• Vendor Neutrality: Leveraging open-source technologies and multiple providers prevents lock-in and promotes interoperability. 
• Resilience and Recovery: The distributed architecture allows for failover mechanisms and robust disaster recovery strategies, essential for mission-critical EO applications. 

The EO4EU multi-cloud infrastructure exemplifies a modern, open, and cloud-native approach to the realisation of an Earth Observation platform. By combining the computational power of CINECA and WEkEO with best-in-class cloud orchestration tools, EO4EU lays a resilient foundation for retrieving, processing, analysing, and fusing EO data to support scientific research and societal decision-making.