Distributed Resources for the Earth System Grid Advanced Management (DREAM)

by   Luca Cinquini, et al.

The DREAM project was funded more than 3 years ago to design and implement a next-generation ESGF (Earth System Grid Federation [1]) architecture which would be suitable for managing and accessing data and services resources on a distributed and scalable environment. In particular, the project intended to focus on the computing and visualization capabilities of the stack, which at the time were rather primitive. At the beginning, the team had the general notion that a better ESGF architecture could be built by modularizing each component, and redefining its interaction with other components by defining and exposing a well defined API. Although this was still the high level principle that guided the work, the DREAM project was able to accomplish its goals by leveraging new practices in IT that started just about 3 or 4 years ago: the advent of containerization technologies (specifically, Docker), the development of frameworks to manage containers at scale (Docker Swarm and Kubernetes), and their application to the commercial Cloud. Thanks to these new technologies, DREAM was able to improve the ESGF architecture (including its computing and visualization services) to a level of deployability and scalability beyond the original expectations.



There are no comments yet.


page 5

page 7

page 8

page 9

page 10

page 11

page 12

page 13


A Novel IoT Architecture based on 5G-IoT and Next Generation Technologies

The Internet of Things (IoT) is a crucial component of Industry 4.0. Due...

INDIGO-DataCloud:A data and computing platform to facilitate seamless access to e-infrastructures

This paper describes the achievements of the H2020 project INDIGO-DataCl...

Managing Distributed MARF with SNMP

The scope of this project's work focuses on the research and prototyping...

The TAAROA Project Specification

Since its introduction, the Grid computing paradigm has been widely adop...

The ShareGrid Portal: an easy way to submit jobs on computational Grids

Grid computing is a distributed computing paradigm which aims to aggrega...

Dynamo – Handling Scientific Data Across Sites and Storage Media

Dynamo is a full-stack software solution for scientific data management....

Seeing poverty from space, how much can it be tuned?

Since the United Nations launched the Sustainable Development Goals (SDG...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.