In the scientific context, such as climate change, the availability of extreme-scale technologies for scientific computing has been enabling the execution of experiments and simulations that generate unprecedented volumes of data. This massive amount of data represents an invaluable opportunity to advance scientific discovery, though it poses several challenges concerning efficient data management, analytics and knowledge extraction. To this end, several efforts have been undertaken by the scientific software community towards the definition of novel systems able to support High Performance Data Analytics (HPDA).
The Ophidia HPDA framework, developed at CMCC, tries to address such data challenges by providing scalable data analysis features. The framework is currently being integrated with the new CMCC SuperComputing infrastructure (i.e., Zeus), jointly with higher-level, Python-based software solutions for experiment definition, post-processing and results visualization. The ultimate goal of this activity is to provide scientists with a HPDA-enabled and user-friendly environment for Data Science. The talk will introduce the environment, discuss early experiences and present preliminary results.
