Computing platforms for big data analytics and artificial intelligence

Big data and ML and AI use cases are becoming omnipresent in business and academia as well as in public institutions, and addressing them effectively calls for continuous technological innovation. Efforts have particularly focused on the implementation of so-called big data and/or highperformance computing (HPC) platforms. Big data platforms enable the storage and processing of very large data sets, and are increasingly requested to deal with information of a semi-structured or unstructured nature. HPC platforms enable fast data processing and complex statistical analysis/simulations, and are typically well suited for dealing with medium-size structured data sets. Certainly, this distinction is a bit artificial, and the boundary between big data and HPC platforms is becoming blurry with the rapid development of IT technologies that are able to serve both types of user needs. 

Against this backdrop, the Bank of Italy, with the support of the BIS and the IFC, organised a workshop on “Computing platforms for big data and machine learning”. Convening in Rome in January 2019, participants from about 30 organisations including central banks, NSOs and international organisations took this opportunity to present on the status of their current big data initiatives and exchange experiences.  This IFC report draws on the examples reviewed on that occasion and presents the main big data / HPC technologies and platforms currently considered or already implemented by public authorities such as central banks.