Paper detail

Pioneering the Exascale era with Astronomy

SURF is the collaborative ICT organisation for Dutch education & research and coordinates the national e-infrastructure (surf.nl). To accelerate scientific discovery, SURF invests in, operates and explores high-end IT solutions for and with researchers in the Netherlands. In this ADASS 2019 contribution we present our latest developments in high performance and high throughput cloud computing. These developments are particularly relevant for Astronomy, as this science domain: (i) generates large (Petabyte sized) data collections, (ii) uses rapid release and deployment schemes for their software, (iii) requires flexible and interactive test and staging environments and (iv) needs to execute complex workflows on diverse data structures. We highlight our new OpenStack-based, cloud infrastructure layer and focus on Spider, our new addition to SURF's high throughput data processing platforms.