The international consortium behind the Square Kilometre Array (SKA) have completed the five-year process of designing one of two supercomputers that will crunch its way through the mountains of celestial data collected by the fields of radio telescopes.
Almost 40 institutions from 11 countries, including Australia’s CSIRO, Pawsey Supercomputing Centre, and the Perth-based Internationational Centre for Radio Astronomy Research (ICRAR) participated in the design of the Science Data Processor (SDP).
The SDP functions will be split across two supercomputers, one in Perth and one in Cape Town, to be close to the SKA-low frequency and SKA-mid frequency sites in each respective country.
When fully functional, the combined power of the SDP is expected to produce the world’s fastest supercomputer, SKA Organisation’s SDP project manager Maurizio Miccolis said.
“We estimate SDP’s total compute power to be around 250 PFlops – that's 25 percent faster than IBM’s Summit, the current fastest supercomputer in the world.".
“In total, up to 600 PB of data will be distributed around the world every year from SDP – that’s enough to fill more than a million average laptops," he said
Doing that will require the SDP to ingest data and move it through data reduction pipelines to then form packages that will be copied and distributed to a global network of regional centres to be studied.
Designing the process of ingesting and sorting the data was handled by ICRAR in Perth.
“Our job has been to create something called the ‘execution framework’, which will run up to
100 million tasks every few hours as it ingests and processes data flowing from the telescope at enormous speeds,” ICRAR’s head of Data Intensive Astronomy Andreas Wicenec said.
“Because of the sheer quantity of data flowing into SDP - some 5 Tb/s, or 100,000 times faster than the projected global average broadband speed in 2022 - it will need to make decisions on its own in almost real-time about what is noise and what is worthwhile data to keep.”
The CSIRO, meanwhile, contributed to the software architecture design and imaging algorithm development.
Pawsey’s existing high performance computing capabilities came in handy, with the organisation developing the local infrastructure interface specification between SDP and the hosting data centres.
It also ported and tested automated scaling test frameworks on its Cray supercomputers.
Rosie Bolton, data centre scientist at the SKA Organisation, added that all of these systems behind the SDP have been designed to evolve with the SKA project over its 50-year lifespan.
“We have to design a system that’s highly scalable and highly extensible so that we can add on modules and change the size of it as time goes by.”
Construction of the systems will begin in Australia and South Africa from 2021.
