CERN is one of the most highly demanding computing environments in the research world. The World Wide Web was originally conceived and developed at CERN to meet the demand for automated information-sharing between scientists in universities and institutes around the world. From software development, to data processing and storage, networks, support for the LHC and non-LHC experimental programme, automation and controls, as well as services for the accelerator complex and for the whole laboratory and its users, computing is at the heart of CERN’s infrastructure.
The Worldwide LHC Computing Grid (WLCG) – a distributed computing infrastructure arranged in tiers – gives a community of thousands of physicists near real-time access to LHC data. The CERN data centre – working in unison with its extension in Budapest – is at the heart of WLCG, the first point of contact between experimental data from the LHC and the grid. Through CERN openlab, a unique public-private partnership, CERN collaborates with leading ICT companies and other research organisations to accelerate the development of cutting-edge ICT solutions for the research community.
A continuing programme of upgrades and consolidations to the LHC and the experiments at CERN will result in hugely increased ICT demands in the coming years, exceeding what is expected to be achievable with a constant investment budget by several factors in both storage and computing capacity. The High-Luminosity LHC, the successor to the LHC, is planned to come online after 2025. By this time, the total computing capacity required by the experiments is expected to be 50-100 times greater than today, with data storage needs expected to be in the order of exabytes.
In order to address this, in 2017 the High Energy Physics community produced a roadmap Community White Paper (arXiv:1712.06982) that explores these software challenges and describes a number of potential mechanisms by which the community can address these problems of capacity and efficiency that will be faced during the next decade.
The large-scale data challenges of HL-LHC are also shared with other science communities. WLCG and CERN have been working with the Square Kilometre Array (SKA) radio astronomy community to explore some of these common challenges. In June 2017, CERN and the SKA Organisation signed a Collaboration Agreement for joint work in computing and data management.
In 2017 CERN openlab also published a white paper identifying the major ICT challenges that face CERN and other ‘big science’ projects in the coming years in order to collaborate with the industry and research institutes to tackle them.