Recently, the Computer Security Office reported on a cybersecurity incident at a remote Tier 2 site of the Worldwide LHC Computing Grid (WLCG). Compromised to the backbone, dozens of servers deeply infiltrated by the attackers, taken over and abused for cryptocurrency mining. For months. While the attackers’ earnings in dollars might reach five digits, the costs to that Tier 2 site are also significant. Let’s look at the ledger:

• Instead of performing physics computing, some nodes were mining cryptocurrency. This implies Tier-2-sponsored electricity being converted into Bitcoin and pocketed by the attackers. And computing resources were blocked from doing physics analysis.
• Since the detection in October 2024, the site has been taken offline. Hence, with their compute nodes down for four months now, this equates to about 5% of the investment in their computing power (assuming a hardware lifetime of eight years).
• On top of that comes the cost of reputational damage due to bad publicity and criticism in the media.
• The whole sysadmin team has since been occupied with incident response, delaying any software development. A team of five people occupied for four months results in twenty months of personpower (PM) in salaries paid for nothing productive. In addition, the CERN Computer Security Office has invested about 0.5 PM in recent months to help out with guidance, forensics and consultancy.
• Re-establishing full functionality will take even more time and resources and will require money for external consultancy, reviews and possibly training to avoid it happening again.

Given that we cannot disclose the local currency and the average salaries, of course, we cannot share a quantitative figure. But in abstract numbers over the whole time span of 6–8 months (detection, response and reinstallation) this adds up to 30–40 PM and 10% of their investment in compute. Or 10% of their committed computing power. Either number is non-negligible.

Compare that with the costs of implementing proper security measures prior to any incident. Actually, “prior” doesn’t even matter anymore as the same security measures will definitely need to be implemented in the aftermath. Any auditor, any incident responder and, in this particular case, even the attackers(!) pushed for such proper measures. And with 40 PM and 10% of the operating expenses of a computer centre, you can already put some decent security mechanisms in place. Firewalls. Monitoring. Better configurations. So, what about you? Ready to act prior to or after an incident?

______

Do you want to learn more about computer security incidents and issues at CERN? Follow our Monthly Report. For further information, questions or help, check our website or contact us at Computer.Security@cern.ch.