Â̲èÖ±²¥

It’s an ordinary Sunday morning, and I’m enjoying my breakfast when suddenly my emergency phone rings. As the Â̲èÖ±²¥ CERT team’s security incident responder, I’m on standby this weekend. I have lots of thoughts and images running through my head. Has a university been hit by a cyberattack? Is it being blackmailed? I realise that this Sunday is going to be different than planned.

IT manager sounds the alarm

I look longingly at my breakfast but answer the phone. On the other end of the line is Polly, the head of central IT at the university in question. She’s in the office because she forgot her gym bag. Polly is flustered as she has just noticed strange activity on the network that points to a computer security incident. The framework that generates these alerts provides the university with an overview of many activities throughout the network. However, it is still under construction. They have not yet defined the processes for dealing with these kinds of alerts.

Polly is concerned about how to explain the incident to management, and I can sense the tremendous pressure on her. The university’s digital resources are currently under acute threat. This incident may have far-reaching consequences, such as financial damage, a data leak, or even endangering the very existence of the organisation.

Â̲èÖ±²¥ CERT provides support in the event of a security incident

Polly’s pretty sure something’s wrong. She has detected activity from an account of a system administrator who is away on holiday! She also sees the name ‘Mimikatz’ in the warnings, which flashes in red.

Therefore, I carry out an initial risk assessment for this security incident. What kind of incident is it? Who and what is affected? What impact could this have on the university? It’s about understanding what situation we’re facing in order to define the best next steps.

Blackmail before encryption

I ask her if she sees anything else. She looks at her computer and notices an email with the subject line ‘Ransom’. She opens this and reads me a blackmail message. The extortionists threaten to publish data on the dark net if the university does not transfer CHF 100,000 in bitcoin within the next 72 hours. We’re both puzzled because the systems aren’t encrypted yet. I know that we need to act quickly now to deal with the incident with as little loss as possible. I recommend that she immediately disconnect her entire infrastructure from the internet. In this way, we reduce the possibility of the perpetrators triggering the encryption.

Attackers also make mistakes

As soon as Polly pulls the plug, nothing and no one at her university can be reached via the internet. She calls me back on her cell phone because at this university, like most others, phone calls are made over the internet.

We then discuss the next steps and move on to the next phase: formulating an incident response strategy. This phase and the detailed assessment of the situation will be repeated iteratively throughout the entire process. The latest information will be consulted on an ongoing basis to allow for any necessary adjustments to the procedure to be made.

All stakeholders at the round table

My next recommendation is that Polly and I put together a crisis team so that we can come together to gather further information and formulate the next measures of the strategy. Who are the members of this crisis team? This includes Polly herself, as head of central IT, and her system managers. At least one person from university management, the legal department and communications is also required. They all form the crisis team and make the difficult decisions. The communications department then communicates these to members of the university and to the public. In this case, it is a small university. The vice-chancellor, therefore, assumes this role. Â̲èÖ±²¥ CERT assumes the role of incident manager. We never do this alone. We always work as part of a team, which is why I ask a colleague for support. We then physically go to the university to work on site. Our job in this incident is to maintain an overview of the overall situation and to help with the analysis. We also involve law enforcement authorities in the crisis team. Almost without exception, a cyber attack is a criminal offence. The mutual support of law enforcement and CERT makes it possible to take more comprehensive measures.

Back to the analogue age

The crisis team requires a physical situation room but in the age of zoom and mobile phones, you’ve got to find one first. When everything runs on the internet, but it suddenly no longer works, on a Sunday evening you find yourself in a stuffy room, and everyone looks at the analogue tools in their hands – paper and pencil.

It is still unclear how far the infection reaches. Any accidental connection to the internet could inexorably trigger encryption. That’s why we move from the digital to the analogue age. We discuss the latest findings. All the information has to be on the table. How did the culprits manage to penetrate the systems? What measures do we need to take to protect the IT infrastructure? How can we make users feel safe again?

The guessing game begins

This is followed by the next step of the incident response process: investigating the facts. We allocate all the tasks that need to be completed in the next few minutes and hours. We collect and analyse all relevant information and try to get to the bottom of the questions. When it comes to digital forensics, I’m in my element – there are lots of puzzles and they need to be solved. The system managers play an important role in this. They can see at a glance whether their systems are behaving normally. Today, we’re dealing with IT infrastructures that consist of thousands of systems, and almost every system requires a specialist. Without dialogue and good collaboration, we would have no chance. In this scenario, every system poses a potential risk to the entire organism.

At Â̲èÖ±²¥ CERT, we do forensic analysis for the university. It takes time and a lot of experience. The system managers review their systems, and the vice-chancellor must devise a strategy for internal and external communication and assess the legal consequences together with the legal department. We then put the results together to form an overall picture.

Finding a solution in iterative steps

On this first evening, the crisis team meets every hour. We discuss the latest findings and then define new measures. This iterative process continues until the incident has been resolved and the university can return to normal operations. The only thing that changes is the time interval in which the crisis team meets. Soon, the briefing only takes place daily. At the university, the entire process takes two weeks and challenges everyone involved to the utmost, by which point we’re all ready for a holiday.

Over time, a picture emerges of how this incident occurred. Using the data in central log management, Polly’s team can identify systems that may have been the target of the attack. Through forensic analysis of these systems, we are able to establish the following facts: a student reported an attack with Info Stealer malware on his personal device a few months ago. He was then instructed to change all his passwords. Unfortunately, the student forgot to change one password: the VPN access to the university network.

Division of labour in the cybercrime economy

As in the world of work, the cybercrime industry also practices division of labour – after all, you have to specialise to be successful. Presumably, the first perpetrator sold the student’s data and logins captured with Info Stealer on the black market. The next perpetrator bought this data and used the VPN access successfully. Vulnerabilities enabled the second perpetrator to gain access to the account of the above-mentioned system administrator, who has the highest privileges. These perpetrators specialise in the theft of data, the encryption of systems and the extortion of ransom payments. The business model often consists of two forms of blackmail: the first blackmail involves the key that the university can use to decrypt the encrypted systems again. With the second extortion, the perpetrators promise not to publish the captured data.

Seizure of stolen data

As soon as it became clear which attack points had been exploited at the university, it was time to repair the systems or even rebuild them. As the perpetrators had access to the infrastructure for several months, we had to treat all backups as compromised. The crisis team, therefore, decided to rebuild all systems and gradually re-connect them to the internet. It takes about two to three months for everything to return to the way it was before the attack and to guarantee security. Fortunately, law enforcement was also on board. The perpetrators managed to steal internal data from the university, but before they can access the exfiltered data, the police manage to seize their server abroad.

It depends on each individual

With the motivation and commitment of everyone involved, we were able to contain the impact of the attack and protect the university’s infrastructure. It all depends on each individual person and the intensive collaboration because every second and every new insight can change the whole situation. The lessons learnt will be incorporated into preparing for the next IT security incident, which we hope won’t be for a long time.