When maintaining a secure server, make sure to take the following precautions: Keep your software up to date, use strong passwords, follow the principal of least privilege and employ defense in depth.

Sometimes the precautions don’t always work. No one ever wants their server to be hacked or compromised, but unfortunately it can and does happen. With a little preparation, you can quickly respond to a hacked server so you can smoothly recover your server – ultimately limiting the amount of data exposed or lost and minimizing your downtime. Below are 6 steps that you should immediately do after your server has been breached; they don’t have to be completed in order but will definitely help in mitigating future risk.

Step 1: Know Who is Involved – Of everyone responding to the security threat, make sure everyone knows what their own role is and what the roles of others are. Understanding how to communicate amongst each other will help all of the subsequent steps.

Step 2: Assess the Scope – Understand what has already been observed, what has already been done and what is currently happening. This quick assessment will help determine which steps should be performed next and in what priority to prevent future network security issues.

Step 3: Contain the security breach – These immediate steps will help prevent further damage or information loss. If the attacker is actively on your system, you may need to kill their processes immediately to limit further damage. This may involve blocking the network to a specific block of IP addresses, especially if data is actively being streamed off of the server to the attacker’s machine.

Step 4: Maintain State – Your classification and impact analysis of the compromised system will help determine if the security breach should be contained or the state maintained first. For further forensic work, it is best to leave the system unaltered (or as unaltered as possible). You can copy log files or take an image of the disk for further analysis. This also prevents the attacker for altering the logs.

Here is a simple example of how to clone a block device or partition: On the secure system, run nc -l -p 9999 | dd of=/dev/incident-xyz.dd. Then on the compromised system, run dd if=/dev/sda | nc <ip of secure system> 9999. You can also use dcfldd, which provides hashing on-the-fly among other useful features. To limit the amount of empty space being transferred, you can also use gzip between dd and netcat on both servers.

Step 5: Notifications – Follow the appropriate process for notifying the impacted, may include PCI or HIPAA regulations or federal and state laws depending upon the nature of the data exposed (or potentially exposed).

Step 6: Remediate – Putting the same vulnerable hacked server back online will likely only lead to another incident, so any workarounds or patches should be applied before bringing services back online. This may include data integrity checking or a complete system re-install.

Throughout this process, make sure you communicate with the rest of the team, keep a cool head, make sure you document what you are observing, and what actions are being taken with your network security issue. If you believe your server has been compromised and still don’t know what to do let us know, we can help.