Credit: Dreamstime
As the number of places where we store data increases, the basic concept of what is referred to as the 3-2-1 rule often gets forgotten.
This is a problem, because the 3-2-1 rule is easily one of the most foundational concepts for designing data protection. It's important to understand why the rule was created, and how it's currently being interpreted in an increasingly tapeless world.
What is the 3-2-1 rule for back-up?
The 3-2-1 rule says there should be at least three copies or versions of data stored on two different pieces of media, one of which is off-site. Let's take a look at each of the three elements and what it addresses.
- 3 copies or versions: Having at least three different versions of data over different periods of time ensures that users can recover from accidents that affect multiple versions. Any good back-up system will have many more than three copies
- 2 different media: Users should not have both copies of data on the same media. Consider, for example, Apple's Time Machine. Users can fool it using Disc Utility to split their hard drive into two virtual volumes, and then use Time Machine to back-up the first volume to the “second” volume. If the primary drive fails, the back-up will fail as well. This is why users always have the back-up on different media than the original.
- 1 back-up off-site: A speaker at a conference once said he didn't like tapes because he put them in a box on top of a server, and they melted when the server caught fire. The problem wasn't tape; the problem was he put his back-ups on top of his server. Back-up copies, or at least one version of them, should be stored in a different physical location than the thing that is being backed up.
Mind the air gap
An air gap is a way of securing a copy of data by placing it on a machine on a network that is physically separate from the data it is backing up. It literally means there is a gap of air between the primary and the back-up. This air gap accomplishes more than simple disaster recovery; it is also very useful for protecting against hackers.
If all back-ups are accessible via the same computers that might be attacked, it is possible that a hacker could use a compromised server to attack the back-up server. By separating the back-up from the primary via an air gap, users make it harder for a hacker to pull that off. It's still not impossible, just harder.
Everyone wants an air gap. The discussion these days is how to accomplish an air gap without using tapes. Back in the days of tape back-up, it was easy to provide an air gap. Users made a back-up copy of data and put it in a box, then handed it to an Iron Mountain driver.
Instantly, there was a gap of air between primary and back-up. It was close to impossible for a hacker to attack both the primary and the back-up.
That is not to say it was impossible; it just made it harder. For hackers to attack the secondary copy, they needed to resort to a physical attack via social engineering. You might think that tapes stored in an off-site storage facility would be impervious to a physical attack via social engineering, but that is definitely not the case. I have personally participated in white hat attacks of off-site storage facilities, successfully penetrated them and been left unattended with other people's back-ups.
Most hackers don’t resort to physical attacks because they are just too risky, so air-gapping back-ups greatly reduces the risk that they will be compromised.
Faulty 3-2-1 implementations
Many things that pass for back-up systems now do not pass even the most liberal interpretation of the 3-2-1 rule. A perfect example of this would be various cloud-based services that store the back-ups on the same servers and the same storage facility that they are protecting, ignoring the “2” and the “1” in this important rule.
For example, it is very common for customers of public cloud vendors to back-up their systems by creating snapshots/images of the resources they are using. The images are typically stored in object storage in the same account that is running the primary systems.
If hackers gain privileged access, they could easily delete both the primary and all secondary copies of the data. The 3-2-1 rule still applies to the cloud. Keep a copy somewhere else – in a different account, in a different availability zone – just keep it somewhere else.
The 3-2-1 rule is also ignored by a lot of people using hundreds of SaaS services. Consider, for example, the advent of Kubernetes and the reality that many people store their Kubernetes configuration in GitHub. Important back-ups are stored in a system that users may or may not be backing up.
Consider other services like email providers or file-sharing services where even the primary copy of data is stored only in a third-party vendor’s platform. The back-ups in many of these services are simply additional copies of data in the same location. Users must also ask vendors how they would help recover if an entire account was hacked by a third-party.
What about electronic air gaps?
A purist would say that the only way to have a true air gap is to put back-ups on removable media such as tape and then physically separate them from the primary. Others acknowledge that many companies have moved on from tape as a protection mechanism and might use it only for long-term storage if they use it at all.
The question is how to make sure a hacker can’t access the primary and the secondary via an electronic hack.
The current best answer is to separate these two copies in as many ways as possible. Consider doing as many of the following:
- Different storage: Use a different storage type than what is used for primary storage. An attack designed for one will probably not work on another
- Different environment: Use a back-up system that isn’t directly reachable via LAN. That’s another way to prevent compromised on-prem servers from attacking back-ups
- Different OS: Use a back-up server or service that runs on an OS other than Windows can go a long way. Most ransomware attacks have been against Windows
- Different account: As much as possible, use completely different credentials in back-up and disaster-recovery systems. That way if an account is compromised, the credentials won’t work to attack back-ups
- Immutable storage: Some cloud vendors offer immutable storage, where back-ups sent there cannot be changed or deleted until the time you specify. Even you can’t delete them.
The 3-2-1 rule is a good rule that has served the data-protection world well for a long time. Always ask how well users are complying with it; it just might save your bacon one day.
