After a hack: The process of restoring once-lost data
Brandon Butler (Network World)
On the first Friday in August Mat Honan, a tech reporter with Wired magazine, got home after work and realized that almost his entire personal digital life had been hacked.
His laptop, phone and tablet had been wiped and his Google, Amazon, Apple and Twitter accounts had been compromised. His pictures, videos and other memories, including photos of his newborn daughter and of relatives that had since passed way, were feared gone forever because he had failed to back them up.
But it wasn't so. Honan brought his MacBook Air to DriveSavers, which specializes in data recovery, and after a 24-hour process of engineers diving deep into Honan's laptop, an estimated 75% of the data on his computer that he thought he lost ended up being recovered.
Here's how DriveSavers did it.
IS THE CLOUD SAFE? Recent cloud critics, including Wozniak, intensify debate
DriveSavers has been around for 25 years and has recovered data from a broad range of situations, anything from an iPhone that was dropped in a toilet to a hospital server that has 20,000 confidential patient records on it failing. Getting a personal device that a customer believes has been completely wiped is nothing new for DriveSavers and its team of engineers. Each case is different though, and it's tough to tell how much, if any, information can be recovered from each unique case until engineers get their hands dirty in examining the device, says Chris Bross, senior enterprise recovery engineer with the company.
A few days after Honan's hacking, he brought the device into DriveSavers. The first step is a detailed discussion with the customer, in this case Honan, of exactly what happened and a prioritized list of what workers should focus on recovering, which in Honan's case were photos and videos that he had not previously backed up. "He basically asked us to recover all the data that we could possibly recover," Bross says.
Engineers began by disassembling Honan's MacBook and getting to the heart of where the engineers would do their work: the 250GB Samsung-manufactured solid-state drive (SSD) inside the laptop. Engineers extracted the disk and immediately made a clone of the SSD, along with a backup, so that engineers wouldn't be working directly on the tampered disk.
When making the copy, DriveSavers workers transferred data at the physical layer of the disk, which Bross describes as the lowest layer that includes everything on the disk, both files that have been formatted as well as any empty space that was on the disk. This proved critical later in the recovery process.
The hackers had used a feature in Apple products called "Find My," which is meant to allow users to remotely wipe their Apple devices if they are lost. Using a social engineering attack, they called into the customer service departments of Amazon and Apple posing as Honan, eventually getting his password changed and giving them access to wipe his devices.
The wipe began by deleting index data and installing a new operating system but, luckily for Honan, it didn't get all the way through the wipe before it was stopped. Upon Honan realizing his accounts were being compromised, he turned off his home router, disconnecting his laptop from the Internet, a move that Bross believes may have ultimately saved his data. Still, when Honan later turned his laptop back on after the attack, none of his files were there. Even the recovery experts initially were worried the data may be lost. "We saw a lot of zeros when we first started scanning the drive," Bross says.
In reality though, the hack had only gotten about a quarter the way through the disk, meaning that about 60GB of the 250GB drive had been affected. This included the logical layer of the disk, which organizes all of the media into files, which is why it appeared to Honan upon an initial review that all his files had been lost.
Bross compares it to having a several-hundred page book. When Honan and the engineers first turned on the computer and looked for the files, the table of contents and the first dozens of pages of the book had been wiped and were blank. The deeper they got into the book though, the more data they began to find. Underlying hex data that makes up those files was still on the disk, which DriveSavers engineers were able to leverage for the recovery. "As soon as we started seeing that raw hex data, we knew we were going to be able to recover at least some files," Bross says.
If Honan had been delayed by just 10 or 20 minutes, Bross believes, the wipe could have been complete and it's possible the entire drive could have cleaned, with even the hex data zeroed out. Instead, engineers were able to recreate the files.
Even with the hex data though, recovery is a delicate process. SSDs have a feature named Garbage Collection, which is an automatic maintenance feature by which the drive cleans itself to maintain optimal performance. Engineers have to be careful when recovering data to not have that information be automatically cleaned up by the GC once it's restored.
The process of actually restoring Honan's data involved combing through millions of blocks of raw hex data and finding clues to piece the files back together. Each file has a signature attached to it identifying it as a photo, video, document or some other type of media. Engineers examined every block of hex data looking for these signatures identifying Honan's photos, videos and documents. The end of each object has a file marker, allowing the engineers to find what they believed was the complete hex data that made up each file.
Using proprietary software, DriveSavers workers were able to remake the media files in their presentable format, such as a JPEG, video or document using the extracted hex data. Additional meta data, or data about the data, revealed to the engineers information about when the file had been created and the source of it. Using this information, DriveSavers engineers were able to chronologically organize the data. They ran a system check to ensure the integrity of each file and manually spot checked files to ensure they were whole.
Honan had specifically asked for the data, once extracted from the device, to be encrypted, which DriveSavers did, and the engineers installed it on another external hard drive. After a full day and night of DriveSavers engineers poring through the data, they got everything they could from the drive and invited Honan in to take a look.
In his own recap of the data recovery process, Honan describes the feeling of seeing the files he once thought might be lost: "DriveSavers called me to come look at what they had found, and my wife and I drove up there on Wednesday morning. My data came back to me on an external hard drive, organized by file types. The thing I cared most about, above all else, was my photo library. And there, in a folder full of JPGs, was photo after photo after photo that I had feared were gone forever. Subfolders were organized by the year, month and day files were created. I went immediately to the folder that bore the date my daughter was born. They were there. Everything was there. We were floored. I nearly cried."
Bross says Honan, ironically, was lucky for a victim of a hacking incident. The wipe was interrupted before it completed, which allowed the hex data and metadata to be used by engineers to recover the photos, videos and documents.
Other circumstances could have doomed the recovery too. OS X Lion and Mountain Lion have a feature named File Vault 2 as an optional service, which automatically encrypts any files stored on the SSD. Honan hadn't enabled the feature on his MacBook, but if he had, DriveSavers would have found the hex data and metadata, but it would have been encrypted, without access to the keys to decrypt it, and the files would have been lost.
Honan says the lessons learned from his situation are multifold. First of all, he says he's a "backup believer now," storing his data both locally and redundantly, backing it up with a third party. Second are the policies that companies like Amazon, Apple and Google use to verify customers that call into their service departments to reset passwords and login credentials.
Honan's story seems to have caught the attention of many in the industry and has even led to Apple and Amazon changing some of their customer service policies around account access.
But Bross says the issue comes down to users taking appropriate steps to protect their own important data. "This whole story first and foremost is about security," Bross says. "This happens every day, it's just not that everyday it's a technology reporter that gets hacked."
Network World staff writer Brandon Butler covers cloud computing and social collaboration. He can be reached at BButler@nww.com and found on Twitter at @BButlerNWW.
Read more about data center in Network World's Data Center section.