Thursday, June 4, 2020

Great news, and advice needed

Last week, we were devastated with a Ransomware attack. Because of some key errors on my part, as well as the QNAP's Linux operating system requiring more expensive options for cloud backup, we didn't have a robust backup and lost nearly everything.

QNAP's malware cleaner identified the problem as the ech0raix ransomware. A second lab that I sent files to identified it as QNAP.encrypter ransomware. The ransomware explicitly targets QNAP products, so I contacted QNAP, and they were no help at all. I contacted several data recovery companies, and they all told me that there was nothing we could do.

 I did research on the ransomware and discovered what I suspected was a key flaw in how the ransomware operates. It encrypts the file and saves a copy with the ".encrypt" extension appended to the end of the file name, and then it deletes the unencrypted original file. Everything that I had read stated that the decrypting the encrypted files was impossible, but I once had software that allowed me to undelete Windows files even after a disk format, so why wouldn't the same be possible on a Linux system? Why try to decrypt a strongly encrypted file when you have an unencrypted file there just waiting to be recovered?

Since the two NAS servers (primary and backup) were RAID1 arrays, we had 4 copies of the entire system. We decided to see what could be done. We put three of the disks in the safe, and sent the other off to a friend that works in IT for a large company. He made a bit for bit copy, and then took that copy and tried to recover the deleted, unencrypted files.

He successfully recovered over 12,000 files. He recovered pictures, videos, Microsoft Office files, and PDFs. There were a few files that were infected and had to be destroyed. Some were damaged by being overwritten. He recovered more than 90% of what was on there.

The hard drives. He says that there is no guarantee that the malware isn't hidden somewhere on the drives to the point where even formatting won't get rid of it, and with the low cost of HDD now, we are going to replace all of them with new, out of the box drives. The NAS is probably going as well. QNAP's products are being targeted, and apathetic is the kindest thing I can say about them.

Everyone I talked to said it couldn't be done. Our friend didn't want payment, but we are giving him $500 for what work he did, even if I have to break into his car to hide it in the glovebox. We sent him a passport drive so he can put our recovered files on it. The directories were all lost, so we have some sorting and filing to do.

Now I do need advice from my readers. I am changing our file storage system here. I want to use a NAS for file storage but also backup. We are going to keep offline copies of everything through the use of periodic backups on disks that we will keep in the safe AND cloud backups. I want something that is easier to understand than having to do all of the workarounds that Linux requires and allows cloud backups at a lower cost than Linux. So here are my requirements:

1 Network drive with RAID capability,
2 Capable of periodic updates for security
3 Capable of running Antivirus software that doesn't cost an arm and a leg
4 Capable of automated cloud backups of either the entire drive, or selected directories
5 Cost less than $400 without hard drives

Does anyone know of such an animal?





4 comments:

Aaron C. de Bruyn said...

> Everyone I talked to said it couldn't be done.

My comments must have been moderated into oblivion. ;)

I think item 5 on your list is going to prevent you from getting a reliable system. Check out the ixSystems FreeNAS Mini. It's a bit outside the price range you listed, but it's basically the same software as the enterprise storage stuff I use, except on a smaller machine... and I recovered 21 offices that were fully encrypted with cryptolocker in under 30 minutes. Simultaneously.

The consumer-brand crap is what made you vulnerable in the first place.

The FreeNAS Mini has multiple network interfaces. This lets you run the management tools on one interface that you can leave unplugged, and the file sharing services on another interface that your computer can talk to. If you ever get infected again, unplug it completely from the network, plug a known good computer into the management interface, and then roll back your files. You can even run tarsnap on the NAS to ensure you have a cloud-based backup.

Divemedic said...

I will look, thanks.

Therefore said...

FreeNAS is a very good product built on FreeBSD with ZFS file system.

They added a good browser based admin interface.

I bid one with 50ish 8tb drives for a contract.

Add Amanda on top for cloud based backups.

Congrats on the recovery. As a Unix person "undelete" is something I never consider as the older filesystems just made it impossible. And on my current system I run a background scrubber that does a triple write on deleted blocks. If I say "delete it you mf" I want it gone.

Backups are encrypted. If I need it back then I pull from backups.

Good luck to you. If you want assistance on a self built NAS just reach out to me.

jwl said...

This is great news, congrats!

We've been using a Synology DiskStation 218j and have been pretty happy with it. For the most part it just seems to work. But then, I'm not particularly tech-savvy in this particular area.