Downloading large Backblaze restores
My computer was recently destroyed from water damage in a thunderstorm, so I had to recover my backups. In addition to my Time Machine and SuperDuper! backups, I use Backblaze for online backup. I wanted to make sure I have all my backups in order, which meant restoring my Backblaze backup too.
I have about 215 GB of data on Backblaze. For sizes up to 400 GB, they offer to mail you a USB hard drive for $199, but I didn’t want to pay that much when downloading a Zip file is much cheaper. They recommend splitting your restores up into chunks of about 20 GB, but that’s a big hassle because you can only have 2 active restores at a time, and I didn’t want to risk missing anything in my manual splitting procedure. This means actually downloading a 215 GB file from them.
Backblaze cannot reliably serve you files larger than 60 GB or so. I’ve tried it: the connection is always dropped after a few hours. So you will need the Backblaze Downloader, a truly shitty Windows program that is able to resume downloads. (I wasn’t able to resume in any other way, they don’t seem to accept HTTP byte ranges.) Which means downloading on a Windows machine.
I don’t happen to have a Windows box on a fast internet connection with enough disk space, so I set up an Amazon EC2 instance for this. After much experimenting with multiple regions and settings, here is my advice:
- Start a large instance with the ami-67095822 image (Windows Server 2003, instance storage), in the US West region. Your instance will be closer to Backblaze in the Bay Area and the instances there offer gigabit connections. Large instances are expensive but will download your files much faster.
- In your security group, allow all ICMP.
Install Backblaze Downloader and start downloading into either the D: or E: drive. Monitor the download frequently, it’s not smart enough to resume after a disconnection.Backblaze Downloader is too slow. Just download in a regular browser like ChromeFirefox, and pray. You should be able to get speeds of at least 4 MB/s. If not, terminate your instance and start a new large instance.
- Optional: Install VNC (remember to open up port 5900 in the firewall settings, preferably limited to your IP address). You will want to log in frequently and monitor the download, and VNC usually connects faster than RDP.
- Inbound traffic to EC2 is free, but outbound is expensive. Test the archive with 7-Zip before you transfer, it will go through all the checksums. This shouldn’t take too long on a large instance.