Category: Backup

AWS Glacier!

Hard drives will fail!

Manufacturers of hard drives like to boast about “MTBF” (Mean Time Between Failures) and “AFR” (Annualized Failure Rate) rates that indicates that the drive would last anywhere from 300 000 hours (~34 years) to having a likely hood of failing during a year as low as 0,8%.

But real life tests, performed by companies like BackBlaze show that disk drives can fail already after 1 year, actually they had a 5.1% AFR during the first year, 1.4% between 1,5 years to 3 years and a staggering 11.8% between the third and forth year. Their study also shows that after 4 years, it was only 80% of the original disk drives that where still “alive”. Meaning that 20% of the drives had failed after 4 years. 

So, why does this matter?

I have videos, photos and important documents that I need to backup and in some cases archive. Archiving being “removing the local copy and storing the item in a secure place” (think of book keeping, where it’s required to archive the books, it’s not required that you have multiple copies in case of a fire), where as backup is a second copy of the object in another location in case of a fire. We’re talking wedding photos, wedding videos, photos of my son growing up etc. that, in the case of an unforeseen catastrophic event in my home, could potentially be lost for ever!

Companies like BackBlaze offer products for backing up your local machine (and also files stored on a server, like in my case), but I optioned for another setup using Amazon Web Services S3 Glacier!

AWS Glacier

So what is S3 Glacier? Glacier is an archiving service, where you can upload all you data and pay a very small service fee per month for Amazon to keep your data safe. Amazon S3 Glacier is designed to provide average annual durability of 99.999999999% for an archive. The service redundantly stores data in multiple facilities and on multiple devices within each facility. To increase durability, Amazon S3 Glacier synchronously stores your data across multiple facilities before returning SUCCESS on uploading archives. S3 Glacier performs regular, systematic data integrity checks and is built to be automatically self-healing.

My setup consists of a software running on my local NAS that monitors specific places on the NAS where I can put important files and documents, these are then uploaded to Amazon S3 Glacier.

Putting files into Glacier is actually quite easy and the price for storing your data in Glacier is very modest, it’s when you want to retrieve the files that you might need to pay (it also depends on how much data you want to retrieve and how fast you need your files).

As there is no GUI for exploring AWS Glacier Vaults (where the files are stored), I found a really great application online called Freeze that makes life much easier for anyone who wants to explore their files and retrieve them from a UI. 

So are you in the market for a simple, easy to use and very secure way of storing files, documents, pictures, videos etc. of your beloved ones, and you want to be sure that even if your house burns down you will still have access to those files? Try out AWS Glacier with the Freeze App!

(This is not a payed advertisement, BackBlaze, AWS and Freeze App have not been consulted and/or informed about this blog post)

Backup you VPS to Google Drive

If you only have a small amount of data to backup, Google Drive is actually quite a decent alternative, but you will need to install a few tools to you VPS.

With the tool rclone (rsync for cloud), you can upload your files to Google Drive, Amazon S3, Dropbox and a few more. The procedure is almost the same no matter of the cloud provider you go for.

Just goto: http://rclone.org/

Install rclone as described and then setup a cronjob to sync the files that you want to Backup.

My crontab to backup the automysqlbackup folders:

# m h  dom mon dow   command
  0 *   *   *   *    export PATH=$PATH:/home/siho/bin && rclone -q sync /var/lib/automysqlbackup/ remote:Backup/automysqlbackup