My Cloud EX2 Backup to Amazon S3

By Brian

October 4, 2015

With all the devices in the house it was finally time to invest in a NAS. I settled on the Western Digital My Cloud EX2. I picked this specifically because it supported back up to Amazon S3. In practice, the backup software sucks and I had to work around a few issues to get it working reliably and inexpensively.

Overall I really like the EX2. It has great features for the price. My version came with two 4TB drives which I configured to mirror for redundancy (you can forgo redundancy and get 8TB of storage). The EX2 supports SMB and NFS. It can act a DLNA (I use an app called Vimu Player on my Fire TV) or iTunes server (unprotected audio only). For the more advanced user, you can also join Active Directory, act as an iSCSI target, and mount ISO images. The EX2 can backup to another EX2, Elephant Drive or Amazon S3. The rest of this post focuses on backup to S3 which is less than perfect, but with a little effort I have it running reliably.

Backup

At a high level, I want the back to protect me from three things: 1) Hardware failure. The EX2 has two disks, but I still want a more protection. 2) My own stupidity. I might accidentally delete or overwrite something. 3) Malware. Most notably CryptoLocker or similar ransom ware. The backup agent built into the EX2 offers three backup types (taken from here):

Overwriting existing file(s): Overwrites files in the target folder that have the identical name as your source file.
Full Backup: Creates a separate folder containing all of the backup data each time the backup is performed.
Incremental Backup: Overwrites files with source files that are newer then the target files.

I wanted the third option, and this is what I am running. Unfortunately, it does not work as advertised. Every once in a while it overwrites files that have not changed. This would be not a big deal, but I want to run versioning to protect against malicious malware overwriting my files. With versioning enabled, S3 stores every version of your files so you can always roll back to an old copy.

The problem is that the EX2 keeps adding versions. Over the past six months it has created as many as 10 copies of a file that has never changed. This has driven my bill up dramatically. To keep my bill in check I resorted to a lifecycle policy that moves my files to glacier and removes old versions after 30 days. Glacier is much cheaper and and 30 days gives me enough time to fix a mistake.

Configuration

The first thing I created was an S3 bucket. There is noting special here, just accept the defaults. Then, I created the lifecycle policy described above. The configuration looks like this:

Next, I needed an IAM user for the backup job on the EX2. I created a user policy that had only those rights needed by the backup job. This way, even if my EX2 were compromised, the attacker could never delete from my bucket or access other resources in my account. My policy looks like this.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26


{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:ListAllMyBuckets",
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation"
            ],
            "Resource": "arn:aws:s3:::BUCKETNAME"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject"
            ],
            "Resource": "arn:aws:s3:::BUCKETNAME/*"
        }
    ]
}

Finally, I could configure the backup job on the EX2. The configuration above has been running for a while now. It still overwrites files that have not changed, but the lifecycle policy keeps them under control.