Automated backup using Amazon S3?
Apr 9, 2006 by
Andre
A really nice application for Amazon's S3 would be a scheduled backup service. As long as I'm wishing someone would write this, here's my feature wish-list:
On a related note, I came across an effort by Elliot Smith to build a Rails front-end to S3. Still in the proof-of-concept stage, but something to keep an eye on.
- Select multiple folders to backup
- Only backup changed files (since you pay for bandwidth)
- Specify conditionals and/or regex's for the files you want backed up
- Runs in the background; only alerts me if it wasn't able to complete a scheduled backup
- Local encryption before sending it to the S3 servers
On a related note, I came across an effort by Elliot Smith to build a Rails front-end to S3. Still in the proof-of-concept stage, but something to keep an eye on.
Comments
Dan on Apr 10
Brad at SixApart/LiveJournal has created Brackup in Perl, dependent upon the the Net::Amazon::S3 module.
Elliot Smith on Apr 11
Thanks for the name check. After messing with my Rails S3 front end, I've started rewriting the S3 client side to make it more robust, and have been thinking about how to write something like rsync to write to S3. Brackup is interesting in this respect. But I'd like to have a single library I could use with my Rails front-end and my rsync clone, hence my separate effort. I think the snapshot idea is the one to go for, which is what rsync uses. If I was really clever, I'd write a Linux filesystem layer for S3 (but I'm not). Another approach would be to go for the WebDAV layer approach (http://www.carion.org/s3dav/index.html).
Cheers.
Cristiano Dias on Jul 11
There are already some "Linux S3 filesystems" out, aren't there? If I am not mistaken you can mount a S3 bucket with jungledisk.com.
You guys could also check this out:
http://developer.amazonwebservices.com/connect/thread.jspa?threadID=10271&start=0&tstart=0
I haven't used any of those yet. ;-)