Non-cloud backup

All things storage here: hard drives, DVD RW drives, little wicker baskets.

Moderators: morphine, Steel

Non-cloud backup

Postposted on Tue Jul 23, 2013 1:51 pm

So in a photography forum I'm on, another user had a problem with these requirements:

1. The tool has to monitor any changes to the source files as I make them, so it must run at all times as a service or a start-up application.
2. It must support networked drives.
3. Because I do not want to keep my USB drives connected and running at all times, the tool has to remember the changes it monitors, and presumably store instructions in its own database until I choose to connect a destination drive and perform the sync... and it has to be smart enough to read and write only as much data as necessary.
4. It has to run under Windows 7, it has to be safe (no malware or spyware), and it has to be reliable.
5. I'll pay for it if necessary, but of course free is better.
6. I'm not interested in cloud storage.

I am a bit stumped as to how to do this in a non-cloud storage system. So, any ideas? They are indeed willing to pay for this. Its 2TB of data and growing.
Sony a7
Sony Zeiss 55/1.8 SSM, 24-70/4 SSM
Minolta 17-35/2.8-4 D, 35-105/3.5-4.5, 100-300 APO
TheEmrys
Minister of Gerbil Affairs
Silver subscriber
 
 
Posts: 2068
Joined: Wed May 29, 2002 7:22 pm
Location: Northern Colorado

Re: Non-cloud backup

Postposted on Tue Jul 23, 2013 2:12 pm

FileHamster or a robocopy script comes to mind. #3 may be a bit tough to come by.
Flatland_Spider
Gerbil Elite
 
Posts: 721
Joined: Mon Sep 13, 2004 7:33 pm
Location: The 918/539

Re: Non-cloud backup

Postposted on Tue Jul 23, 2013 2:34 pm

Robocopy has a /mon (monitors for specified number of changes, then tries to copy) or /mot (monitors for changes over the specified interval, then copies). The retry switches (/r specifies the number of times to retry on failure) (/w specifies how long to wait between retries).

While this is kludgey, it should work, copying changed files once your drive is reconnected. Specification of /r and /w parameters will depend on how often you expect the drives to be connected/disconnected.

The /z might also be helpful, allowing copying to be restarted. This consumes overhead, but can be useful if you are going to be taking the drive in and out a lot.

You can also try /TBD, which waits for a share to be reconnected before attempting to copy. I don't think this works for local drives (which USB drives are) but you could try to get around it by accessing the USB drive as a UNC path (for example, if the drive is R:, you could try \\yourpcnameORip\R$).
Last edited by mattshwink on Tue Jul 23, 2013 2:44 pm, edited 1 time in total.
mattshwink
Gerbil
 
Posts: 73
Joined: Wed Jul 16, 2008 6:54 am
Location: Alexandria, VA

Re: Non-cloud backup

Postposted on Tue Jul 23, 2013 2:44 pm

EDIT: I just realized this is probably a single-machine setup, in which case BTSync might not work well for you... you might have to resort to scripts as mentioned above. Sorry about that. You might try SyncToy, I've heard good things but never used it. http://www.microsoft.com/en-us/download ... x?id=15155

Honestly have you tried Bittorrent Sync? It's free, stable, and actively developed.

I am able to back up remotely to my own home server when I am programming across different devices. I have a desktop/laptop that I program on, and my changes are synced in near-real time (I have it set to a 10m delay) to the server. When I am on the go, I can program, go home, then fire up my desktop, and pretty much pick up where I left off. I also use BT Sync for backing up my photos (I've got just under 100GB of those, it works just fine).

The initial sync takes awhile, but afterwards everything is differential. It's easy to set up, free, cross-platform, and relatively customizeable.

http://labs.bittorrent.com/experiments/sync.html
Duct Tape Dude
Gerbil
Gold subscriber
 
 
Posts: 99
Joined: Thu May 02, 2013 11:37 am

Re: Non-cloud backup

Postposted on Tue Jul 23, 2013 2:52 pm

I'd also try duplicati. It's free, and while it's primary purpose is cloud, it can do local and network drives as well. It also does incremental backups. Create a job to run at periodic intervals and it should only pick up changed files since the last successful backup.
mattshwink
Gerbil
 
Posts: 73
Joined: Wed Jul 16, 2008 6:54 am
Location: Alexandria, VA

Re: Non-cloud backup

Postposted on Tue Jul 23, 2013 3:09 pm

Have you looked at Crashplan? It's been discussed a number of places on TR and the forums before so maybe you've considered and ruled it out but I thought I would make its case for you.

1. You can set it to backup changes as frequently as every minute - i think that would meet your requirements

2. Network drive as source or target? I think you can overcome both (possibly) but I'm not sure how the unofficial solutions meet your needs:
http://support.crashplan.com/doku.php/r ... ped_drives
http://virtualj.net/20120126/use-crashp ... -drive-nas
http://homeservershow.com/forums/index. ... re-solved/

3. Maybe I'm misunderstanding what you want, but this one might be a hard target to hit. So you want to track all changes (with versioning on all?) and only backup when you connect a drive? I don't know if it works how you want - you can certainly have multiple folder destinations on different drives and when you connect them, the program will sync any changes between the present and what is stored on the destination and make block level changes. If you want to track any/all changes I think you're going to need some kind of always on/connect backup destination to sync changes with. For example, I use it at work and it backs up our main database, quickbooks files, etc. to a local drive as well as the cloud every 15 minutes and there is a larger set of items that get backed up overnight. The backup takes all of 3 minutes to make the block level changes to what are 800mb+ files. If you pay for Crashplan+ you get unlimited cloud backup (which you appear to not need) but also the ability to have multiple backup sets. I wonder if you need all 2TB of data backed up/monitored constantly or if you have a smaller data set that could use an always on destination with another set that could be on a daily schedule or as you connect the other drives (more like an archive/cold storage set). The external drives I have (WD and Seagate) are both smart about spinning down when they haven't been accessed for a while if your concern is power usage.

4. I've run it under XP, Vista, 7, 8, Server 2003, and 2012. I had an issue with my 2012 install and support has been great and I think it's fixed now (something with Java and a reinstall seems to have fixed it).

5. Tons of functionality in the free edition (including offsite backup to other computers, friends, family, etc.) and even more with unlimited cloud storage in the paid version.

6. Completely optional

Good luck!
i7-3770K | Asus P8Z77-V LK | 8GB DDR3-1600 | HD5850 | 128GB 840 Pro | Samsung F3 1TB | U2412M | Define R4 | Seasonic 520W M12II | Win7 Pro x64.
frumper15
Gerbil Team Leader
Silver subscriber
 
 
Posts: 224
Joined: Mon Jan 18, 2010 2:25 pm

Re: Non-cloud backup

Postposted on Tue Jul 23, 2013 4:25 pm

Duct Tape Dude wrote:EDIT: I just realized this is probably a single-machine setup, in which case BTSync might not work well for you... you might have to resort to scripts as mentioned above. Sorry about that. You might try SyncToy, I've heard good things but never used it. http://www.microsoft.com/en-us/download ... x?id=15155


SyncToy isn't very good. I've found SyncBack to be better.
Flatland_Spider
Gerbil Elite
 
Posts: 721
Joined: Mon Sep 13, 2004 7:33 pm
Location: The 918/539


Return to Storage

Who is online

Users browsing this forum: No registered users and 2 guests