Solved

Script to move only modified content to folder

Posted on 2014-01-20
10
283 Views
Last Modified: 2014-01-21
G'day guys,

This has probably been asked before, but I searched and couldn't find exactly what I was after.

Its a simple thing I'm sure but scripting is not my forte.

Basically I have a client who works away from her server, but comes back in house to that server every Friday of every week.

On the server there is a shared folder that contains all her work, which she has a mirror copy of stored locally on her laptop. Obviously she works on the local copy and then when she comes back on Friday she needs to copy all the modified data to the servers copy.

Now my question is would it be a good idea to use something like Bit Torrent Sync or Wuala to keep these in sync? Or is a script just as good?

The total folder size currently is 4GB, and it contains a lot of static unchanging files. I'm guessing that she only edits a fraction of those files around 100 MB per month maximum.

The other option is to use a simple script that she just runs each time she comes into the office.

Ideas are appreciated guys thanks in advance

Steve
0
Comment
Question by:sjswarts
  • 4
  • 3
  • 2
  • +1
10 Comments
 
LVL 8

Expert Comment

by:Surrano
ID: 39796241
What client OS are we talking about?
If you already have some backup solution in place that supports clients of the proper OS type, I recommend creating a new backup job specific for that client that takes incremental backups every so often.
We use bacula: http://www.bacula.org/en/
It is open source so theoretically supports any and all OS's but practically it's a day's work or two to
- install and setup a Linux (or any other Unix) server
- learn how to configure tape and disk backup media
- configure a windows client with arbitrary schedule

You can also start jobs manually so given sufficient privileges she can initiate the backup for her very own client at the time of her choice (i.e. before leaving workplace)
0
 

Author Comment

by:sjswarts
ID: 39796573
Thanks for the response. I have looked into Bacula awhile ago, but haven't set up anything for a long while now.

This is the system as it stands:

Server: \\*SERVER*\data\folder_to_sync
Client: C:\folder_to_sync

Now what I setup today was bittorrent sync which uses a P2P system. One install on the server, one install on the client.

Very easy software to use, plus its fully secured, the data stays locally and its great because it is free even for commercial.

But even though that is how I have got around the issue, would anyone know how to use an actual script? Or be able to point me in a direciton where I can learn off tutorials??

cheers,
Steve
0
 
LVL 20

Expert Comment

by:SelfGovern
ID: 39796843
Just a couple of suggestions on cautions you need to take.

1) remember that if you sync, you'll be syncing unwanted changes (deletions, virus infections, etc.) as well as the ones you want to be syncing.  You need a backup, not just a sync, so you can go back to a week or two ago if you don't realize for a few days that something bad happened.

2) Make sure that if there is *any* chance for the files in the office to be changed other than through your sync process, that you have some protection built in to prevent those wanted changes from being overwritten.
0
 

Author Comment

by:sjswarts
ID: 39796921
Thank you for your post. You hit the main issues exactly.

The server does backups using Windows Server Backup every 1/2 day. So that gives us some grace.

As for the issue with people editting the same file, I don't know
0
 
LVL 20

Expert Comment

by:SelfGovern
ID: 39797245
Robocopy has options for checking periodically to see if files have changed, and for copying only changed files.  You can create a batch file with the necessary command-line switches, and let it check periodically throughout the day.   Should be unobtrusive, and do what you need when the employee is in the office.
0
Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

 
LVL 15

Expert Comment

by:ZabagaR
ID: 39797881
I use Robocopy to perform copying, mirroring, synching. It can be easy up to complex.

An easy script, for example, is below at the bottom

This is a batch script that runs as a scheduled task on a server. It maps a folder as a network drive.  The folder is called "scanimg" on the laptop.  The robocopy copies only new or changed content to my server's d:\rximages\ folder.   /R:3 = retires 3x to copy a file and /W:5 is wait 5 seconds between retries. /LOG stores a log file of results.

I'm mapping the drive without credentials because my user launching the task exists on the laptop with the same password. Otherwise it would prompt which is bad for automation! You could always add /user:joesmith password_here to the net use statement if you didn't mind keeping that in the open.

test.bat:

net use u: \\laptop\scanimg
robocopy U:\ d:\RxImages\ /S /R:3 /W:5 /LOG:d:\mylog.txt
net use u: /d


I have other more complex examples if you want...what OS would the Robocopy run under? You can also have Robocopy completely mirror folder A on machine A to folder B on machine B....while just copying new/updated files (including mirroring deletes).
0
 

Author Comment

by:sjswarts
ID: 39799076
Thank you ZabagaR,

that is what I'm after.

The server OS is SBS 2011 and the client is Windows XP currently. The client is using a temporary laptop and will be replacing that with a Windows 7 in the next month.

I tried reading through the robocopy manual but I couldn't find the switch that only moved modified data. Obviously I am blind.

Does the switch /s do that because according to the Microsoft site it says:

/s - Copies subdirectories. Note that this option excludes empty directories.

Like you have mentioned

/r:# - amount of retries
/w:# - wait time for retries

On their website it doesn't mention copy only modified
0
 
LVL 20

Accepted Solution

by:
SelfGovern earned 500 total points
ID: 39799114
You want to check the Archive bit, which is done with the /a or /m flag.  I suggest /m, which will reset the archive bit (so that the file is not copied again unless it is changed again).

If you don't include the /s flag, you'll only copy files at the current directory, none of the subfolders.  the /s flag has nothing to do with whether files are changed, only with whether robocopy looks just *here*, or if it also looks to find matches deeper in the directory tree.
0
 

Author Comment

by:sjswarts
ID: 39799118
Ok fantastic, thankyou mate.

I was confused with the terminology used. But according to wiki ...

Archive: When set, it indicates that the hosting file has changed since the last backup operation. Windows' file system sets this attribute on any file that has changed. Backup software then has the duty of clearing it upon a successful backup.

So now I understand.

Thank you muchly this is what I was looking for.
0
 
LVL 8

Expert Comment

by:Surrano
ID: 39799155
wow an excellent solution indeed. Reminds me of Occam's razor.
0

Featured Post

Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

Join & Write a Comment

This article is an update and follow-up of my previous article:   Storage 101: common concepts in the IT enterprise storage This time, I expand on more frequently used storage concepts.
The Delta outage: 650 cancelled flights, more than 1200 delayed flights, thousands of frustrated customers, tens of millions of dollars in damages – plus untold reputational damage to one of the world’s most trusted airlines. All due to a catastroph…
The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

22 Experts available now in Live!

Get 1:1 Help Now