Solved

Smart Sync or Binary Diff Tool

Posted on 2010-09-01
6
665 Views
Last Modified: 2012-05-10
We have the need to transfer copies of our builds to India for testing. These builds are about 4G in total but have huge amounts which do not change between files.
So the build contains a combination of many files and big CABs (or equivalents) . Some of the binaries might only change by a few bytes for the version stamping.

=> Is there a copy or sync tool out there which is really smart and can build remotely (either directly or by a copy and process) a copy of a file area based on a "seed" area. This seed area on the remote site would simply be a copy of the previous build ? It would need to recognise the common parts in the binary files.
Context : Windows Server 2003 / 2008

Is this essentially Microsofts DFS Replication ?

Now a normal sync tool would be file for the stand alone files that don;t change but hopeless for a very large binary which only has say 2% of the file changed. It would have no choice but to copy the large binary file.

Thanks
0
Comment
Question by:greg_roberts
  • 3
  • 2
6 Comments
 
LVL 3

Accepted Solution

by:
AndyEastham earned 125 total points
ID: 33596434
Hi,

Have you looked a rsync?
http://www.samba.org/rsync/

Or for windows:
http://www.aboutmyip.com/AboutMyXApp/DeltaCopy.jsp

Andy
0
 
LVL 18

Expert Comment

by:BigSchmuh
ID: 33596493
Most international development team use some Repository tool ( like "SubVersioN" ) that allows :
-any "client" to replicate locally all (or a branch) of it (using CheckIn/CheckOut methods to avoid data write conflicts)
-versioning all files in the Repository
-a global Repository backup

You may although use a replication system like DFS to synchronize a directory in 2(+) locations but it requires sharing some Domain info

If your build does change all files at every new build, both Win2008 R2 DFS and SVN should allow for a differential/incremental evaluation...that is to be tested anyway
0
 
LVL 18

Expert Comment

by:BigSchmuh
ID: 33596521
SVN (SubVersioN)
   http://en.wikipedia.org/wiki/Apache_Subversion
On Windows, I have been using http://www.visualsvn.com/server (and its VisualSVN client) with success
0
Complete VMware vSphere® ESX(i) & Hyper-V Backup

Capture your entire system, including the host, with patented disk imaging integrated with VMware VADP / Microsoft VSS and RCT. RTOs is as low as 15 seconds with Acronis Active Restore™. You can enjoy unlimited P2V/V2V migrations from any source (even from a different hypervisor)

 

Author Comment

by:greg_roberts
ID: 33601586
Andy
Thanks. i looked at rsync but could not run. No csyxxx.dll (some library dll to convert between windows to linux)

BigSchmuh
We use TFS 2080 for our source control and subversion is more of a source control tool than a replication tool.

Normal "sync" tools result in the 4G build needing 2G of b/w to copy. I am hoping that a block level trial will reduce this dramatically. Was trialing R1Soft but can't get the s/w to work. This has block level copy feature.
0
 
LVL 18

Assisted Solution

by:BigSchmuh
BigSchmuh earned 125 total points
ID: 33602245
My suggestion is to try using SVN (a source control tool) to replicate a directory locally:
-costless
-should allow for some incremental updates (Unsure with your binaries)
0
 

Author Closing Comment

by:greg_roberts
ID: 33602265
Got rsync working using the url within the url for cwRsync but haven't got all the options right as i end up with the same sync transfer count as other tools (even using --inplace). Suspect issue with file times being used insead of CRC of file content.

As builds have complete binaries and files, exe, pdb, dlls , cab files etc i can't see how the SVN solution will help but thats for contrinuting.
0

Featured Post

Best Practices: Disaster Recovery Testing

Besides backup, any IT division should have a disaster recovery plan. You will find a few tips below relating to the development of such a plan and to what issues one should pay special attention in the course of backup planning.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Hi, I've made you some graphics for a better understanding how RAID works. First of all, there are two ways a raid can be generated: - By hardware - By software What does that mean? This means: If you have a hardware RAID controller, there…
Problem description :  Some external hard disks / USB flash drives do not show actual space as mentioned in the factory settings. This is a common problem when you use an 8 GB USB drive to make it bootable to install a firmware/ driver on a serv…
This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…
This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting. This w…

914 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now