Solved

Smart Sync or Binary Diff Tool

Posted on 2010-09-01
6
656 Views
Last Modified: 2012-05-10
We have the need to transfer copies of our builds to India for testing. These builds are about 4G in total but have huge amounts which do not change between files.
So the build contains a combination of many files and big CABs (or equivalents) . Some of the binaries might only change by a few bytes for the version stamping.

=> Is there a copy or sync tool out there which is really smart and can build remotely (either directly or by a copy and process) a copy of a file area based on a "seed" area. This seed area on the remote site would simply be a copy of the previous build ? It would need to recognise the common parts in the binary files.
Context : Windows Server 2003 / 2008

Is this essentially Microsofts DFS Replication ?

Now a normal sync tool would be file for the stand alone files that don;t change but hopeless for a very large binary which only has say 2% of the file changed. It would have no choice but to copy the large binary file.

Thanks
0
Comment
Question by:greg_roberts
  • 3
  • 2
6 Comments
 
LVL 3

Accepted Solution

by:
AndyEastham earned 125 total points
Comment Utility
Hi,

Have you looked a rsync?
http://www.samba.org/rsync/

Or for windows:
http://www.aboutmyip.com/AboutMyXApp/DeltaCopy.jsp

Andy
0
 
LVL 18

Expert Comment

by:BigSchmuh
Comment Utility
Most international development team use some Repository tool ( like "SubVersioN" ) that allows :
-any "client" to replicate locally all (or a branch) of it (using CheckIn/CheckOut methods to avoid data write conflicts)
-versioning all files in the Repository
-a global Repository backup

You may although use a replication system like DFS to synchronize a directory in 2(+) locations but it requires sharing some Domain info

If your build does change all files at every new build, both Win2008 R2 DFS and SVN should allow for a differential/incremental evaluation...that is to be tested anyway
0
 
LVL 18

Expert Comment

by:BigSchmuh
Comment Utility
SVN (SubVersioN)
   http://en.wikipedia.org/wiki/Apache_Subversion
On Windows, I have been using http://www.visualsvn.com/server (and its VisualSVN client) with success
0
Save on storage to protect fatherhood memories

You're the dad who has everything. This Father's Day, make sure your family memories are protected. My Passport Ultra has automatic backup and password protection to keep your cherished photos and videos safe. With up to 3TB, you have plenty of room to hold the adventures ahead.

 

Author Comment

by:greg_roberts
Comment Utility
Andy
Thanks. i looked at rsync but could not run. No csyxxx.dll (some library dll to convert between windows to linux)

BigSchmuh
We use TFS 2080 for our source control and subversion is more of a source control tool than a replication tool.

Normal "sync" tools result in the 4G build needing 2G of b/w to copy. I am hoping that a block level trial will reduce this dramatically. Was trialing R1Soft but can't get the s/w to work. This has block level copy feature.
0
 
LVL 18

Assisted Solution

by:BigSchmuh
BigSchmuh earned 125 total points
Comment Utility
My suggestion is to try using SVN (a source control tool) to replicate a directory locally:
-costless
-should allow for some incremental updates (Unsure with your binaries)
0
 

Author Closing Comment

by:greg_roberts
Comment Utility
Got rsync working using the url within the url for cwRsync but haven't got all the options right as i end up with the same sync transfer count as other tools (even using --inplace). Suspect issue with file times being used insead of CRC of file content.

As builds have complete binaries and files, exe, pdb, dlls , cab files etc i can't see how the SVN solution will help but thats for contrinuting.
0

Featured Post

Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

Join & Write a Comment

A bootable USB key can be very handy now-a-days. My favorite USB key consists of our Windows 7 image, network card drivers (to connect up to a Ghost server), the latest BIOS updates for all of our PCs and CopyWipe (to erase a retired PC) Creating…
Ever notice how you can't use a new drive in Windows without having Windows assigning a Disk Signature?  Ever have a signature collision problem (especially with Virtual Machines?)  This article is intended to help you understand what's going on and…
This video teaches viewers how to encrypt an external drive that requires a password to read and edit the drive. All tasks are done in Disk Utility. Plug in the external drive you wish to encrypt: Make sure all previous data on the drive has been …
This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now