Solved

Smart Sync or Binary Diff Tool

Posted on 2010-09-01
6
675 Views
Last Modified: 2012-05-10
We have the need to transfer copies of our builds to India for testing. These builds are about 4G in total but have huge amounts which do not change between files.
So the build contains a combination of many files and big CABs (or equivalents) . Some of the binaries might only change by a few bytes for the version stamping.

=> Is there a copy or sync tool out there which is really smart and can build remotely (either directly or by a copy and process) a copy of a file area based on a "seed" area. This seed area on the remote site would simply be a copy of the previous build ? It would need to recognise the common parts in the binary files.
Context : Windows Server 2003 / 2008

Is this essentially Microsofts DFS Replication ?

Now a normal sync tool would be file for the stand alone files that don;t change but hopeless for a very large binary which only has say 2% of the file changed. It would have no choice but to copy the large binary file.

Thanks
0
Comment
Question by:greg_roberts
  • 3
  • 2
6 Comments
 
LVL 3

Accepted Solution

by:
AndyEastham earned 125 total points
ID: 33596434
Hi,

Have you looked a rsync?
http://www.samba.org/rsync/

Or for windows:
http://www.aboutmyip.com/AboutMyXApp/DeltaCopy.jsp

Andy
0
 
LVL 18

Expert Comment

by:BigSchmuh
ID: 33596493
Most international development team use some Repository tool ( like "SubVersioN" ) that allows :
-any "client" to replicate locally all (or a branch) of it (using CheckIn/CheckOut methods to avoid data write conflicts)
-versioning all files in the Repository
-a global Repository backup

You may although use a replication system like DFS to synchronize a directory in 2(+) locations but it requires sharing some Domain info

If your build does change all files at every new build, both Win2008 R2 DFS and SVN should allow for a differential/incremental evaluation...that is to be tested anyway
0
 
LVL 18

Expert Comment

by:BigSchmuh
ID: 33596521
SVN (SubVersioN)
   http://en.wikipedia.org/wiki/Apache_Subversion
On Windows, I have been using http://www.visualsvn.com/server (and its VisualSVN client) with success
0
Enterprise Mobility and BYOD For Dummies

Like “For Dummies” books, you can read this in whatever order you choose and learn about mobility and BYOD; and how to put a competitive mobile infrastructure in place. Developed for SMBs and large enterprises alike, you will find helpful use cases, planning, and implementation.

 

Author Comment

by:greg_roberts
ID: 33601586
Andy
Thanks. i looked at rsync but could not run. No csyxxx.dll (some library dll to convert between windows to linux)

BigSchmuh
We use TFS 2080 for our source control and subversion is more of a source control tool than a replication tool.

Normal "sync" tools result in the 4G build needing 2G of b/w to copy. I am hoping that a block level trial will reduce this dramatically. Was trialing R1Soft but can't get the s/w to work. This has block level copy feature.
0
 
LVL 18

Assisted Solution

by:BigSchmuh
BigSchmuh earned 125 total points
ID: 33602245
My suggestion is to try using SVN (a source control tool) to replicate a directory locally:
-costless
-should allow for some incremental updates (Unsure with your binaries)
0
 

Author Closing Comment

by:greg_roberts
ID: 33602265
Got rsync working using the url within the url for cwRsync but haven't got all the options right as i end up with the same sync transfer count as other tools (even using --inplace). Suspect issue with file times being used insead of CRC of file content.

As builds have complete binaries and files, exe, pdb, dlls , cab files etc i can't see how the SVN solution will help but thats for contrinuting.
0

Featured Post

Use Case: Protecting a Hybrid Cloud Infrastructure

Microsoft Azure is rapidly becoming the norm in dynamic IT environments. This document describes the challenges that organizations face when protecting data in a hybrid cloud IT environment and presents a use case to demonstrate how Acronis Backup protects all data.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

We all have limited time to study long and complicated information about RAID theories, but you may be interested as to how RAID 5 works. We made it simple for you by providing the shortest and easiest explanation ever.   First we need to remind …
This article aims to explain the working of CircularLogArchiver. This tool was designed to solve the buildup of log file in cases where systems do not support circular logging or where circular logging is not enabled
This video teaches viewers how to encrypt an external drive that requires a password to read and edit the drive. All tasks are done in Disk Utility. Plug in the external drive you wish to encrypt: Make sure all previous data on the drive has been …
This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question