Solved

perl copy the files by avoiding duplicates

Posted on 2014-10-06
4
267 Views
Last Modified: 2014-10-07
Hi,

I am writing the below script to copy the files from one location to another location, but how can avoid copying duplicates.
I know what copy in perl does - if it see any duplicate files it will replace it, but I don't think this is a good idea becoz I generally have atleast 1000 files in a source folder and each day I will get some 20 files max...so when i run my perl program it just needs to copy those 20 files instead of all 1000 .... copying all 1000 files again is a waste of time and resource of server.

so how can i check before hand whether the file already exists in the destination or not.
Also what is the best solution in terms of memory and speed replacing duplicates files or checking for duplicates files.
from what i know checking for duplicates is effective way because of file size that it is going to replace.




my perl script:
my script:
#!/usr/bin/perl

use strict;
use warnings;

my $source = "C:\\test";
my $destination = "C:\\test1";

opendir(DIR,"$source") or die "Cannot open $source\n";
my @files = readdir(DIR);
closedir(DIR);

foreach my $file (@files) {
  next if ($file !~ /\.txt$/i);
  system("copy \"$source\\$file\" \"$destination\"");
 
}
0
Comment
Question by:shragi
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
4 Comments
 
LVL 26

Assisted Solution

by:wilcoxon
wilcoxon earned 350 total points
ID: 40364673
in your foreach loop, add:
next if (-f "$destination/$file");

Open in new window

0
 
LVL 48

Expert Comment

by:Tintin
ID: 40364890
Why not just use rsync?
0
 

Author Comment

by:shragi
ID: 40364898
What is rsync ?
0
 
LVL 48

Accepted Solution

by:
Tintin earned 150 total points
ID: 40365096
rsync is a fantastic tool for copying files (locally or remotely) where it will only copy the changes.

It is very fast and efficient.

See http://rsync.samba.org for details.

For WIndows, there are various ports (including cygwin) available.
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Background Still having to process all these year-end "csv" files received from all these sources (including Government entities), sometimes we have the need to examine the contents due to data error, etc... As a "Unix" shop, our only readily …
How to remove superseded packages in windows w60 or w61 installation media (.wim) or online system to prevent unnecessary space. w60 means Windows Vista or Windows Server 2008. w61 means Windows 7 or Windows Server 2008 R2. There are various …
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.
Video by: Mark
This lesson goes over how to construct ordered and unordered lists and how to create hyperlinks.

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question