?
Solved

perl copy the files by avoiding duplicates

Posted on 2014-10-06
4
Medium Priority
?
277 Views
Last Modified: 2014-10-07
Hi,

I am writing the below script to copy the files from one location to another location, but how can avoid copying duplicates.
I know what copy in perl does - if it see any duplicate files it will replace it, but I don't think this is a good idea becoz I generally have atleast 1000 files in a source folder and each day I will get some 20 files max...so when i run my perl program it just needs to copy those 20 files instead of all 1000 .... copying all 1000 files again is a waste of time and resource of server.

so how can i check before hand whether the file already exists in the destination or not.
Also what is the best solution in terms of memory and speed replacing duplicates files or checking for duplicates files.
from what i know checking for duplicates is effective way because of file size that it is going to replace.




my perl script:
my script:
#!/usr/bin/perl

use strict;
use warnings;

my $source = "C:\\test";
my $destination = "C:\\test1";

opendir(DIR,"$source") or die "Cannot open $source\n";
my @files = readdir(DIR);
closedir(DIR);

foreach my $file (@files) {
  next if ($file !~ /\.txt$/i);
  system("copy \"$source\\$file\" \"$destination\"");
 
}
0
Comment
Question by:shragi
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
4 Comments
 
LVL 26

Assisted Solution

by:wilcoxon
wilcoxon earned 1400 total points
ID: 40364673
in your foreach loop, add:
next if (-f "$destination/$file");

Open in new window

0
 
LVL 48

Expert Comment

by:Tintin
ID: 40364890
Why not just use rsync?
0
 

Author Comment

by:shragi
ID: 40364898
What is rsync ?
0
 
LVL 48

Accepted Solution

by:
Tintin earned 600 total points
ID: 40365096
rsync is a fantastic tool for copying files (locally or remotely) where it will only copy the changes.

It is very fast and efficient.

See http://rsync.samba.org for details.

For WIndows, there are various ports (including cygwin) available.
0

Featured Post

What is a Denial of Service (DoS)?

A DoS is a malicious attempt to prevent the normal operation of a computer system. You may frequently see the terms 'DDoS' (Distributed Denial of Service) and 'DoS' used interchangeably, but there are some subtle differences.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article is meant to give a basic understanding of how to use R Sweave as a way to merge LaTeX and R code seamlessly into one presentable document.
In threads here at EE, each comment has a unique Identifier (ID). It is easy to get the full path for an ID via the right-click context menu. However, we often want to post a short link within a thread rather than the full link. This article shows a…
Video by: Mark
This lesson goes over how to construct ordered and unordered lists and how to create hyperlinks.
In a recent question (https://www.experts-exchange.com/questions/29004105/Run-AutoHotkey-script-directly-from-Notepad.html) here at Experts Exchange, a member asked how to run an AutoHotkey script (.AHK) directly from Notepad++ (aka NPP). This video…
Suggested Courses

800 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question