Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win


copying files using perl/unix

Posted on 2011-03-07
Medium Priority
Last Modified: 2012-05-11
 I want to copy a file from source to destination by matching the directory names.

Example:   Batch0001  is the higher level directory and 4567890R is the sub directory which consists of several  log files.
Batch0001 -> 4567890R
                 ->  1237896R
                 ->   4561089R
Batch0002  ->  ....

Batch0001 -> 4567890R
                 ->  1237896R
                 ->   4561089R
Batch0002  ->  ....

My source and destination will have the same directory structure and filenames. So it has to compare them and when it encounters same  directory and sub directory it has to copy from  source to des a file by name :  4567890R(subdirectory name)_nornalize.log
Question by:new_perl_user
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
  • 2
LVL 10

Expert Comment

ID: 35057319
If you just want to copy all of the *_normalize.log files across from /source to /dest on Linux and the directory structures already match between /source and /dest then all you should need to do is:

cd /source
find  .  -name  '*_normalize.log'  -exec  cp  {}  /dest/{}  ';'

and if you want it to list the files as it is copying use:

cd /source
find  .  -name  '*_normalize.log'  -exec  cp  {}  /dest/{}  ';'  -print
LVL 27

Accepted Solution

wilcoxon earned 1000 total points
ID: 35058731
What if the destination directory doesn't exist?  Should it be created or should those files not be copied?

Change the variable values where indicated in the comments and this should do what you want.  If it doesn't work, let me know what is wrong and I'll fix it.

use strict;
use warnings;
use File::Copy qw(cp);
use File::Find;
use File::Path;

use vars qw/*name *dir *prune/;
*name   = *File::Find::name;
*dir    = *File::Find::dir;
*prune  = *File::Find::prune;

# change these to the real source and destination root folders
my $src_dir = '/src';
my $dest_dir = '/dest';

# change this to 1 if you want to make the dest dir if it does not exist
# 0 will skip the file if the dest dir does not exist
my $mk_dest = 0;

use Cwd ();
my $cwd = Cwd::cwd();

# Traverse desired filesystems
File::Find::find({wanted => \&wanted}, $src);

sub wanted {
    return unless /^_normalize.log$/;
    my $file = $_;
    my $dpath = $dir;
    # remove $src_dir from the dir
    $dpath =~ s{^$src_dir}{$dest_dir};
    if ($mk_dest) {
        make_path($dpath) unless (-d $dpath);
    } else {
        return unless (-d $dpath);
    cp $name, "$dpath/$file" or die "could not copy $name: $!";

Open in new window

LVL 10

Assisted Solution

Martin_J_Parker earned 1000 total points
ID: 35058936
If the destination directory does not exist you can still do something similar with a simple bash script
- just change /source and /dest to the source and destination directory names:

#Start in the source directory
cd /source
#Find all the normalize.log files:
for i in `find . -name *_normalize.log`; do
# For each file find what it's directory name is relative to /source
   j=`dirname $i`
# If the directory does not exist in /dest then create it
   if [ ! -d /dest/$j ]
      mkdir /dest/$j
#Copy the file over to /dest
   cp $i /dest/$i    
#End of bash script

You can make the script executable with the command
chmod 755 scriptname
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.


Expert Comment

ID: 35083808
Very simple one liner that will create the directories as needed.  If you need the original dates of the files use the -m option with cpio

find /source -name *_normalize.log -depth | cpio -pmdv /dest
LVL 27

Expert Comment

ID: 35084183
If his system has cpio (most but not all do).

Also, won't that command copy the files to /dest/src/path/*_normalize.log (rather than /dest/path/*_normalize.log)?
LVL 27

Expert Comment

ID: 35084215
One minor tweak to my code - line 30 should be:

return unless /_normalize.log$/;

Otherwise, it will only look for files named (exactly) _normalize.log.

Expert Comment

ID: 35084394
possibly depends on the version, but if it does simply cd into source directory first then execute.

find . -name *_normalize.log -depth | cpio -pmdv /dest

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

On Microsoft Windows, if  when you click or type the name of a .pl file, you get an error "is not recognized as an internal or external command, operable program or batch file", then this means you do not have the .pl file extension associated with …
There are many situations when we need to display the data in sorted order. For example: Student details by name or by rank or by total marks etc. If you are working on data driven based projects then you will use sorting techniques very frequently.…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
Six Sigma Control Plans

604 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question