Solved

perl script to get file names and counts of sub directories

Posted on 2010-08-30
23
438 Views
Last Modified: 2012-05-10
Hi,
 I am beginner in perl. My goal is to get the file names and counts og sub directories from a directory.
For example my directory structure looks like

 New folder/ Sub folder 1/ sub folder 2/sub folder 3...
 sub folder 1/ file 1/file2/file3..
sub folder 2 / file 1/ file 2/file 3..


 I am copying the updated script below. when I am trying to run the script it is not giving the files count in sub folders but it is giving the counts of sub folders.

#!/usr/local/bin/perl

use strict;
use warnings;

my $dir= 'c:\Documents and Settings\user\Desktop\new folder';

my $directory_count=0;
my $file_count=0;
my $file =0;



my $outfile = 'log.txt';

open my $OUTF, '>', $outfile or die "can't create logfile;$!";

opendir (DIR, $dir);

my @files = readdir(DIR);


   closedir( DIR ) ;

 
   foreach my $files(@files)
   {
   
       
   
    if (-d "$dir/$file")
     
     {
         $directory_count++;
   }
   
     else {
     
    $file_count++;
   
    print {$OUTF} "$files|";
   
    print { $OUTF} "$directory_count |";
   print  { $OUTF}   "$file_count \n";
     
    }
   
   }
   
 Required Output;

  sub folder 1  |  no. of files in sub folder 1
 sub folder 2   | no.of files in sub folder 2...


Thanks,
 




0
Comment
Question by:new_perl_user
  • 11
  • 8
  • 2
  • +1
23 Comments
 
LVL 84

Expert Comment

by:ozo
Comment Utility

    if (-d "$dir/$file")
     
     {
         $directory_count++;
         print  { $OUTF}   "$file | " .. (() = <$dir/$file/*>) . "\n" ;
   }
0
 

Author Comment

by:new_perl_user
Comment Utility
Hi,
  Thanks for the reply but It did not make any change when I tried the above suggestion.

 The output is in this way:

 Sub Folder 1  |  1
Sub Folder  2  |   2..

 I mean it is listing out the sub folder names but instead of counting the files in sub folders. it is still taking the count of sub folders.
0
 
LVL 84

Expert Comment

by:ozo
Comment Utility
#sorry, that should have been . not ..
 print  { $OUTF}   "$file | " . (() = <$dir/$file/*>) . "\n" ;
0
 

Author Comment

by:new_perl_user
Comment Utility

Hi,
 I am sorry to say that but still it is doesn't make any difference in my output .
0
 
LVL 84

Expert Comment

by:ozo
Comment Utility
#it looks like $file and $files were meant to be the same
#either change

  foreach my $file(@files)
 
#or

  if (-d "$dir/$files")
     
     {
         $directory_count++;
         print  { $OUTF}   "$files | " . (() = <$dir/$files/*>) . "\n" ;
   }
0
 

Author Comment

by:new_perl_user
Comment Utility
Hi,
 Made some improvement but it is showing the wrong counts. I mean some sub folders does not have any files in them so it should be zero, but it is showing all the counts as 5 for every sub folder. I am pasting the new code after making changes according to the suggestions.

#!/usr/local/bin/perl

use strict;
use warnings;

my $dir= 'c:\Documents and Settings\user\Desktop\new folder';

my $directory_count=0;
my $file_count=0;
my $file =0;



my $outfile = 'log.txt';

open my $OUTF, '>', $outfile or die "can't create logfile;$!";

opendir (DIR, $dir);

my @files = readdir(DIR);


   closedir( DIR ) ;

 
   foreach my $file(@files)
   {
   
       
   
    if (-d "$dir/$file")
     
     {
         $directory_count++;
        print  { $OUTF}   "$file | " . (() = <$dir/$file/*>) . "\n" ;
   }
   
     else {
     
    $file_count++;
   
    print {$OUTF} "$file|";
   
    print { $OUTF} "$directory_count |";
   print  { $OUTF}   "$file_count \n";
     
    }
   
   }

Output:
  .  | 3
 .. | 28
Sub Folder 1  | 5
Sub Folder 2 | 5
sub folder 3  | 5...

Thanks,

 
0
 
LVL 84

Expert Comment

by:ozo
Comment Utility
The spaces in $dir and $file may be confusing the glob
Try <\Q$dir/$file\E/*>
I'm not sure if that works in mswindows

print  { $OUTF}   "$file | " , <\Q$dir/$file\E/*> , "\n" ;
#should list the files in the folder instead of counting them so you can see what it is getting
#note , instead of .
0
 

Author Comment

by:new_perl_user
Comment Utility
Hi,
 It is not working. I mean it is listing all the files  in that location regardless of my declared directory. Is there any other approach in perl  I could do this regardless of the way I was trying.

Thanks,
0
 
LVL 84

Expert Comment

by:ozo
Comment Utility
What files is it listing?
0
 

Author Comment

by:new_perl_user
Comment Utility

Hi,
  For example the directory I declared is at the desktop location. So it is listing all the files that are saved on my desktop and the directory I declared in my script . It is also not giving the counts.

Thanks,
0
 
LVL 84

Expert Comment

by:ozo
Comment Utility
Is that when $file is . or ..
0
Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

 

Author Comment

by:new_perl_user
Comment Utility
Hi,
 When I tried the above solution  print  { $OUTF}   "$file | " , <\Q$dir/$file\E/*> , "\n"  it printed all the filenames regardless of the directory declared.

When  I modified back to  print {$OUTF} "$file|".(()= <$dir/$file/*>). "\n";  it  is printing the filenames correctly but the counts are wrong as I said earlier.  Please help if there is any other way.

Thanks,
0
 
LVL 26

Expert Comment

by:wilcoxon
Comment Utility
This should give you what you want (plus sub-dir count which I'm not sure you want but it's easy to change if you don't want it).

Let me know if this doesn't do what you want.
#!/usr/local/bin/perl
use strict;
use warnings;

my $dir= 'c:\Documents and Settings\user\Desktop\new folder';

my $directory_count=0;
my $file_count=0;
my $file =0;

my $outfile = 'log.txt';

open my $OUTF, '>', $outfile or die "can't create logfile;$!";
print {$OUTF} "dir | dir cnt | file cnt\n";

process($dir);

sub process {
    my ($dir) = @_;
    opendir (DIR, $dir);
    my @ents = readdir(DIR);
    closedir( DIR ) ;
    my @dirs = grep { -d "$dir/$_" and not /^\.+$/ } @ents;
    my $cnt = scalar grep { -f "$dir/$_" } @ents;
    print {$OUTF} "$dir | " . scalar(@dirs) . " | $cnt\n";
    foreach my $sub (@dirs) {
        process("$dir/$sub");
    }
}

Open in new window

0
 

Author Comment

by:new_perl_user
Comment Utility
Hi,
 When I am trying to execute the above code it is showing up the below errors.  

readdir() attempted on invalid dirhandle DIR at count.pl line 21.
closedir() attempted on invalid dirhandle DIR at  count.pl line 22

Thanks,
0
 
LVL 84

Expert Comment

by:ozo
Comment Utility
opendir (DIR, $dir) or warn "opendir($dir) failed because $!";
0
 

Author Comment

by:new_perl_user
Comment Utility
Hi,
 Still the same problem after making changes according to the above suggestion.
0
 
LVL 84

Expert Comment

by:ozo
Comment Utility
It reports no warnings?

0
 

Author Comment

by:new_perl_user
Comment Utility
Hi,
 I am sorry. It is working now a small code error i  did, @specifying the directory location. It is almost  close to the output I want  as shown below. This is the output I am getting when I am executing the above script. But  I need a small change  it should not traverse into files , I mean into file2345, file 567.... please.

Current:
dir | dir cnt | file cnt
c:\Documents and Settings\user\Desktop\new folder | 4 | 0
c:\Documents and Settings\user\Desktop\new folder/sub folder 1 | 2 | 0
c:\Documents and Settings\user\Desktop\new folder/sub folder 1/file2345 | 0 | 0
c:\Documents and Settings\user\Desktop\new folder/sub folder 1/file567 | 0 | 0
c:\Documents and Settings\user\Desktop\new folder/sub folder 2 | 0 | 0
c:\Documents and Settings\user\Desktop\new folder/sub folder 3 | 0 | 0
c:\Documents and Settings\user\Desktop\new folder/sub folder 4 | 0 | 0

Required:
 dir cnt  | file cnt
sub folder 1 | 2
sub folder 2 | 0..

Thanks,
 
0
 
LVL 26

Expert Comment

by:wilcoxon
Comment Utility
The code I posted does not traverse into files (only dirs).  Could you please post your current script so that I can try to spot the problem?

I'm unclear on your required output.  The labels say "dir cnt" and "file cnt" but the data lines look like they are "dir name" and "file cnt".  Do you want "dir cnt" in the output or is the label wrong (and it should be "dir name").
0
 

Author Comment

by:new_perl_user
Comment Utility
Hi,
 I am so sorry it was the label mistake. I want the sub- directory name and count of sub directories in the sub directory. For example there is a Main directory and in that there are several sub directories and each sub directory contains so many sub directories again.

 Main directory/ sub directory 1/ directory1, directory 2, directory 3...
                           subdirectory 2 / directory 1, directory 2, directory 3...
 
  So the requirement is  the script should traverse into this directory get the sub directory name( sub directoryr 1) and count  of directories in that sub directory( directory1, directory 2, directory 3) . The script which was given by you in the previous reply is the one I am using now. It is doing almost what I want but

1) it is also traversing into the directories and getting their dir names(sub folder 1/file2345)  which is not required
2) In the output it is always giving out the  Main directory path(c:\Documents and Settings\user\Desktop\new folder) which is not required.  

  I  tried to explain it clearly but in case if it is still confusing please let me know. I am attaching an image so that you can better understand  the directory structure although I am not good at drawing.
This is the script I am currently using.

#!/usr/local/bin/perl
use strict;
use warnings;

my $dir= 'c:\Documents and Settings\user\Desktop\new folder';

my $directory_count=0;
my $file_count=0;
my $file =0;

my $outfile = 'log.txt';

open my $OUTF, '>', $outfile or die "can't create logfile;$!";
print {$OUTF} "dir | dir cnt | file cnt\n";

process($dir);

sub process {
    my ($dir) = @_;
    opendir (DIR, $dir);
    my @ents = readdir(DIR);
    closedir( DIR ) ;
    my @dirs = grep { -d "$dir/$_" and not /^\.+$/ } @ents;
    my $cnt = scalar grep { -f "$dir/$_" } @ents;
    print {$OUTF} "$dir | " . scalar(@dirs) . " | $cnt\n";
    foreach my $sub (@dirs) {
        process("$dir/$sub");
    }
}


Dircetory-Structure.JPG
0
 
LVL 13

Expert Comment

by:Carl Bohman
Comment Utility
With minor modifications to your script, I was able to come up with the following (attached).  I believe it does what you are looking for.  It allows you to control the search depth (with a $basedepth of zero being only the base directory, a $basedepth of one goes one level deep, etc.).
#!/usr/local/bin/perl

use strict;

use warnings;



my $basedir= 'c:\Documents and Settings\user\Desktop\new folder';

my $basedepth = 1;



my $directory_count=0;

my $file_count=0;

my $file =0;



my $outfile = 'outfile.txt';



open my $OUTF, '>', $outfile or die "can't create logfile;$!";

print {$OUTF} "dir | dir cnt | file cnt\n";



process($basedir, $basedepth);



sub process {

    my ($dir, $depth) = @_;

    return unless ($depth >= 0);

    opendir (DIR, $dir);

    my @ents = readdir(DIR);

    closedir( DIR ) ;

    my @dirs = grep { -d "$dir/$_" and not /^\.+$/ } @ents;

    my $cnt = scalar grep { -f "$dir/$_" } @ents;

    my $displaydir = ($dir eq $basedir ? $dir : substr($dir, length($basedir)));

    print {$OUTF} "$displaydir | " . scalar(@dirs) . " | $cnt\n";

    foreach my $sub (@dirs) {

        process("$dir/$sub", $depth - 1);

    }

}

Open in new window

0
 

Author Comment

by:new_perl_user
Comment Utility
Hi,
when I executed the above script the output was:

o/p:
 
dir | dir cnt | file cnt
 c:\documents and settings\user\new folder | 4 |0
/ sub directory1  | 2 |  0
/sub directory2   | 4 |  0
/ sub directory 3  | 0 |  0
/ sub directory 4  | 2 |  0

  But if possible can I get something like  the one shown below. c:\documents and settings\user\new folder | 4 |0  is the main directory which is not required  Because this output file should be loaded into database so i am trying to get only the required info.

 sub directory1  |  2
sub directory2   |  4
sub directory 3  | 0
sub directory 4  |  2


Thanks,
0
 
LVL 13

Accepted Solution

by:
Carl Bohman earned 500 total points
Comment Utility
This one allows for setting a minimum and maximum depth.  In all honesty, making it this configurable is likely unnecessary, but maybe it will help you learn a bit more about how Perl works to do it this way.  Also note that the way the subdirectory is displayed is configurable by editing the like that sets $displaydir.
#!/usr/local/bin/perl

use strict;

use warnings;



use File::Basename;



my $basedir= 'c:\Documents and Settings\user\Desktop\new folder';

my $basemindepth = 1;

my $basemaxdepth = 1;



my $directory_count=0;

my $file_count=0;

my $file =0;



my $outfile = 'outfile.txt';



open my $OUTF, '>', $outfile or die "can't create logfile;$!";

print {$OUTF} "dir | dir cnt | file cnt\n";



process($basedir, $basemindepth, $basemaxdepth);



sub process {

    my ($dir, $mindepth, $maxdepth) = @_;

    return unless ($maxdepth >= 0);

    opendir (DIR, $dir);

    my @ents = readdir(DIR);

    closedir( DIR ) ;

    my @dirs = grep { -d "$dir/$_" and not /^\.+$/ } @ents;

    my $cnt = scalar grep { -f "$dir/$_" } @ents;

    my $displaydir = basename($dir);

    print {$OUTF} "$displaydir | " . scalar(@dirs) . " | $cnt\n" if ($mindepth <= 0);

    foreach my $sub (@dirs) {

        process("$dir/$sub", $mindepth - 1, $maxdepth - 1);

    }

}

Open in new window

0

Featured Post

Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

Join & Write a Comment

I have been pestered over the years to produce and distribute regular data extracts, and often the request have explicitly requested the data be emailed as an Excel attachement; specifically Excel, as it appears: CSV files confuse (no Red or Green h…
In the distant past (last year) I hacked together a little toy that would allow a couple of Manager types to query, preview, and extract data from a number of MongoDB instances, to their tool of choice: Excel (http://dilbert.com/strips/comic/2007-08…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…

728 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now