Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

How can I get a formatted directory listing using awk?

Posted on 2003-11-26
12
Medium Priority
?
2,463 Views
Last Modified: 2013-12-26
I want to get a formatted directory listing that I can parse using other utilities such as perl, Excel, etc.  The unix "ls" command does not provide an easy way to accomplish this.  I am collecting the listings from servers running Linux, Solaris, HP-UX, or AIX.  All four OS's have some form of awk on it.

I would prefer the listing to be delimited and contain the full path names...  i.e.

permissions|owner|group|size|date|/full/path/to/file

example:
-rwxr-xr-x|root|system|125713|Sep 14 12:23|/usr/local/bin/sshd2
-rwxr-xr-x|root|system|125021|Sep 14 12:23|/usr/local/bin/scp2

Can this be done and how?  A working example would be great!  I could do this in perl, but I am not guaranteed that perl is available on all of the systems.  Also, if you can suggest another way of doing this for all 4 OS's if it is not possible with awk, that would be cool too!

Thanks!
0
Comment
Question by:ddurmon
  • 5
  • 3
  • 3
  • +1
12 Comments
 
LVL 9

Expert Comment

by:HamdyHassan
ID: 9825952
Idea,
you can use cut -c command to parse the output of ls -lt command
I think all these OS's has cut command, but the difference could be positions

Another idea, to use Perl

 
0
 
LVL 9

Expert Comment

by:HamdyHassan
ID: 9826077
Using Perl

parse_dir - parse directory listing

SUPPORTED PLATFORMS
Linux
Solaris
Windows


SYNOPSIS
 use File::Listing;
 for (parse_dir(`ls -l`)) {
     ($name, $type, $size, $mtime, $mode) = @$_;
     next if $type ne 'f'; # plain file
     #...
 }
 # directory listing can also be read from a file
 open(LISTING, "zcat ls-lR.gz|");
 $dir = parse_dir(\*LISTING, '+0000');


DESCRIPTION
The parse_dir() routine can be used to parse directory listings. Currently it only understand Unix 'ls -l' and 'ls -lR' format. It should eventually be able to most things you might get back from a ftp server file listing (LIST command), i.e. VMS listings, NT listings, DOS listings,...

The first parameter to parse_dir() is the directory listing to parse. It can be a scalar, a reference to an array of directory lines or a glob representing a filehandle to read the directory listing from.

The second parameter is the time zone to use when parsing time stamps in the listing. If this value is undefined, then the local time zone is assumed.

The third parameter is the type of listing to assume. The values will be strings like 'unix', 'vms', 'dos'. Currently only 'unix' is implemented and this is also the default value. Ideally, the listing type should be determined automatically.

The fourth parameter specifies how unparseable lines should be treated. Values can be 'ignore', 'warn' or a code reference. Warn means that the perl warn() function will be called. If a code reference is passed, then this routine will be called and the return value from it will be incorporated in the listing. The default is 'ignore'.

Only the first parameter is mandatory.

The return value from parse_dir() is a list of directory entries. In a scalar context the return value is a reference to the list. The directory entries are represented by an array consisting of [ $filename, $filetype, $filesize, $filetime, $filemode ]. The $filetype value is one of the letters 'f', 'd', 'l' or '?'. The $filetime value is the seconds since Jan 1, 1970. The $filemode is a bitmask like the mode returned by stat().



0
 
LVL 9

Expert Comment

by:HamdyHassan
ID: 9826247


Example
The following script will list "filename" , "|" , "file size"

#!/export/vol/bin/perl
use File::Listing;
 for (parse_dir(`ls -l`)) {
     ($name, $type, $size, $mtime, $mode) = @$_;
     next if $type ne 'f'; # plain file
     print $name . "|" . $size .  "\n" ;
 }
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 

Author Comment

by:ddurmon
ID: 9826660
I would love to use perl, but perl is not installed on all of the systems (and I don't control the system so I can't install it).  Is there a way using built-in utilities like awk?
0
 

Author Comment

by:ddurmon
ID: 9826686
P.S.  The reason I don't control the systems is that I am an Internal Auditor (I used to be a System Admin, but I'm not any good with awk beyond the very basic stuff).  I need to collect the listings to support my audit work.
0
 
LVL 7

Expert Comment

by:glassd
ID: 9828010
To get the full path names of a single directory you would have to do something like this:

#!/bin/ksh

ThisDir=$(pwd)

ls -l | awk -v ThisDir=$ThisDir '{
printf("%s|%s|%s|%s|%s %s %s|%s|%s/%s\n",$1,$2,$3,$4,$5,$6,$7,$8,ThisDir,$9)
}'

There are neater ways, but see if this does what you want first.
If you need to do a recursive listing through sub directories then I think this could be done.

I would stick to ksh if you are not sure what shells are likely to be available.

0
 
LVL 7

Assisted Solution

by:glassd
glassd earned 1000 total points
ID: 9828178
A couple of corrections (your required format isn't quite the same as the ls -l output, certainly on Solaris.

Also some awk versions are more capable than others. For example in Solaris you would need to use nawk, and in Linux you may need gawk.

I don't know the uname replies for OS versions other than Solaris, so you may need to do some editting.

#!/bin/ksh

ThisDir=$(pwd)
OS=$(uname -s)

case $OS in
   SunOS)   AWK=nawk;;
   Linux)     AWK=gawk;;
   *)           AWK=awk;;
esac

ls -l | $AWK -v ThisDir=$ThisDir '{
   printf("%s|%s|%s|%s|%s %s %s|%s/%s\n",$1,$3,$4,$8,$5,$6,$7,ThisDir,$9)
}'

This would fail if the file names contained spaces. If that is a problem then there is always a solution.
0
 

Author Comment

by:ddurmon
ID: 9828372
Perfect!  ...  Now how do I get it to do the list recursively?  :-)  
I can do this:


#!/bin/ksh
IFS='
'
RootDir=$1
for dir in $(find $RootDir -type d -print); do
  cd "$dir"
  ThisDir=$(pwd)
  ls -la "$ThisDir" | awk -v ThisDir=$ThisDir '{
  printf("%s|%s|%s|%s|%s|%s %s %s|%s/%s\n",$1,$2,$3,$4,$5,$6,$7,$8,ThisDir,$9)
  }' | grep -v "total|"
done


and it works, but is there a easier/faster/cheaper/better way to do it?
0
 

Author Comment

by:ddurmon
ID: 9828386
I do realize that I am going to have to tweak it a little for each OS.  Linux did this:

-rw-------|1|ddurmon|sysadmin|116 Feb 21|2002|/home/ddurmon/.Xauthority

instead of this:

-rw-------|1|ddurmon|sysadmin|116|Feb 21 2002|/home/ddurmon/.Xauthority

So I had to tweak the printf line...  :-)

0
 
LVL 1

Accepted Solution

by:
mbekker earned 1000 total points
ID: 9830485
Hi ddurmon,

For AIX you could try the following, it's more compact and will possibly also work on other systems (except for the date-part, which is different as you found out):

RootDir=$1
find $RootDir -type f -ls 2>/dev/null | awk '{printf "%s|%s|%s|%s|%s|%s %s %s|%s\n",$3,$4,$5,$6,$7,$8,$9,$10,$11}'

If you want it recursively just put " | sort -t'|' +6 " behind it ;-)

Good luck!
0
 
LVL 7

Expert Comment

by:glassd
ID: 9830639
May be you need to include the whole awk statement inside the case construct, rather than just setting the name of the executable there.
0
 

Author Comment

by:ddurmon
ID: 9833822
Thank you glassd and mbekkler.  This is exactly what I was looking for!  :-)

Thanks,

ddurmon
0

Featured Post

Hire Technology Freelancers with Gigs

Work with freelancers specializing in everything from database administration to programming, who have proven themselves as experts in their field. Hire the best, collaborate easily, pay securely, and get projects done right.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Here is how to use MFC's automatic Radio Button handling in your dialog boxes and forms.  Beginner programmers usually start with a OnClick handler for each radio button and that's just not the right way to go.  MFC has a very cool system for handli…
If you use Adobe Reader X it is possible you can't open OLE PDF documents in the standard. The reason is the 'save box mode' in adobe reader X. Many people think the protected Mode of adobe reader x is only to stop the write access. But this fe…
This video will show you how to get GIT to work in Eclipse.   It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.
Screencast - Getting to Know the Pipeline

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question