How can I get a formatted directory listing using awk?

I want to get a formatted directory listing that I can parse using other utilities such as perl, Excel, etc.  The unix "ls" command does not provide an easy way to accomplish this.  I am collecting the listings from servers running Linux, Solaris, HP-UX, or AIX.  All four OS's have some form of awk on it.

I would prefer the listing to be delimited and contain the full path names...  i.e.

permissions|owner|group|size|date|/full/path/to/file

example:
-rwxr-xr-x|root|system|125713|Sep 14 12:23|/usr/local/bin/sshd2
-rwxr-xr-x|root|system|125021|Sep 14 12:23|/usr/local/bin/scp2

Can this be done and how?  A working example would be great!  I could do this in perl, but I am not guaranteed that perl is available on all of the systems.  Also, if you can suggest another way of doing this for all 4 OS's if it is not possible with awk, that would be cool too!

Thanks!
ddurmonAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

HamdyHassanCommented:
Idea,
you can use cut -c command to parse the output of ls -lt command
I think all these OS's has cut command, but the difference could be positions

Another idea, to use Perl

 
HamdyHassanCommented:
Using Perl

parse_dir - parse directory listing

SUPPORTED PLATFORMS
Linux
Solaris
Windows


SYNOPSIS
 use File::Listing;
 for (parse_dir(`ls -l`)) {
     ($name, $type, $size, $mtime, $mode) = @$_;
     next if $type ne 'f'; # plain file
     #...
 }
 # directory listing can also be read from a file
 open(LISTING, "zcat ls-lR.gz|");
 $dir = parse_dir(\*LISTING, '+0000');


DESCRIPTION
The parse_dir() routine can be used to parse directory listings. Currently it only understand Unix 'ls -l' and 'ls -lR' format. It should eventually be able to most things you might get back from a ftp server file listing (LIST command), i.e. VMS listings, NT listings, DOS listings,...

The first parameter to parse_dir() is the directory listing to parse. It can be a scalar, a reference to an array of directory lines or a glob representing a filehandle to read the directory listing from.

The second parameter is the time zone to use when parsing time stamps in the listing. If this value is undefined, then the local time zone is assumed.

The third parameter is the type of listing to assume. The values will be strings like 'unix', 'vms', 'dos'. Currently only 'unix' is implemented and this is also the default value. Ideally, the listing type should be determined automatically.

The fourth parameter specifies how unparseable lines should be treated. Values can be 'ignore', 'warn' or a code reference. Warn means that the perl warn() function will be called. If a code reference is passed, then this routine will be called and the return value from it will be incorporated in the listing. The default is 'ignore'.

Only the first parameter is mandatory.

The return value from parse_dir() is a list of directory entries. In a scalar context the return value is a reference to the list. The directory entries are represented by an array consisting of [ $filename, $filetype, $filesize, $filetime, $filemode ]. The $filetype value is one of the letters 'f', 'd', 'l' or '?'. The $filetime value is the seconds since Jan 1, 1970. The $filemode is a bitmask like the mode returned by stat().



HamdyHassanCommented:


Example
The following script will list "filename" , "|" , "file size"

#!/export/vol/bin/perl
use File::Listing;
 for (parse_dir(`ls -l`)) {
     ($name, $type, $size, $mtime, $mode) = @$_;
     next if $type ne 'f'; # plain file
     print $name . "|" . $size .  "\n" ;
 }
Learn SQL Server Core 2016

This course will introduce you to SQL Server Core 2016, as well as teach you about SSMS, data tools, installation, server configuration, using Management Studio, and writing and executing queries.

ddurmonAuthor Commented:
I would love to use perl, but perl is not installed on all of the systems (and I don't control the system so I can't install it).  Is there a way using built-in utilities like awk?
ddurmonAuthor Commented:
P.S.  The reason I don't control the systems is that I am an Internal Auditor (I used to be a System Admin, but I'm not any good with awk beyond the very basic stuff).  I need to collect the listings to support my audit work.
glassdCommented:
To get the full path names of a single directory you would have to do something like this:

#!/bin/ksh

ThisDir=$(pwd)

ls -l | awk -v ThisDir=$ThisDir '{
printf("%s|%s|%s|%s|%s %s %s|%s|%s/%s\n",$1,$2,$3,$4,$5,$6,$7,$8,ThisDir,$9)
}'

There are neater ways, but see if this does what you want first.
If you need to do a recursive listing through sub directories then I think this could be done.

I would stick to ksh if you are not sure what shells are likely to be available.

glassdCommented:
A couple of corrections (your required format isn't quite the same as the ls -l output, certainly on Solaris.

Also some awk versions are more capable than others. For example in Solaris you would need to use nawk, and in Linux you may need gawk.

I don't know the uname replies for OS versions other than Solaris, so you may need to do some editting.

#!/bin/ksh

ThisDir=$(pwd)
OS=$(uname -s)

case $OS in
   SunOS)   AWK=nawk;;
   Linux)     AWK=gawk;;
   *)           AWK=awk;;
esac

ls -l | $AWK -v ThisDir=$ThisDir '{
   printf("%s|%s|%s|%s|%s %s %s|%s/%s\n",$1,$3,$4,$8,$5,$6,$7,ThisDir,$9)
}'

This would fail if the file names contained spaces. If that is a problem then there is always a solution.
ddurmonAuthor Commented:
Perfect!  ...  Now how do I get it to do the list recursively?  :-)  
I can do this:


#!/bin/ksh
IFS='
'
RootDir=$1
for dir in $(find $RootDir -type d -print); do
  cd "$dir"
  ThisDir=$(pwd)
  ls -la "$ThisDir" | awk -v ThisDir=$ThisDir '{
  printf("%s|%s|%s|%s|%s|%s %s %s|%s/%s\n",$1,$2,$3,$4,$5,$6,$7,$8,ThisDir,$9)
  }' | grep -v "total|"
done


and it works, but is there a easier/faster/cheaper/better way to do it?
ddurmonAuthor Commented:
I do realize that I am going to have to tweak it a little for each OS.  Linux did this:

-rw-------|1|ddurmon|sysadmin|116 Feb 21|2002|/home/ddurmon/.Xauthority

instead of this:

-rw-------|1|ddurmon|sysadmin|116|Feb 21 2002|/home/ddurmon/.Xauthority

So I had to tweak the printf line...  :-)

mbekkerCommented:
Hi ddurmon,

For AIX you could try the following, it's more compact and will possibly also work on other systems (except for the date-part, which is different as you found out):

RootDir=$1
find $RootDir -type f -ls 2>/dev/null | awk '{printf "%s|%s|%s|%s|%s|%s %s %s|%s\n",$3,$4,$5,$6,$7,$8,$9,$10,$11}'

If you want it recursively just put " | sort -t'|' +6 " behind it ;-)

Good luck!

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
glassdCommented:
May be you need to include the whole awk statement inside the case construct, rather than just setting the name of the executable there.
ddurmonAuthor Commented:
Thank you glassd and mbekkler.  This is exactly what I was looking for!  :-)

Thanks,

ddurmon
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
System Programming

From novice to tech pro — start learning today.