Link to home
Start Free TrialLog in
Avatar of cbeverly
cbeverly

asked on

total of all files

Hello,

We are running solaris 2.9 on a Unix machine and I wanted to know how to get a total number of files in each directory and subdirectory of our server.
For example, if I am in a directory and underneath it are the subdirectories
collections
images
products

I would like a command that will give me the name of each subdirectory (and including their subdirectories) with a total number of files in each so that it would read
collections 40 (number of files)
  subdirectory X 30
  subdirectory Y 50
images 50
  subdirectory A 33
  subdirectory B 22
products 30

Is there a command to do this or anything close?
Also is there a way to separate out types of files such as .html files from .jpg for example and subtotal each of those as well?

Thank you!
Avatar of sunnycoder
sunnycoder
Flag of India image

Hi cbeverly,

find /top/level/dir -type d | while read dirname
do
      echo -n "$dirname      "
      find $dirname -maxdepth 1 -type f | ls -l
done

>Also is there a way to separate out types of files such as .html files from .jpg for example and subtotal each of those as well?
Just modify the find command inside the loop with -name option e.g. to find total number of .jpg files in every directory

find /top/level/dir -type d | while read dirname
do
      echo -n "$dirname      "
      find $dirname -maxdepth 1 -type f -name "*.jpg" | ls -l
done

Cheers!
Sunnycoder
Avatar of bira
bira

ls -lR|grep -E "total|:"
sunnycoder. Solaris find doesn't have -maxdepth flag

bira, Solaris grep doesn't have -E flag.

The following script will work on Solaris:

for dir in `find . -type d`
do
  total=`ls -Fl $dir|egrep -vc '/|total'`
  echo "$dir $total"
done

bira, Titin, there is one big mistake in your suggestion:
total: is NOT a number of subdirectories or file... but a number of blocks used by that directory.

cbeverly, do you need only two-level solution or recursive to infinite number of sublevels?
for infinit number of sublevels this script works as you asked:

#!/bin/sh
# Get number of files and directories in current dir and all subdirs

tab=0
dir=`pwd`
total_files () {
  # list non-directories
  count=`ls -aF | sed '1,2d;/[^\/]$/p;d' | wc -l`
  printf "%${1}s%s [%d]\n" " " $2 $count
  # if we have subdirs - process them with the same way
  if [ `ls -aF | sed '1,2d;/\/$/p;d' | wc -l` -eq 0 ]
  then
    return 0
  fi
  for sub in `ls -aF | sed '1,2d;/\/$/s/.$//p;d'`
  do
    cd $sub
    total_files `expr $1 + 2` $sub
    cd ..
  done
  return 0
}
total_files $tab $dir
Avatar of cbeverly

ASKER

Nopius,

First of all thank you so much for this great program. It is doing exactly what I need. The only problem is that when it gets to a particular directory we have it loops over in that one directory:
 ./filenum.sh
 /teamsite/www/collections/exhibits [2]
  abstraction [90]
  allusions [88]
  amerphotos [1]
  catlin [86]
    images [6]
  catlinclassroom [70]
    alb [34]
    alma [42]
    alpop [30]
      .. [70]
        alb [34]
        alma [42]
        alpop [30]
          .. [70]
            alb [34]
            alma [42]
            alpop [30]
              .. [70]
                alb [34]
                alma [42]
                alpop [30]
                  .. [70]
                    alb [34]
                    alma [42]
                    alpop [30]
                      .. [70]


Here is a list of our main directory:
-rwxr-xr--   1 root     other       8018 Apr 14 09:13 404-header.gif
drwxr-xr-x   2 root     other       2560 Apr 14 09:08 abstraction
drwxr-xr-x   2 root     other       2560 Apr 14 09:08 allusions
drwxr-xr-x   2 root     other        512 Apr 14 09:08 amerphotos
drwxr-xr-x   3 root     other       2560 Apr 14 09:09 catlin
drwxr-xr-x  35 root     other       2560 Apr 14 09:10 catlinclassroom
drwxr-xr-x   2 root     other        512 Apr 14 09:10 cng
drwxr-xr-x   4 root     other       6144 Apr 14 09:10 cottingham
-rwxr-xr-x   1 cbeverly admins       522 May 12 09:20 filenum.sh
drwxr-xr-x   4 root     other       4096 Apr 14 09:10 gross
drwxr-xr-x   9 root     other       1536 Apr 14 09:12 helios

and here is the directory that it is looping in:

cd catlinclassroom
ls
-rwxr-xr--   1 root     other         43 Apr 14 09:09 1.gif
-rwxr-xr--   1 root     other      14874 Apr 14 09:09 1985.66.153,295_1c.jpg
-rwxr-xr--   1 root     other        336 Apr 14 09:10 Application.cfm
-rwxr-xr--   1 root     other     287175 Apr 14 09:10 SIL22-03-01 catlin painting.jpg
-rwxr-xr--   1 root     other       6282 Apr 14 09:09 al-theme.html
drwxr-xr-x   2 root     other       1024 Apr 14 09:09 alb
drwxr-xr-x   2 root     other       1536 Apr 14 09:09 alma
drwxr-xr-x   3 root     other       1024 Apr 14 09:09 alpop
drwxr-xr-x   2 root     other       1024 Apr 14 09:09 als
drwxr-xr-x   2 root     other       2048 Apr 14 09:09 altime
-rwxr-xr--   1 root     other      24981 Apr 14 09:10 artworks.html
drwxr-xr-x   2 root     other        512 Apr 14 09:09 backgrounds
-rwxr-xr--   1 root     other       2726 Apr 14 09:10 backnext.cfm

Do you have any ideas why it might be looping?
Thank you again!
find /top/level/dir -type d | while read dirname
do
      echo -n "$dirname      "
      ls -A1 $dirname | ls -l
done

In case ls on solaris does not support -1 option, use

find /top/level/dir -type d | while read dirname
do
      echo -n "$dirname      "
      count=`ls -Al $dirname | ls -l`
      count=`expr $count - 1`
      echo $count
done
oooops
> ls -A1 $dirname | ls -l
should have been
 ls -A1 $dirname | wc -l
it's looping on .. directory. Ok, let's modify my program:


#!/bin/sh
# Get number of files and directories in current dir and all subdirs

tab=0
dir=`pwd`
total_files () {
  # list non-directories
  count=`ls -aF | sed '/^[.][.]*\//d;/[^\/]$/p;d' | wc -l`
  printf "%${1}s%s [%d]\n" " " $2 $count
  # if we have subdirs - process them with the same way
  if [ `ls -aF | sed '/^[.][.]*\//d;/\/$/p;d' | wc -l` -eq 0 ]
  then
    return 0
  fi
  for sub in `ls -aF | sed '/^[.][.]*\//d;/\/$/s/.$//p;d'`
  do
    cd $sub
    total_files `expr $1 + 2` $sub
    cd ..
  done
  return 0
}
total_files $tab $dir
Nopius.

You must not have correctly read my solution as is does *not* total the blocks used by the directory.

Anyway, here's a short, simple solution that works and provides the same output (but with correct totals).

#!/bin/sh
total=`ls -Fl|egrep -vc '/|total'`
echo "`pwd` [$total]"

for dir in `find * -type d`
do
  size=`echo $dir | awk -F/ '{print NF}'`
  total=`ls -Fl $dir|egrep -vc '/|total'`
  printf "%${size}s %s [%s]\n" ' ' `basename $dir`  $total
done
Tintin, I'm sorry.

Yes, it works, I've just tested it.
The only difference - all subdirectories are shown in the same level.
SUNWut [0]
   config [0]
    displays [3]
    idle [4]
    dispinfo [3]
    rescache [0]
    xconfig [3]
   session_proc [3]
   sessions [0]
    4 [0]
    2 [0]
    3 [0]
   units [0]
    IEEE802.080020b91d2e [0]
     devices [0]
     dev [0]
    IEEE802.080020d17eb0 [0]
     devices [0]
     dev [0]
   mnt [0]
    artyomk [0]
   dev [0]
    utaudio [6]
  hsperfdata_artyomk [0]
  hsperfdata_root [1]
  mc-artyomk [0]


Here is an example of an error:
   mnt [0] is really below sessions [0] / 4 [0]

while in that output it's on the same level as sessions [0]
Titin, I'm sorry once again. Your program is correct and it works faster.
It breaks only on special files (symlinks, pipes etc).
Probably if you fix it it would be better solution.

Here is the output, from yours (first column) and mine (second column) scripts.

/tmp/SUNWut [1]  /tmp/SUNWut [2]
  config [0]      config [0]
   tokens [0]       ctokens [3]
    pseudo [2]      dispinfo [3]
    Payflex [1]     displays [3]
    unknown [0]     idle [4]
   displays [3]     itokens [3]
   ctokens [3]      rescache [0]
   itokens [3]      tokens [0]
   idle [4]           Payflex [1]
   dispinfo [3]       pseudo [2]
   rescache [0]       unknown [0]
   xconfig [3]      xconfig [3]
  dev [0]         dev [0]
   utaudio [6]      utaudio [6]
  mnt [0]         mnt [0]
   artyomk [0]      artyomk [1]
  session_info [3]        session_info [3]
  session_proc [3]        session_proc [3]
  sessions [0]    sessions [0]
   4 [0]            2 [2]
   2 [0]            3 [2]
   3 [0]            4 [1]
  units [0]       units [0]
   IEEE802.080020b91d2e [0]         IEEE802.080020b91d2e [0]
    devices [0]       dev [0]
    dev [0]           devices [0]
   IEEE802.080020d17eb0 [0]         IEEE802.080020d17eb0 [0]
    devices [0]       dev [0]
    dev [0]           devices [0]
Are you sure you cut and pasted correctly?  Here's the output I get from /opt/SUNwexplo (note that subdirs are indented)

/opt/SUNWexplo [0]
  bin [18]
  doc [5]
  java [2]
  lib [18]
   locale [0]
    C [0]
     LC_MESSAGES [2]
  man [1]
   man1m [1]
   man4 [1]
   man5 [1]
  output [2]
    patch+pkg [10]
    sysconfig [49]
     drv [81]
     crash [2]
    disks [28]
     prtvtoc [1]
     ufs [6]
     sds [8]
      etc [0]
       lvm [7]
     t3 [1]
    etc [23]
     opt [0]
      SUNWexplo [5]
       instinfo [1]
     default [25]
     inet [8]
     init.d [1]
     shells [2]
    fru [2]
    init [0]
     rc0.d [46]
     rc1.d [43]
     rc2.d [63]
     rc3.d [16]
     rcS.d [55]
    j2se [1]
     usr_java [5]
     usr_j2se [5]
    var [8]
     ldap [0]
      client [1]
      server [1]
     sun [1]
      install-ORIG [5]
     cron [6]
      log [2]
     ntp [0]
      ntpstats [0]
    lp [9]
    messages [3]
    netinfo [23]
     ndd [10]
      ip [66]
      tcp [72]
      udp [15]
      icmp [9]
      eri.0 [32]
      lo.0 [2]
     ticlts-hosts [1]
     ticots-hosts [1]
     ticotsord-hosts [1]
    san [1]
    sc [1]
    system [0]
     admin [1]
       patch+pkg [10]
    sysconfig [50]
     drv [93]
     crash [2]
    disks [28]
     prtvtoc [2]
     ufs [8]
     sds [8]
      etc [0]
       lvm [7]
     t3 [1]
    etc [24]
     opt [0]
      SUNWexplo [5]
     default [26]
     inet [8]
     init.d [1]
     shells [2]
    fru [2]
    init [0]
     rc0.d [47]
     rc1.d [46]
     rc2.d [65]
     rc3.d [21]
     rcS.d [55]
    j2se [1]
     usr_java [5]
     usr_j2se [5]
    var [8]
     ldap [0]
      client [1]
      server [1]
     sun [1]
      install-ORIG [5]
     cron [6]
      log [2]
     ntp [0]
      ntpstats [0]
    lp [13]
    messages [7]
     logs [0]
      var-adm [1]
      var-log [3]
    netinfo [23]
     ndd [10]
      ip [67]
      tcp [75]
      udp [15]
      icmp [9]
      eri.0 [32]
      lo.0 [2]
     ticlts-hosts [1]
     ticots-hosts [1]
     ticotsord-hosts [1]
    san [1]
    sc [1]
    system [0]
     admin [1]
  tmp [0]
  tools [54]
It breaks on hidden files (that starts with a dot), which are not accounted with ls -Fl.
That's your version without that bug and I agree, it's better.

#!/bin/sh
total=`ls -Fl|egrep -vc '/|total'`
echo "`pwd` [$total]"

for dir in `find * -type d`
do
  size=`echo $dir | awk -F/ '{print NF}'`
  total=`ls -Fa $dir | egrep -vc '^(\.\.\/)|^(\.\/)'`
  printf "%${size}s %s [%s]\n" ' ' `basename $dir`  $total
done
Simple change to include hidden files.

#!/bin/sh
total=`ls -Fa|grep -vc /`
echo "`pwd` [$total]"

for dir in `find * -type d`
do
  size=`echo $dir | awk -F/ '{print NF}'`
  total=`ls -Fa $dir|grep -vc /`
  printf "%${size}s %s [%s]\n" ' ' `basename $dir`  $total
done
Now you need more change to include hidden directoryes, since they are also not traversed with a find command by default...
This version works with hidden directories too :-)

#!/bin/sh
total=`ls -Fa $dir | egrep -vc /`
echo "`pwd` [$total]"

for dir in `find .[!.]* * \( -name '.[!.]*' -o -name \* \) -type d`
do
  size=`echo $dir | awk -F/ '{print NF}'`
  total=`ls -Fa $dir | egrep -vc /`
  printf "%${size}s %s [%s]\n" ' ' `basename $dir`  $total
done
it still breaks on directories with spaces... Now is the very last version :-)

#!/bin/sh
total=`ls -Fa $dir | egrep -vc /`
echo "`pwd` [$total]"

for dir in `find .[!.]* * \( -name '.[!.]*' -o -name \* \) -type d 2> /dev/null`
do
  size=`echo $dir | awk -F/ '{print NF*2}'`
  total=`ls -Fa "$dir" | egrep -vc /`
  printf "%${size}s %s [%s]\n" ' ' `basename $dir`  $total
done


the above version breaks on directories with spaces. My program also breaks on such directories.

also it breaks on that file name 'The SCO Group, Inc. | Developers | SCO Developer Network.html'

it's more serious problem

Nopius.

I'm not aware of any version of find (and certainly not on Solaris) not following hidden directories.

Anyway, until cbeverly gets back to us, it's probably not worth considering every single potential combination of directories and files when the original specs only show standard directories.
Titin

man find, then search for -name, then read:

Unless  the  character  '.'  is   explicitly
                     specified  in  the  beginning  of pattern, a
                     current file name beginning  with  '.'  does
                     not  match pattern when using /usr/bin/find.
                     

I disaggree with you about any potential combination. Usually such commands are executed from root. I have this file name with '|', it was not intentionnaly created, but the results of such command may be very destructive (incorrect counters is the less problem in that case):
ls -Fa la | lalala | egrep -c ...

ASKER CERTIFIED SOLUTION
Avatar of Arty K
Arty K
Flag of Kazakhstan image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Nopius.

Regarding find and hidden dirs.  Note the man page mentions "file" names and not "dir" names.

Here's proof it works.

$ mkdir -p foo/.bar
$ mkdir foo/bar1
$ find foo -type d
foo
foo/.bar
foo/bar1
I agree, it works
But in current directory, this command will not expand to .dir subdirs (this pattern is used in your script): find * -type d

With your  example:
$ mkdir foo
$ mkdir foo/.bar
$ mkdir foo/bar
$ cd foo
$ find * -type d
bar
$
Nopius, thank you so much (and to everyone else also)
This program works perfectly and does exactly what I needed it to do.
I really appreciate it !!!