Avatar of YZlat
YZlatFlag for United States of America

asked on 

Need help modifying my script

I need to write a script that loops throught all the directories that contain $var in the path and get all the files with .txt extension and calculate the total size


for example if $var="sample" and directories below contain

/u01/test/samples

test1.txt 1GB
test2.txt 1GB

/u04/data/sample

sample2.txt 2Gb

then the script should add up file sizes in those directories and assing them to a variable

So far I came up with something like this:

$dir=/u01/test/samples/*.txt
du -g $dir | total_size=`awk '{c+=$1};END { print c }'`

which works great but only returns total file size for one specified directory. Can someone help fix my code so that it returns total size fo all directories containing a specific variable in the path?
Shell ScriptingUnix OS

Avatar of undefined
Last Comment
simon3270
Avatar of ozo
ozo
Flag of United States of America image

wc -c `find . -path '*/sample/*.txt'`
Avatar of YZlat
YZlat
Flag of United States of America image

ASKER

ozo, but it will not loop through all fielsystems, will it?

The thing is I have a number of different filesystems: /u01, /u02, /u03, /u04 and many more. I need to loop through them because almost every one of them have word "sample" in it
Avatar of YZlat
YZlat
Flag of United States of America image

ASKER

i get an error

find: 0652-017 -path is not a valid option.

Also wc command counts the characters not file size, I know it is basically the same thing but I feel like it is not the right way to go.

The bottom line this code does not work either way
Avatar of simon3270
simon3270
Flag of United Kingdom of Great Britain and Northern Ireland image

A command to generate the result could be:
find /u0? -type f | grep "/${var}/.*\.txt$" | xargs ls -s | awk '{c+=$1};END { print c }'

Open in new window

To get this into a variable, use
totsize=$(find /u0? -type f | grep "/${var}/.*\.txt$" | xargs ls -s | awk '{c+=$1};END { print c }')

Open in new window


(editted to add the "-type f", in case a directory name ended with ".txt")
Avatar of skullnobrains
skullnobrains

du -ch `find / -type d -name sample`

you should try and toy with the find command to make it faster. maybe something like this

du -ch `find /u* -maxdepth 4 -type d -name sample`

i guess you can figure that part out easily if you know where to expect the files

if locate is available and reasonably up to date, you may want to replace find with "locate sample"
Avatar of YZlat
YZlat
Flag of United States of America image

ASKER

simon3270, your code produces an error:

find: 0652-019 The status on /u0? is not valid.
Avatar of simon3270
simon3270
Flag of United Kingdom of Great Britain and Northern Ireland image

That's /u zero questionmark (or whatever matches the base directories you want to search)
Avatar of YZlat
YZlat
Flag of United States of America image

ASKER

yes, I tried /u zero question mark and that's what gave me the error. I also tried /u? but still same error
Avatar of simon3270
simon3270
Flag of United Kingdom of Great Britain and Northern Ireland image

I think you get that error either if there is a problem with a disk (are they NFS mounts with a dead server?  Or actual disk failures?) or simply if the path specified doesn't exist - you don't want to try text until it works - use a pattern which finds the actual disks (assuming that /u01 or /uo1 for example is the mount point for a mounted disk).  I can't see your system, so unless you copy and paste the output, for example, or "ls -p /" or "mount", I can't see what your directories are called.

If find can't get past the lowest level directory, it won't be able to check for "sample" directories - you need to fix that problem first.

I only used /u0? because you implied that you needed to check all such directories (and I didn't expect them to be /uo?).  if you only need to check some of them, first check that the ones you want work with "find" (test them one at a time with, for example, "find /u01 | head"), then just list them on the command line, like:

    find /u01 /disk2 /cc0703 /mdisk04 -type f |

(and then the rest of the line is the same as it was).
here is an example on my machine

$ du -ch `find /v* -maxdepth 2 -type d -name \*sa\*` | tail
4,0K	/var/log/boot-sav/log/2013-06-21__14h13boot-repair20/sdb1
4,0K	/var/log/boot-sav/log/2013-06-21__14h13boot-repair20/sda5
1,1M	/var/log/boot-sav/log/2013-06-21__14h13boot-repair20/sda
4,0K	/var/log/boot-sav/log/2013-06-21__14h13boot-repair20/sda1
4,0K	/var/log/boot-sav/log/2013-06-21__14h13boot-repair20/sda2
1,2M	/var/log/boot-sav/log/2013-06-21__14h13boot-repair20
7,1M	/var/log/boot-sav/log
7,1M	/var/log/boot-sav
4,0K	/var/log/samba
7,2M	total

Open in new window


try something like this

du -ch `find /u[0-9] -maxdepth 4 -type d -name sample` | cut -d \  -f 1

Open in new window

Avatar of YZlat
YZlat
Flag of United States of America image

ASKER

simon, directories are called

/u01
/u02
/u10
/u40
/u80
Avatar of YZlat
YZlat
Flag of United States of America image

ASKER

skullnobrains, what does this part do?

cut -d \  -f 1
Avatar of simon3270
simon3270
Flag of United Kingdom of Great Britain and Northern Ireland image

The command becomes:
find /u01 /u02 /u10 /u40 /u80 -type f | grep "/${var}/.*\.txt$" | xargs ls -s | awk '{c+=$1};END { print c }'

Open in new window

If that returns the same "find" error, check that each disk can be read by running:
    ls /u01 /u02 /u10 /u40 /u80 >/dev/null
and letting us know if there are any errors reported
ASKER CERTIFIED SOLUTION
Avatar of skullnobrains
skullnobrains

Blurred text
THIS SOLUTION IS ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
Avatar of simon3270
simon3270
Flag of United Kingdom of Great Britain and Northern Ireland image

I thought you only wanted to add up the size of the *.txt files, not of the entire contents of the directories called "sample".
Unix OS
Unix OS

Unix is a multitasking, multi-user computer operating system originally developed in 1969 at Bell Labs. Today, it is a modern OS with many commercial flavors and licensees, including FreeBSD, Hewlett-Packard’s UX, IBM AIX and Apple Mac OS-X. Apart from its command-line interface, most UNIX variations support the standardized X Window System for GUIs, with the exception of the Mac OS, which uses a proprietary system.

33K
Questions
--
Followers
--
Top Experts
Get a personalized solution from industry experts
Ask the experts
Read over 600 more reviews

TRUSTED BY

IBM logoIntel logoMicrosoft logoUbisoft logoSAP logo
Qualcomm logoCitrix Systems logoWorkday logoErnst & Young logo
High performer badgeUsers love us badge
LinkedIn logoFacebook logoX logoInstagram logoTikTok logoYouTube logo