Link to home
Start Free TrialLog in
Avatar of pieman8080
pieman8080

asked on

Shell script to check file size

I have a linux server and there are zip files being uploaded to a particular directory, we'll call it /log/zipfiles.

I want to create a script that does the following.
- Checks the file size of all the files in /log/zipfiles, like 10 seconds apart (to see which ones are completed and which ones are still uploading)

- for the ones that have the same file size (meaning the upload is complete), i want to move another directory, we'll call it /log/done.

Can someone tell me how to write that? Let me know if my question isnt' clear.
Avatar of MatthewP
MatthewP
Flag of United Kingdom of Great Britain and Northern Ireland image

Do you mean the script needs to be triggered to run every ten seconds?

Look into cron or crontab for scheduling the task I would say, although you can only set this to run on a minutely basis or longer. If thats any good, you can perhaps call a perl script on this to do the checking - read the contents in, loop through and use stat to find the filesize for each, and I tend to use the system command to call mv if I need to move files around, although Perl does also have a rename function that will do the job I think.

If you really need it to run every ten seconds, id be tempted to just set a perl script off to read the time it started, time it ended, and with a sleep command at the very end before setting it to run again.



Avatar of pieman8080
pieman8080

ASKER

No i don't need help on the cron. I just need help on setting up the code to do what I described above. I've written a shell script and it's working for the most part, except some of the files have spaces in them so for those it doesn't work.
Hello pieman8080,

if your main problem is with space characters, you could post the part which is not working.

Your problem is listing the files ? Finding their sizes ?

Best regards
#!/bin/bash
FILENAME="foo bar.zip"
FILESIZE=$(stat -c%s "$FILENAME")
echo "Size of $FILENAME = $FILESIZE bytes."

Open in new window

here is my code so far, I "think" it's working, maybe someone with more programming experience can look at it and confirm it

To repeat, this is what i want to do:
- there are zip files currently being uploaded to /log/zipfiles
- i want to move all the files from /log/zipfiles to /log/done for ONLY the files that have completed their upload (that's why i have the 'sleep 5' to compare file sizes)
- some files have spaces in them (that's why i put the " " around the variable $i

I'm still having occasional bad zip files in the /log/done directory so I'm not sure if i'm missing something

also if there's a better or more simpler way of writing this, please let me know. thanks guys
#!/bin/sh
 
LOG=/log/zipfiles
DONE=/log/done
cd $LOG
 
for i in *; do
   LS1=`ls -l "$i" | awk '{print $5}'`
   sleep 5
   LS2=`ls -l "$i" | awk '{print $5}'`
      if [ $LS1 -eq $LS2 ]; then
          mv "$i" $DONE
      fi
done

Open in new window

Hello,

yes your code does not have (no longer have?) problem with spaces in filenames.

The problem is with your heuristic. Upload could pause for more than 5 seconds depending on network condition. A solution, will be to upload a file containing the checksum of the file and verify it along with the file and only move the file when the signature is OK.

foo.txt will be accompanied by a foo.txt.md5

You could be tempted to pass the size of the file but depending on the filesystem and how do you get the file size.

Best regards
That sounds better. How would u modify my code to do the checksum? thanks again
Let me add, I changed the code like the following, but i'm getting an error " [: too many arguments" for the line: if [ $MD51 -eq $MD52 ]; then


#!/bin/sh
 
LOG=/log/zipfiles
DONE=/log/done
cd $LOG
 
for i in *; do
   MD51=`md5sum "$i"`
   sleep 5
   MD52=`md5sum "$i"`
      if [ $MD51 -eq $MD52 ]; then
         mv "$i" $DONE
      fi
done

Open in new window

ASKER CERTIFIED SOLUTION
Avatar of davidgeldreich
davidgeldreich
Flag of France image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Thx for your help. This worked. But I ended up user the fuser command. Basically if the command 'fuser