Solved

Linux script: monitoring directories.

Posted on 2010-11-10
11
719 Views
Last Modified: 2012-06-21
I am not a linux scripting guy, therefore I need to know how to do the following steps in a linux script.

Files are daily uploaded to a directory “UploadDir”, it could be one, two or three new files,  the script will automatically knows about the new files and move only the new files to a queue directory “QueueDir” with a sequence number so I can process one file at a time by first moving it to another directory “ProcessingDir”. I need this to happen dynamically.

Thanks  a lot.
Abdellah
0
Comment
Question by:AbdellahT
  • 6
  • 4
11 Comments
 
LVL 4

Assisted Solution

by:torque_200bc
torque_200bc earned 50 total points
ID: 34105318
do you need the script to move the file to 'Processing Dir'?
0
 

Author Comment

by:AbdellahT
ID: 34105377
Yes.  at this point I need to make sure that the processingDIR is empty then I would move the file to it. because my third party application only processes one file at a time.
Thanks
0
 
LVL 7

Accepted Solution

by:
Hatrix76 earned 450 total points
ID: 34105654
This is rather easy, which linux are you using? If you use debian or ubuntu
install the package incron. (apt-get install incron).

Edit /etc/incron.allow and add the users you want to be able to execute incron scripts. (one user per line).

then, as the user you want to execute the scripts, call:

incrontab -e

now enter:
/directory IN_CLOSE_WRITE /path/to/copyscript $@/$#


where /directory is the directory you want to monitor.

IN_CLOSE_WRITE is the event to monitor, in this case the copyscript will be called when the file is fully-written.

/path/to/copyscript is the script you want to execute.

$@ is the path to the file and $# is the filename, so $@/$# puts the complete file path as a parameter to the copyscript.


for more explanations check /usr/share/doc/incron/README on debian or ubuntu, if you use another distribution you have to check where you have this docs.


Now you can create a script which does the copy process, a simple script would be:


#!/bin/bash
mv $1 /path/to/QueueDir


if you need id's for the filenames try something like uuid (apt-get install uuid).

best,

ray
0
Master Your Team's Linux and Cloud Stack!

The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.

 

Author Comment

by:AbdellahT
ID: 34106233
It sounds interesting, but I am using SUSE10 and I don't have access to the repos.
I could not find a good rpm for suse 10.
Thanks anyway.
Abdellah
0
 
LVL 7

Expert Comment

by:Hatrix76
ID: 34106497
Maybe it is already installed, try if you can execute incrontab
0
 

Author Comment

by:AbdellahT
ID: 34106577
I found an rpm and installed, however I don't see /etc/incron.allow nor .deny.  may be this done differenty I will find out.
Now I am still trying to understand the parameters:  $@/ $#
are these varaibles that i need to replace with different values or how this works?
Thanks
by the way, I did read the README file .

 
0
 
LVL 7

Expert Comment

by:Hatrix76
ID: 34106719
no, this should be fine

$@ get's replaced with the path to the new file
$# get's replaced with the filename,

so $@/$# will give you the path, then a "/", then the filename.

It's just a flexible way for cases where you need the differents parts of the path seperated.

I just cross-checked and I believe the incron.deny and incron.allow files are debian specific addons to specify users allowed and not allowed to use incron, if you do not have them, it's possible that in SUSE everyone has the right to install incron jobs per default.

0
 

Author Comment

by:AbdellahT
ID: 34107112
If you would like to help understan the command well, what will be the command for this?
once the two files LPL_1234567.zip and LPL_123456789.zip are  done being written to directory "upload" I want to move them to directory "Queue" .
 
0
 

Author Comment

by:AbdellahT
ID: 34107289

When I try saving a command to incrontab I get this message and nothing is saved.
"cannot move temporary table: Permission denied"
my /etc/permission.d/incron looks like this.
/etc/incron.d                   root:root       0755
/var/spool/incron               incron:root       0755
/usr/bin/incrontab             incron:root       4755
/usr/sbin/incrond               root:root       0755
and I also tried
/etc/incron.d                   root:root       0755
/var/spool/incron               root:root       0755
/usr/bin/incrontab              root:root       4755
/usr/sbin/incrond               root:root       0755

same error masage.
any idea?

 
0
 
LVL 7

Expert Comment

by:Hatrix76
ID: 34109716
I assume you are trying this as root?

if so, please try the following and attach the output:

strace -f crontab -e

then we see where the problem is,

best
ray
0
 

Author Closing Comment

by:AbdellahT
ID: 34139228
Could not implemented it.
0

Featured Post

Ransomware: The New Cyber Threat & How to Stop It

This infographic explains ransomware, type of malware that blocks access to your files or your systems and holds them hostage until a ransom is paid. It also examines the different types of ransomware and explains what you can do to thwart this sinister online threat.  

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Over the years I've spent many an hour playing on hardened, DMZ'd servers, with only a sub-set of the usual GNU toy's to keep me company; frequently I've needed to save and send log or data extracts from these server back to my PC, or to others, and…
Setting up Secure Ubuntu server on VMware 1.      Insert the Ubuntu Server distribution CD or attach the ISO of the CD which is in the “Datastore”. Note that it is important to install the x64 edition on servers, not the X86 editions. 2.      Power on th…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.

821 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question