Solved

Linux script: monitoring directories.

Posted on 2010-11-10
11
712 Views
Last Modified: 2012-06-21
I am not a linux scripting guy, therefore I need to know how to do the following steps in a linux script.

Files are daily uploaded to a directory “UploadDir”, it could be one, two or three new files,  the script will automatically knows about the new files and move only the new files to a queue directory “QueueDir” with a sequence number so I can process one file at a time by first moving it to another directory “ProcessingDir”. I need this to happen dynamically.

Thanks  a lot.
Abdellah
0
Comment
Question by:AbdellahT
  • 6
  • 4
11 Comments
 
LVL 4

Assisted Solution

by:torque_200bc
torque_200bc earned 50 total points
ID: 34105318
do you need the script to move the file to 'Processing Dir'?
0
 

Author Comment

by:AbdellahT
ID: 34105377
Yes.  at this point I need to make sure that the processingDIR is empty then I would move the file to it. because my third party application only processes one file at a time.
Thanks
0
 
LVL 7

Accepted Solution

by:
Hatrix76 earned 450 total points
ID: 34105654
This is rather easy, which linux are you using? If you use debian or ubuntu
install the package incron. (apt-get install incron).

Edit /etc/incron.allow and add the users you want to be able to execute incron scripts. (one user per line).

then, as the user you want to execute the scripts, call:

incrontab -e

now enter:
/directory IN_CLOSE_WRITE /path/to/copyscript $@/$#


where /directory is the directory you want to monitor.

IN_CLOSE_WRITE is the event to monitor, in this case the copyscript will be called when the file is fully-written.

/path/to/copyscript is the script you want to execute.

$@ is the path to the file and $# is the filename, so $@/$# puts the complete file path as a parameter to the copyscript.


for more explanations check /usr/share/doc/incron/README on debian or ubuntu, if you use another distribution you have to check where you have this docs.


Now you can create a script which does the copy process, a simple script would be:


#!/bin/bash
mv $1 /path/to/QueueDir


if you need id's for the filenames try something like uuid (apt-get install uuid).

best,

ray
0
 

Author Comment

by:AbdellahT
ID: 34106233
It sounds interesting, but I am using SUSE10 and I don't have access to the repos.
I could not find a good rpm for suse 10.
Thanks anyway.
Abdellah
0
 
LVL 7

Expert Comment

by:Hatrix76
ID: 34106497
Maybe it is already installed, try if you can execute incrontab
0
Free Gift Card with Acronis Backup Purchase!

Backup any data in any location: local and remote systems, physical and virtual servers, private and public clouds, Macs and PCs, tablets and mobile devices, & more! For limited time only, buy any Acronis backup products and get a FREE Amazon/Best Buy gift card worth up to $200!

 

Author Comment

by:AbdellahT
ID: 34106577
I found an rpm and installed, however I don't see /etc/incron.allow nor .deny.  may be this done differenty I will find out.
Now I am still trying to understand the parameters:  $@/ $#
are these varaibles that i need to replace with different values or how this works?
Thanks
by the way, I did read the README file .

 
0
 
LVL 7

Expert Comment

by:Hatrix76
ID: 34106719
no, this should be fine

$@ get's replaced with the path to the new file
$# get's replaced with the filename,

so $@/$# will give you the path, then a "/", then the filename.

It's just a flexible way for cases where you need the differents parts of the path seperated.

I just cross-checked and I believe the incron.deny and incron.allow files are debian specific addons to specify users allowed and not allowed to use incron, if you do not have them, it's possible that in SUSE everyone has the right to install incron jobs per default.

0
 

Author Comment

by:AbdellahT
ID: 34107112
If you would like to help understan the command well, what will be the command for this?
once the two files LPL_1234567.zip and LPL_123456789.zip are  done being written to directory "upload" I want to move them to directory "Queue" .
 
0
 

Author Comment

by:AbdellahT
ID: 34107289

When I try saving a command to incrontab I get this message and nothing is saved.
"cannot move temporary table: Permission denied"
my /etc/permission.d/incron looks like this.
/etc/incron.d                   root:root       0755
/var/spool/incron               incron:root       0755
/usr/bin/incrontab             incron:root       4755
/usr/sbin/incrond               root:root       0755
and I also tried
/etc/incron.d                   root:root       0755
/var/spool/incron               root:root       0755
/usr/bin/incrontab              root:root       4755
/usr/sbin/incrond               root:root       0755

same error masage.
any idea?

 
0
 
LVL 7

Expert Comment

by:Hatrix76
ID: 34109716
I assume you are trying this as root?

if so, please try the following and attach the output:

strace -f crontab -e

then we see where the problem is,

best
ray
0
 

Author Closing Comment

by:AbdellahT
ID: 34139228
Could not implemented it.
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

If you have a server on collocation with the super-fast CPU, that doesn't mean that you get it running at full power. Here is a preamble. When doing inventory of Linux servers, that I'm administering, I've found that some of them are running on l…
Using 'screen' for session sharing, The Simple Edition Step 1: user starts session with command: screen Step 2: other user (logged in with same user account) connects with command: screen -x Done. Both users are connected to the same CLI sessio…
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

22 Experts available now in Live!

Get 1:1 Help Now