Solved

Linux script: monitoring directories.

Posted on 2010-11-10
11
718 Views
Last Modified: 2012-06-21
I am not a linux scripting guy, therefore I need to know how to do the following steps in a linux script.

Files are daily uploaded to a directory “UploadDir”, it could be one, two or three new files,  the script will automatically knows about the new files and move only the new files to a queue directory “QueueDir” with a sequence number so I can process one file at a time by first moving it to another directory “ProcessingDir”. I need this to happen dynamically.

Thanks  a lot.
Abdellah
0
Comment
Question by:AbdellahT
  • 6
  • 4
11 Comments
 
LVL 4

Assisted Solution

by:torque_200bc
torque_200bc earned 50 total points
ID: 34105318
do you need the script to move the file to 'Processing Dir'?
0
 

Author Comment

by:AbdellahT
ID: 34105377
Yes.  at this point I need to make sure that the processingDIR is empty then I would move the file to it. because my third party application only processes one file at a time.
Thanks
0
 
LVL 7

Accepted Solution

by:
Hatrix76 earned 450 total points
ID: 34105654
This is rather easy, which linux are you using? If you use debian or ubuntu
install the package incron. (apt-get install incron).

Edit /etc/incron.allow and add the users you want to be able to execute incron scripts. (one user per line).

then, as the user you want to execute the scripts, call:

incrontab -e

now enter:
/directory IN_CLOSE_WRITE /path/to/copyscript $@/$#


where /directory is the directory you want to monitor.

IN_CLOSE_WRITE is the event to monitor, in this case the copyscript will be called when the file is fully-written.

/path/to/copyscript is the script you want to execute.

$@ is the path to the file and $# is the filename, so $@/$# puts the complete file path as a parameter to the copyscript.


for more explanations check /usr/share/doc/incron/README on debian or ubuntu, if you use another distribution you have to check where you have this docs.


Now you can create a script which does the copy process, a simple script would be:


#!/bin/bash
mv $1 /path/to/QueueDir


if you need id's for the filenames try something like uuid (apt-get install uuid).

best,

ray
0
Master Your Team's Linux and Cloud Stack!

The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.

 

Author Comment

by:AbdellahT
ID: 34106233
It sounds interesting, but I am using SUSE10 and I don't have access to the repos.
I could not find a good rpm for suse 10.
Thanks anyway.
Abdellah
0
 
LVL 7

Expert Comment

by:Hatrix76
ID: 34106497
Maybe it is already installed, try if you can execute incrontab
0
 

Author Comment

by:AbdellahT
ID: 34106577
I found an rpm and installed, however I don't see /etc/incron.allow nor .deny.  may be this done differenty I will find out.
Now I am still trying to understand the parameters:  $@/ $#
are these varaibles that i need to replace with different values or how this works?
Thanks
by the way, I did read the README file .

 
0
 
LVL 7

Expert Comment

by:Hatrix76
ID: 34106719
no, this should be fine

$@ get's replaced with the path to the new file
$# get's replaced with the filename,

so $@/$# will give you the path, then a "/", then the filename.

It's just a flexible way for cases where you need the differents parts of the path seperated.

I just cross-checked and I believe the incron.deny and incron.allow files are debian specific addons to specify users allowed and not allowed to use incron, if you do not have them, it's possible that in SUSE everyone has the right to install incron jobs per default.

0
 

Author Comment

by:AbdellahT
ID: 34107112
If you would like to help understan the command well, what will be the command for this?
once the two files LPL_1234567.zip and LPL_123456789.zip are  done being written to directory "upload" I want to move them to directory "Queue" .
 
0
 

Author Comment

by:AbdellahT
ID: 34107289

When I try saving a command to incrontab I get this message and nothing is saved.
"cannot move temporary table: Permission denied"
my /etc/permission.d/incron looks like this.
/etc/incron.d                   root:root       0755
/var/spool/incron               incron:root       0755
/usr/bin/incrontab             incron:root       4755
/usr/sbin/incrond               root:root       0755
and I also tried
/etc/incron.d                   root:root       0755
/var/spool/incron               root:root       0755
/usr/bin/incrontab              root:root       4755
/usr/sbin/incrond               root:root       0755

same error masage.
any idea?

 
0
 
LVL 7

Expert Comment

by:Hatrix76
ID: 34109716
I assume you are trying this as root?

if so, please try the following and attach the output:

strace -f crontab -e

then we see where the problem is,

best
ray
0
 

Author Closing Comment

by:AbdellahT
ID: 34139228
Could not implemented it.
0

Featured Post

Master Your Team's Linux and Cloud Stack

Come see why top tech companies like Mailchimp and Media Temple use Linux Academy to build their employee training programs.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Linux tcp ports listening for testing 4 63
AWS - HAProxy- KeepAlived 5 43
networking details on centos 6.6 4 62
ifconfig 4 46
How to remove superseded packages in windows w60 or w61 installation media (.wim) or online system to prevent unnecessary space. w60 means Windows Vista or Windows Server 2008. w61 means Windows 7 or Windows Server 2008 R2. There are various …
Google Drive is extremely cheap offsite storage, and it's even possible to get extra storage for free for two years.  You can use the free account 15GB, and if you have an Android device..when you install Google Drive for the first time it will give…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question