Solved

Linux script: monitoring directories.

Posted on 2010-11-10
11
720 Views
Last Modified: 2012-06-21
I am not a linux scripting guy, therefore I need to know how to do the following steps in a linux script.

Files are daily uploaded to a directory “UploadDir”, it could be one, two or three new files,  the script will automatically knows about the new files and move only the new files to a queue directory “QueueDir” with a sequence number so I can process one file at a time by first moving it to another directory “ProcessingDir”. I need this to happen dynamically.

Thanks  a lot.
Abdellah
0
Comment
Question by:AbdellahT
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 4
11 Comments
 
LVL 4

Assisted Solution

by:torque_200bc
torque_200bc earned 50 total points
ID: 34105318
do you need the script to move the file to 'Processing Dir'?
0
 

Author Comment

by:AbdellahT
ID: 34105377
Yes.  at this point I need to make sure that the processingDIR is empty then I would move the file to it. because my third party application only processes one file at a time.
Thanks
0
 
LVL 7

Accepted Solution

by:
Hatrix76 earned 450 total points
ID: 34105654
This is rather easy, which linux are you using? If you use debian or ubuntu
install the package incron. (apt-get install incron).

Edit /etc/incron.allow and add the users you want to be able to execute incron scripts. (one user per line).

then, as the user you want to execute the scripts, call:

incrontab -e

now enter:
/directory IN_CLOSE_WRITE /path/to/copyscript $@/$#


where /directory is the directory you want to monitor.

IN_CLOSE_WRITE is the event to monitor, in this case the copyscript will be called when the file is fully-written.

/path/to/copyscript is the script you want to execute.

$@ is the path to the file and $# is the filename, so $@/$# puts the complete file path as a parameter to the copyscript.


for more explanations check /usr/share/doc/incron/README on debian or ubuntu, if you use another distribution you have to check where you have this docs.


Now you can create a script which does the copy process, a simple script would be:


#!/bin/bash
mv $1 /path/to/QueueDir


if you need id's for the filenames try something like uuid (apt-get install uuid).

best,

ray
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:AbdellahT
ID: 34106233
It sounds interesting, but I am using SUSE10 and I don't have access to the repos.
I could not find a good rpm for suse 10.
Thanks anyway.
Abdellah
0
 
LVL 7

Expert Comment

by:Hatrix76
ID: 34106497
Maybe it is already installed, try if you can execute incrontab
0
 

Author Comment

by:AbdellahT
ID: 34106577
I found an rpm and installed, however I don't see /etc/incron.allow nor .deny.  may be this done differenty I will find out.
Now I am still trying to understand the parameters:  $@/ $#
are these varaibles that i need to replace with different values or how this works?
Thanks
by the way, I did read the README file .

 
0
 
LVL 7

Expert Comment

by:Hatrix76
ID: 34106719
no, this should be fine

$@ get's replaced with the path to the new file
$# get's replaced with the filename,

so $@/$# will give you the path, then a "/", then the filename.

It's just a flexible way for cases where you need the differents parts of the path seperated.

I just cross-checked and I believe the incron.deny and incron.allow files are debian specific addons to specify users allowed and not allowed to use incron, if you do not have them, it's possible that in SUSE everyone has the right to install incron jobs per default.

0
 

Author Comment

by:AbdellahT
ID: 34107112
If you would like to help understan the command well, what will be the command for this?
once the two files LPL_1234567.zip and LPL_123456789.zip are  done being written to directory "upload" I want to move them to directory "Queue" .
 
0
 

Author Comment

by:AbdellahT
ID: 34107289

When I try saving a command to incrontab I get this message and nothing is saved.
"cannot move temporary table: Permission denied"
my /etc/permission.d/incron looks like this.
/etc/incron.d                   root:root       0755
/var/spool/incron               incron:root       0755
/usr/bin/incrontab             incron:root       4755
/usr/sbin/incrond               root:root       0755
and I also tried
/etc/incron.d                   root:root       0755
/var/spool/incron               root:root       0755
/usr/bin/incrontab              root:root       4755
/usr/sbin/incrond               root:root       0755

same error masage.
any idea?

 
0
 
LVL 7

Expert Comment

by:Hatrix76
ID: 34109716
I assume you are trying this as root?

if so, please try the following and attach the output:

strace -f crontab -e

then we see where the problem is,

best
ray
0
 

Author Closing Comment

by:AbdellahT
ID: 34139228
Could not implemented it.
0

Featured Post

NFR key for Veeam Backup for Microsoft Office 365

Veeam is happy to provide a free NFR license (for 1 year, up to 10 users). This license allows for the non‑production use of Veeam Backup for Microsoft Office 365 in your home lab without any feature limitations.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Fine Tune your automatic Updates for Ubuntu / Debian
In the first part of this tutorial we will cover the prerequisites for installing SQL Server vNext on Linux.
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Suggested Courses

739 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question