Unix ksh: scripting

Dear Expert,

Upon arrival of a file a script is kicked off.

1) first thing it does is a check on available resources.
That check creates an outputfile.
Output file example is like this:

CPU#Resource       Available Qty  UsedBy
WKS1#SMALL               1       1     No holders of this resource       
WKS1#LARGE               1       1     No holders of this resource
WKS1#SMALL_LARGE    2       2     No holders of this resource

2) the script should read this output file and check which resource has a qty > 0.
This should be set off to a variable holding number of records ($REC).

If resource >0 = WKS1#SMALL       and $REC << 2500       => action1
If resource >0 = WKS1#LARGE       and $REC  > 2500    => action2
If resource >0 = WKS1#SMALL_LARGE        => action3 (size doesn't matter)

3) If the outputfile shows that no resources are free like in below example it should redo the check each 2 minutes until an appropriate resource is found.
CPU#Resource            Available Qty         UsedBy
WKS1#SMALL              1             0             JOB1
WKS1#LARGE              1             0             JOB2
WKS1#SMALL_LARGE   2             0             JOB3
    Possible scenario's here:
If resource >0 = WKS1#SMALL       and $REC  > 2500   => redo check
If resource >0 = WKS1#LARGE       and $REC << 2500   => redo check

I all comes down to this in fact:

Queue 1: Dedicated to Small requests
Queue 2: Small and Large
Queue 3: Small and Large
Queue 4: Dedicated to Large requests

I understand this may not be too obvious, and not so easy either.
Let me know if you need more explanation, and also if you would think this is worth more than 500.
Thank you very much.
Who is Participating?
WatnogAuthor Commented:
I will close this question.
Possibly cut into pieces and rephrase it.
Thanks anyway.
perl -an - <<'EOF' outputfile                                                              
  next unless $F[2];                                                                        
  if( $F[2] < 2500 && $F[0] eq 'WKS1#SMALL' ){                                              
    system $a="action1";                                                                    
  }elsif( $F[2] > 2500 && $F[0] eq 'WKS1#LARGE' ){                                          
    system $a="action2";                                                                    
  if( $F[0] eq 'WKS1#SMALL_LARGE' ){                                                        
    system $a="action3";                                                                    
}continue{ sleep 120 and seek ARGV,0,0 if !$a && eof                                        
WatnogAuthor Commented:
Thank you Ozo but I couldn't make it work.

This is what I get:
+ perl -an - outputfile
+ 0< /tmp/sh26304.1

There is no reference to 'number of records' [$REC] but when I change 2500 to $REC that does not make any difference.

I don't mean to chase you on this though.
Cloud Class® Course: SQL Server Core 2016

This course will introduce you to SQL Server Core 2016, as well as teach you about SSMS, data tools, installation, server configuration, using Management Studio, and writing and executing queries.

I was confused about where you were getting $REC
From your examples, it looked like was from the Qty column.
Could you clarify what $REC refers to?
Also, where does "resource" come from?
And what should be done if more than one of the 5 conditions you list are satisfied?
WatnogAuthor Commented:
Ok, sorry Ozo.

The context is a 'job scheduling' software.
A limited number of jobs (4) are allowed to run in parallel.
Those jobs process files.
The number of jobs is limited to 4 by assigning to each job a 'need' for a (virtual) resource (this long time ago used to be physical, a tape drive or so).
The number of resources is limited to 4.
Some files are small others are large.
The are transferred together (mget), so upto 10 files can land at a time.

§REC is generated elsewhere in the script (a [wc] reads number of records in the file).

I'm searching for the most efficient way of processing, in fact mimic some kind of load balancing, so that the max potential of 4 jobs is used while ensuring that at all times at least 1 resource is reserved for small and 1 for large files.  The two others can be either small or large, or both large or both small.

The check on availble resources  is a commmand that outputs a snapshot like this:
(I made the names a bit shorter ...)

CPU#Resource       Available Qty  UsedBy
SMALL                      1              1     No holders of this resource      
LARGE                      1              1     No holders of this resource
SMALL_LARGE          2              2     No holders of this resource

Available= total
Qty=remaining, so that is the column to look at.

Your question on 5 conditions: I hope these are the ones...
If there would be a choice (small file and both SMALL and SMALL_LARGE is available, SMALL would be first choice).

file is either large or small (smaller or larger than 2500 records)
1) if < 2500 and recource small is available: small
2) if > 2500 and resource large is available: large
3) if  < 2500 and resource small_large is availble: small_large
4) if  > 2500 and resource small_large is availble: small_large
5) if no resource is available: wait and redo check (re-produce available resources snapshot)

So the script begins like this:

#functions per chosen queue/resource
action1 ()
kick of job using SMALL resource
action2 ()
kick of job using LARGE resource
action3 ()
kick of job using SMALL_LARGE resource
`check_available_resources_cmd` > snapshot
`REC=[cat inputfile | wc -l `                                 #
`your lines`                                                         # hopefully

A whole lot of explaining, hope you got the picture.
This check would be done for each file (one by one), the process for that is in place.

If you think this is all too much asked, well maybe it is, you can just say it...

WatnogAuthor Commented:
Ok for me.
WatnogAuthor Commented:
Too complex a question I suppose.
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.