Why so many jobs for a qshell command?

Posted on 2005-03-15
Medium Priority
Last Modified: 2008-01-09
I've created a command, thanks to the help of some of the experts here, that retrieves the record count of a .csv file in the IFS.  When I call this command interactively, I don't have any problems.  But if I call the command from a program running in batch, all heck breaks loose.  I get 5 different jobs that start, which causes my command to fail because the subsystem we run it in only allows a total of 4 jobs to run at a time.  I'll post the QSH command that I run at the end of this post.  The questions I have are as follows:

1.  Why so many Jobs for this command?
2.  Is there a way I can limit the number of Jobs a QSH command starts?

Here is the command I run.

QSH CMD('datarea -w /qsys.lib/mylib.lib/ifsrcdcnt.dtaara  `cat file.csv | wc -l`')

Any help will be appreciated.
Question by:peteie
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
  • 2
LVL 27

Expert Comment

ID: 13551028

Simple answer is that you're running a kind of emulated Unix shell on a non-Unix system. Those additional jobs are kind of the equivalent of additional threads. If you limited the number of active threads under Unix, you'd have similar results. Under OS/400, these generally appear as BCI (batch immediate) job type.

However, that doesn't address your concern about your subsystem limit of 4 jobs.

We limit our subsystems similarly, but don't have the same problem.  We place our limits on the job queues that feed the subsystem. Beyond that, we generally allow *NOMAX for the subsystems themselves.

So, while 4 jobs at a time might be allowed to enter the subsystem through job queue ABC, once a job starts, it's allowed to spawn additional helper jobs as needed. Since the only way such jobs can get into the subsystem is by an already running job spawning them off, we haven't had to be concerned about *NOMAX.

I'm not sure that limiting QShell spawned jobs is a workable direction to go. I haven't explored the possibility, but I don't see how it could be made to work.


Author Comment

ID: 13551898
I found a post on iseries network, that showed some examples of calling QShell commands just like a regular program.  Have you heard of anyone else doing this?  It seems like a feasible sollution and apparently when you call a command this way it runs in the same job that you called it from.  I've begun playing with it, but I'm not to where I would like to be yet.

As for the limiting the JOBQ to 4 jobs at a time and having the subsystem be *nomax, wouldn't that cause a bunch of jobs to be running in the subsystem at the same time?  Correct me if I'm wrong but wouldn't the following happen?

jobq max jobs = 1
subsystem max jobs = *nomax

The operator submits pgm1 to the jobq.  Since there is open space on the subsystem to run it is run, thereby taking it off the jobq.
The operator submits pgm2 to the jobq while pgm1 is still running.  Again there is open space on the subsystem to run so it too is run, again taking it off the jobq.
Now I've got 2 or more programs running even though the jobq only allows 1 at a time?
Or does the program stay on the JobQ until it is done running?

We don't actually limit the jobq or subsystem to 1 job, I just used it for simplicity.

Thanks for any help.
LVL 27

Accepted Solution

tliotta earned 1000 total points
ID: 13559359

A subsystem might have hundreds of job queues feeding it, for example say 100 jobqs. Imagine the case where each jobq is limited to a single job and the subsystem limit is 100 jobs.

A job can be placed on JOBQ1 and it will start immediately assuming there are currently no jobs from JOBQ1 already running and less than 100 jobs are active in the subsystem.

While that job is running, a second job is placed on JOBQ1 and a third is placed on JOBQ2. The job on JOBQ2 will start immediately (assuming no current JOBQ2 jobs, etc.), but the job on JOBQ1 will sit there until the first job finishes regardless of how many jobs are in the subsystem.

The number of jobs is limited by jobq in order to allow control of jobs that must run in sequence. A jobq that's limited to a single job is referred to as a "single-threaded" jobq. You can have one jobq that's set for a single job and another that's set for four jobs at the same time attached to the same subsystem. The maximum number of jobs can possibly be active from those jobqs is therefore five -- one from the first jobq and four from the second, regardless of how many jobs are queued.

The settings for different jobqs have no relationship to each other. The settings for the subsystem as a whole has no relationship to the settings on the jobqs that are attached to that subsystem -- except that a limit on the number of subsystem jobs will act as a throttle over all jobqs at any given time.

But jobs can start without being fed through a jobq. A primary example is a batch-immediate job; there are various others.

We generally use the jobq setting as our control point rather than the subsystem itself. Our basic batch subsystem always has at least one single-threaded jobq and at least one multi-threaded jobq. If sequential execution is needed, we know what jobq to submit to. Likewise if we have a whole bunch of small functions that can run concurrently without affecting each other.

Say I have a physical file with a dozen logical files over it. I want to run RGZPFM, so I remove the members from the logicals and then do the reorganization. After the file is reorganized, I need to rebuild all of the logical file access paths. Since the rebuild of one has no logical conflict with any other, I can submit the ADDLFM commands all to a multi-threaded jobq and they'll run side by side as long as they don't exceed the jobq (or subsystem) limit.

In any case, when a QShell function needs to spawn a secondary thread, there's no reason to stop it from doing so. The result would be just as much of a mess as if you stopped secondary threads under Unix (or Windows for that matter).


Author Comment

ID: 13567180

Thanks for the help.  After reviewing your suggestions with my dept. manager, we decided to raise the number of jobs allowed on the subsystem.  We only went to 10 for now.  We wanted to see what affects it might have before raising it any higher.  That was a great explanation of the relationship between JOBQs and SubSystems.


Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In the absence of a fully-fledged GPO Management product like AGPM, the script in this article will provide you with a simple way to watch the domain (or a select OU) for GPOs changes and automatically take backups when policies are added, removed o…
New style of hardware planning for Microsoft Exchange server.
In this video we outline the Physical Segments view of NetCrunch network monitor. By following this brief how-to video, you will be able to learn how NetCrunch visualizes your network, how granular is the information collected, as well as where to f…
Visualize your data even better in Access queries. Given a date and a value, this lesson shows how to compare that value with the previous value, calculate the difference, and display a circle if the value is the same, an up triangle if it increased…

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question