• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1045
  • Last Modified:

as400 slowdown

I am pretty novice at as400 problems but need some help on this.  We have an iseries 520 and about 3 weeks ago when we run reports it is taking much longer than normal.  Our payroll manager has a payroll edit register that she runs and it usually takes about 7 minutes to complete.  It compiles all of the employees info for the payroll check run.  Now it is taking 50 minutes to run.  This just happened 3 weeks ago on a Wednesday when we run this job.  Before that it was fine for as long as I can remember.  ANything I can check?  I looked at wrkactjob and I can see that the job is taking up 20% of the cpu. There is also another job with the status of run called QRWTRSVR type is PJ and it is at 5.7% cpu.  It keeps going back and forth with a status of TIMA and RUN.
  • 2
1 Solution
Gary PattersonVP Technology / Senior Consultant Commented:
Dramatically increased runtimes can have a LOT of causes, and it probably isn't going to be possible to diagnose this at a distance.

First question:  What changed in the system 3 weeks ago?  

Did you load a bunch of new data onto the system for some reason?  Are the new or updated programs running?  Was this program changed?  Were there any system tuning changes made?  Did you apply PTFs or vendor updates?

You mention QRWTSVR.  This is a system prestart job that services IBM DRDA coming in over TCP/IP.  It shows usage when a remote system established a DRDA database connection to the AS/400

Look at the job log from the long-running job.  Are there errors?  

Are you running Collection Services to capture system performance data? (GO PERFORM, Option 2 - Status=Started?).  If so, and you have performance data from before the problem, it would be possible to compare and see what is different.

Jobs run longer than usual for the following reasons:

Errors or contention with other jobs (maybe something else is locking a file this one needs).

Increased file sizes (maybe as the result of new business or maybe just the result of a program error).

Increased competition for CPU, memory, I/O or other limited resources by new or changed programs.

System tuning changes - someone moved memory to a different pool, or turned off auto-tune.

Are any other users complaining of problems, or any other processes experiencing runtime issues?

- Gary Patterson
mkramer777Author Commented:
It is only the payroll manager that is experiencing these problems.  Also I have noticed that when users sign on to the system (which defaults to send them to the our accounting software main menu) this is taking longer than it has before.  A month or so ago when a user would sign on it would bring them to the main menu of our software in about 2 seconds.  Now it takes about 10-15.
Gary PattersonVP Technology / Senior Consultant Commented:
No answer to "what changed" questions, so I can't give much guidance.
I have a client that is a small AS/400 shop that starts to experience slowdowns about this time every year - actually it starts to get noticeable in September, and gets worse until the end of the year.  

Running their year-end process fixes the problem and they are good for another 8-9 months.  They've decided to live with it since they are involved in a search for a new system.

Anyway, in their case the problem is just due to fact that the application they use doesn't purge or archive transactional data until year-end, and in the last two quarters, the file sizes get a little big for optimum performance on their small system.  This particular application doesn't perform well when the transactional tables are too big (not uncommon).  

File grown performance issues tend to sneak up on you - gradual degradation each day or week until one day a user starts to complain, and then you get a cascade of similar "slow" complaints.  Unless you do regular performance monitoring and compare back to a baseline, you probably won't know until it has become an "issue".

Maybe that is what is happening here if nothing else has changed.

Anyway, diagnosing performance problems can be complicated.  In general, you start wide and go narrow, using the following tools.  Some of these tools are not part of the base OS, so you may not have them.  

In that case, you might want to engage a consultant that can take your data and analyze it for you.  You might want to do that anyway.  I've seen a lot of performance and database disasters that resulted from inexperienced technicians trying to "fix" performance problems by changing something they saw on Google.

1) Performance Collector / Performance Tools allow you to look at overall system performance and trends, and identify timeframes with poor performance.  There are also Navigator and iDoctor interface that allows you to view performance data in a variety of graphical formats.

2) iDoctor: Job Watcher allows you to zero in on individual jobs and determine how those jobs are spending their time, and held determine which specific programs are responsible for delays

3) Performance Explorer: PEX is a program profiling and trace tool that allows you to profile the entire system, individual jobs, or individual programs of modules and determine down to the line level of code where they are spending their time.

I'll warn you, there is a significant learning curve on all of these tools, and this is a big topic.  Here are some useful links:




Lots of additional version-specific resources in the Information Center for your OS release:


Search for the tools above, or just "performance".

- Gary Patterson

Check out my EE profile for more AS/400 Performance resources:  http://www.experts-exchange.com/M_4382324.html
I don't suppose you have both a current joblog and an old joblog from before the slowdown? Timestamps in joblogs can help narrow things down significantly sometimes. (That question is a real shot-in-the-dark.)

The QRWTSRVR job is interesting. There should be no reason for it to take up any significant CPU at all unless there are active DRDA/DDM requests being serviced. At 5.7%, I'd be noticing.

The 20% for your job is also interesting. That's a major percentage, at least double what I'd expect for a "payroll" related job.

The point is in how those figures were obtained.

When you look next time, run WRKACTJOB at least a few seconds after your payroll job starts. Then press <F10> to ensure that all statistics are specifically from the time WRKACTJOB started. Then wait 5 or 10  minutes and press <F5>.

Then check the percentages.

You probably should not see 20% unless the interval was too short. I'd like to see a more certain figure. That much CPU would be strange.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Cloud Class® Course: Microsoft Exchange Server

The MCTS: Microsoft Exchange Server 2010 certification validates your skills in supporting the maintenance and administration of the Exchange servers in an enterprise environment. Learn everything you need to know with this course.

  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now