Solved

AIX: user is not logged in, but user processes are still running

Posted on 2013-11-18
7
817 Views
Last Modified: 2013-12-05
Hi !

Please watch this:

[root@p750 bin]# w | grep "cl00-3"

from where I can imagine the user cl00-3 is not logged in anymore.

But then you see a process eating a lot of cpu on topas: 48628662

[root@p750 bin]# pstree -p 48628662
-+- 00001 root /etc/init
 \-+= 4194592 root /usr/sbin/srcmstr
   \-+= 39781132 root /usr/sbin/inetd
     \-+= 16582992 root telnetd -ac
       \-+= 11012174 cl00-3 /bin/bash /uv6/bin/uv.login
         \--- 48628662 cl00-3 /uv1/uv/bin/uvdls
[root@p750 bin]#

last also shows the user is not logged in anymore.
It's obvious the user disconnected, probably with a "hard disconnect", and somehow AIX noted it - as it's not in the user list - but user processes are still running.

How can I trap this disconnection and kill the user proctree ?

Thanks in advance,

Ronald
0
Comment
Question by:rsekkel
  • 3
  • 3
7 Comments
 
LVL 76

Expert Comment

by:arnold
ID: 39658215
Not sufficiently familiar with AIX, but a unix/Linux system has an option to trap sighup events which is often sent to child processes on disconnect.  See whether the task is being spawn within cron if allowed.  If you have screen available, the process might be managed/controlled by screen.

There might be soothing else that is spawning this process with user credentials.
0
 

Author Comment

by:rsekkel
ID: 39659001
Hi Arnold !

Thanks for your message.

Yes, in AIX there's the "trap sighup", and we are already using it - but it's not working. It seems to me the sighup is not reaching the user login script.

This is a green screen app, and users use a windows terminal emulator to reach the server. The problem is some users do not follow the logoff procedure and just close the emulator in the "X". Somehow AIX figure out this, and take the user out of utmp, but the client process tree stay alive, eating all CPU from one thread.

Thanks,

Ronald
0
 
LVL 4

Expert Comment

by:popesy
ID: 39659019
Hi,

Trapping the disconnect may be able to be done using your syslog daemon config using the 'auth' facility and logging to a file of your choice.

If by 'hard disconnect' you mean that this user session is killed unexpectedly, I'm not sure you'll be able to trap that using syslog.  I'm not aware of how that might be captured.

Killing the proctree; assuming you want to do this manually then a simple 'kill -9 <pid>' will sort out any redundant/orphan processes.  Just remember to start with the highest value <pid> for the user in question.

If you want some automated way of doing this, that's over to you to script a solution :-)

Cheers.
0
What Security Threats Are You Missing?

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 
LVL 76

Accepted Solution

by:
arnold earned 500 total points
ID: 39661072
The application/program env would either have a built-in mechanism to trap (ignore or do some predefine function) received SIGHUP reverts.
A sighup often triggers a reset/exit.  Some programs use SIGHUP as a reload/reread settings, thus preventing auto termination/end of the process.

In user accessed systems, there are idle session cleanup script one could/would setup using cron or any available scheduler.


Type of inbound connection could also be the cause for the issue.
0
 

Author Comment

by:rsekkel
ID: 39662207
Hi all !

Thanks for your messages.

We already have a cron script to clean lost users - but it's heavy (and ugly). It has to capture all processes using ps, extract all userid's, unify them and then start checking one by one if the user is still logged in using who | grep user (for each user). When you have around 2.500 users logged in this is not the kind of script you want to run every five minutes.

The "trap exit sighup" for some (so far unknown) reason is failing. If the user is not on utmp anymore, somehow AIX got the info about the disconnection, but this is not killing the user process tree.

I'll try to get some more info. Any ideas are welcome.

Thanks,

Ron
0
 
LVL 76

Expert Comment

by:arnold
ID: 39664517
Check if your script get use last instead.  See who recently logged out, then see if any of these users have processes.
Does your AIX handle inbound email? Do users setup filtering, process to deal with emails?  That might be one way a process is started by the LDA of the mail server.

Using who to build a hash/array and then checking for the existence of the element might be another approach.
0
 

Author Closing Comment

by:rsekkel
ID: 39699466
After I fixed the login script, sighup was able to kill the user session, and so I don't have to be concerned about processess eating all cpu.

Thanks for the heads up and for the solution.

Ron
0

Featured Post

Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

Join & Write a Comment

My previous tech tip, Installing the Solaris OS From the Flash Archive On a Tape (http://www.experts-exchange.com/articles/OS/Unix/Solaris/Installing-the-Solaris-OS-From-the-Flash-Archive-on-a-Tape.html), discussed installing the Solaris Operating S…
Why Shell Scripting? Shell scripting is a powerful method of accessing UNIX systems and it is very flexible. Shell scripts are required when we want to execute a sequence of commands in Unix flavored operating systems. “Shell” is the command line i…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now