"system" using 99% of cpu

Hi, one of my server with windows 2003 the SYSTEM is using hogging all the cpu. started this morning. it's weird started after all the sudden server got really slow and lots of users couldnt log saying windows wasnt activated and to contact administrator.

i then run all the windows update that where critical and rebooted, when i log i got that same message myself but had the option to activate, i did and then that solved that.

but now most of the time my cpu is used at almost 100 and its always SYSTEM thats using it (not system idle process like usual)

im in the mid of running trendmicro to see if anything happend but so far nothing. and i didnt see anything odd with a hijackthis scan :/
planetemAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

briancassinCommented:
first off you'll want to download and run process explorer from systernals then go to view and select show dlls in the lower pane see. This will show exactly what is using all the cpu time.

http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx

then get hijack this run it and post a logfile

http://www.tomcoyote.org/hjt 

then if necessary we will go to the other tools and make sure it is not a virus or malware

Download  combofix.exe and save it to your desktop
Close any open browsers.
Before starting ComboFix disable and exit any anti-virus software, anti-spyware or any other security related software as they may interfere with ComboFix's operation.
Double click combofix.exe & follow the prompts.
When finished, it shall produce a log for you and display it on your desktop called c:\combofix.txt. By default this log is located on your 'C' drive. Post that log in your next reply along with a fresh HJT log as well
Note:Do not mouseclick combofix's window whilst it's running. That may cause it to stall.
Combofix http://download.bleepingcomputer.com/sUBs/ComboFix.exe

Also go to http://www.tomcoyote.org/hjt download hijack this run it and post the logfile here


If combofix reports it removed files then continue on through this if not then post it's logfile and a hijack this logfile

I would then get

http://www.ccleaner.com   download it and run it to clear out all the windows junk files and make the scans faster.

http://www.superantispyware.com download it update it and run it

http://security.kolla.de spybot s&d - download it install it (do not install tea timer, ) update it then run it

http://lavasoft.com - adaware - download it run it and then uninstall it
http://pack.google.com/intl/en/pack_installer_new.html?hl=en&gl=us&utm_source=en_US-et-more&utm_medium=et&utm_campaign=en_US&ciNum=11    - select to only download and install spyware doctor.

online anti virus scanners

http://www.pandasoftware.com   - panda activescan
http://www.bitdefender.com 
http://housecall.trendmicro.com
0
ryansotoCommented:
Definately check the processes running.
I know a stuck print job will kick the spool service up to 99%
0
briancassinCommented:
Also post the logfile from process explorer...
0
Cloud Class® Course: MCSA MCSE Windows Server 2012

This course teaches how to install and configure Windows Server 2012 R2.  It is the first step on your path to becoming a Microsoft Certified Solutions Expert (MCSE).

planetemAuthor Commented:
Ok my trendmicro's almost done with the scan but still nothing came up

i've installed process explorer, how do i go around to retrieve the log of that?

Let me pull also the hijackthis (as i said i had already done that part but didnt see anything odd)
0
planetemAuthor Commented:
system is what i took out of system process but it wasnt at 100% when i did
hijackthis.log
System.txt
0
briancassinCommented:
ok I see a couple of problems here

C:\Documents and Settings\Steve\WINDOWS\System32\smss.exe

O10 - Broken Internet access because of LSP provider 'c:\documents and settings\steve\windows\system32\mswsock.dll' missing   - DO NOT USER HIJACK THIS TO FIX THIS PROBLEM!


smss.exe should not be running from documents and settings KILL THIS!

download and run the LSP fix

http://www.cexx.org/lspfix.htm




do you know what this website is ? if not this needs to be fixed
O16 - DPF: {05D96F71-87C6-11D3-9BE4-00902742D6E0} (QuickPlace Class) - https://quickplace01.geextranet.com/qp2.cab

is this something you installed ?

E:\CELLWIN\PM\BIN\OLAP Service\OlapService.exe
O23 - Service: CellSell32 OLAP Service (CellSellOLAPService) - Unknown owner - E:\CELLWIN\PM\BIN\OLAP Service\OlapService.exe
0
planetemAuthor Commented:
yeah the cellwin are stuff we are using
the other site looks like belonging to GE Security, dunno why this server would need access to it though

0
planetemAuthor Commented:
ouch after trying the lspfix now the machine lost internet access and i cant connect to it remotely
0
briancassinCommented:
did you run the LSP fix remotely ?
0
briancassinCommented:
try using the microsoft method instead

http://support.microsoft.com/kb/317518
0
planetemAuthor Commented:
nope no good, i'll try to repair windows 2003 at this point
0
planetemAuthor Commented:
reinstall service pack 2 didnt fix anything.
grabbing back mswsock.dll from the cd seemed to fix the error message but i still cant get any internet connection, i cant even get an ip from dhcp (normally set to static)
ipsec is also turned off for good measure

this is not good :/
0
briancassinCommented:
Try going into device manager and removing the NIC card then rebooting and let it get redetected. Then go through the winsock reset procedure.
0
planetemAuthor Commented:
That didnt work, it's not even getting an ip from the dhcp. i put back the manual ip address in there

seems i can ping it from other machine, i can view the file on it
and now i can remote desktop to it
but i still cant get it to see the internet, i tried to rebuild the tcp/ip stack with netsh int ip reset log.txt
to no avail
0
briancassinCommented:
what does the log file say can you post it ?
0
planetemAuthor Commented:
here's the application and system log
application.txt
0
briancassinCommented:
Actually I meant the log file for netsh reset log file
0
planetemAuthor Commented:
oh here it is
log2.txt
0
briancassinCommented:
Have you tried going into the properties of the connection and removing the TCP/ IP protocol rebooting and then reinstalling it ? if not try this then
at the command prompt run this command: netdiag /test:winsock /v
and tell me what the results are



0
planetemAuthor Commented:
Another thing now i can remoted desktop externally to that machine.
other things i noticed an ip config shows me the ip and subnet but gateway is blank
even tho it's setup

its almost like a dns issue but more weird :S
0
planetemAuthor Commented:
netdiag not a valid command
0
planetemAuthor Commented:
and i cant uninstall tcp also, tried that but the tcp/ip gets shaded

so many things im not used to apparently under 2003 :/
0
briancassinCommented:
if it is shaded out then there is most likely still a problem with the winsock ...

You have to load the support tools from the windows server 2003 CD to have netdiag if they are not loaded already.

0
briancassinCommented:
I want to check something else too

goto the start menu and then run and type on the white line cmd and then hit the enter key

from the dos prompt try this so i can see the variable settings.

set >var.txt

on the next line type:

var.txt

Notepad should open with a logfile. Paste that here.
0
planetemAuthor Commented:
installed, rebooting server for the x th time today, cant believe im still at work
0
planetemAuthor Commented:
ALLUSERSPROFILE=C:\Documents and Settings\All Users
APPDATA=C:\Documents and Settings\Steve\Application Data
CLASSPATH=C:\PVSW\BIN\psql.jar;C:\PVSW\Tango2000\BeanHandler.jar;C:\PVSW\Tango2000\java
CLIENTNAME=NETADMIN
ClusterLog=C:\WINDOWS\Cluster\cluster.log
CommonProgramFiles=C:\Program Files\Common Files
COMPUTERNAME=SERVER1
ComSpec=C:\WINDOWS\system32\cmd.exe
FP_NO_HOST_CHECK=NO
HOMEDRIVE=C:
HOMEPATH=\Documents and Settings\Steve
LOGONSERVER=\\SERVER1
NUMBER_OF_PROCESSORS=2
OS=Windows_NT
Path=C:\Program Files\Support Tools\;C:\PVSW\BIN;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\JavaSoft\JRE\1.2\bin;C:\Program Files\JavaSoft\JRE\1.2\bin\classic;C:\PVSW\Tango2000
PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH
PERVASIVE_PATH=C:\PVSW\BIN
PROCESSOR_ARCHITECTURE=x86
PROCESSOR_IDENTIFIER=x86 Family 15 Model 4 Stepping 3, GenuineIntel
PROCESSOR_LEVEL=15
PROCESSOR_REVISION=0403
ProgramFiles=C:\Program Files
PROMPT=$P$G
SESSIONNAME=RDP-Tcp#1
SystemDrive=C:
SystemRoot=C:\WINDOWS
TEMP=C:\DOCUME~1\Steve\LOCALS~1\Temp\1
TMP=C:\DOCUME~1\Steve\LOCALS~1\Temp\1
USERDOMAIN=SERVER1
USERNAME=Steve
USERPROFILE=C:\Documents and Settings\Steve
VSL=C:\PVSW\BIN
windir=C:\WINDOWS
0
planetemAuthor Commented:
here's the output of netdiag


    Gathering IPX configuration information.
    Querying status of the Netcard drivers... Passed
    Testing Domain membership... Passed
    Gathering NetBT configuration information.
    Gathering Winsock information.

    Tests complete.


    Computer Name: SERVER1
    DNS Host Name: server1
    DNS Domain Name: (null)
    System info : Windows 2000 Server (Build 3790)
    Processor : x86 Family 15 Model 4 Stepping 3, GenuineIntel
    Hotfixes :
        Installed?      Name
           Yes          KB911564
           Yes          KB921503
           Yes          KB924667-v2
           Yes          KB925398_WMP64
           Yes          KB925902
           Yes          KB926122
           Yes          KB927891
           Yes          KB929123
           Yes          KB930178
           Yes          KB931784
           Yes          KB932168
           Yes          KB933729
           Yes          KB933854
           Yes          KB935839
           Yes          KB935840
           Yes          KB936021
           Yes          KB936357
           Yes          KB936782
           Yes          KB938127
           Yes          KB938680
           Yes          KB941202
           Yes          KB941568
           Yes          KB941569
           Yes          KB941644
           Yes          KB942615
           Yes          KB942763
           Yes          KB942840
           Yes          KB943460
           Yes          KB943485
           Yes          KB944653
           Yes          Q147222
           No           ServicePackUninstall


Netcard queries test . . . . . . . : Passed

    Information of Netcard drivers:

    ---------------------------------------------------------------------------
    Description: Intel(R) PRO/1000 MT Dual Port Network Connection
    Device: \DEVICE\{C0B12DCA-BDCB-49E0-9227-4FC6C705BFB4}

    Media State:                     Connected

    Device State:                    Connected
    Connect Time:                    00:04:16
    Media Speed:                     100 Mbps

    Packets Sent:                    1185
    Bytes Sent (Optional):           0

    Packets Received:                2006
    Directed Pkts Recd (Optional):   1316
    Bytes Received (Optional):       0
    Directed Bytes Recd (Optional):  0

    ---------------------------------------------------------------------------
    [PASS] - At least one netcard is in the 'Connected' state.



Per interface results:

    Adapter : Local Area Connection 3
        Adapter ID . . . . . . . . : {C0B12DCA-BDCB-49E0-9227-4FC6C705BFB4}

        Netcard queries test . . . : Passed


Global results:


Domain membership test . . . . . . : Passed
    Machine is a . . . . . . . . . : Standalone Server
    Netbios Workgroup name . . . . : WORKGROUP
    Dns domain name is not specified.
    Dns forest name is not specified.
    Domain Guid. . . . . . . . . . : {00000000-0000-0000-0000-000000000000}
    Logon User . . . . . . . . . . : Steve
    Logon Domain . . . . . . . . . : SERVER1


NetBT transports test. . . . . . . : Passed
    List of NetBt transports currently configured:
        NetBT_Tcpip_{C0B12DCA-BDCB-49E0-9227-4FC6C705BFB4}
    1 NetBt transport currently configured.


Winsock test . . . . . . . . . . . : Failed
    Failed to get UDP packet size information. The error occurred was: An address incompatible with the requested protocol was used.

 
    The number of protocols which have been reported : 8
        Description: MSAFD NetBIOS [\Device\NetBT_Tcpip_{403A411D-F27D-4FD6-AB23-2152AE88B967}] SEQPACKET 3
            Provider Version   :2
        Description: MSAFD NetBIOS [\Device\NetBT_Tcpip_{403A411D-F27D-4FD6-AB23-2152AE88B967}] DATAGRAM 3
            Provider Version   :2
        Description: MSAFD NetBIOS [\Device\NetBT_Tcpip_{C0B12DCA-BDCB-49E0-9227-4FC6C705BFB4}] SEQPACKET 0
            Provider Version   :2
        Description: MSAFD NetBIOS [\Device\NetBT_Tcpip_{C0B12DCA-BDCB-49E0-9227-4FC6C705BFB4}] DATAGRAM 0
            Provider Version   :2
        Description: MSAFD NetBIOS [\Device\NetBT_Tcpip_{A30D6AAD-ADDB-4B7B-A0E9-A4E09320F216}] SEQPACKET 1
            Provider Version   :2
        Description: MSAFD NetBIOS [\Device\NetBT_Tcpip_{A30D6AAD-ADDB-4B7B-A0E9-A4E09320F216}] DATAGRAM 1
            Provider Version   :2
        Description: MSAFD NetBIOS [\Device\NetBT_Tcpip_{17D6F603-363E-45B1-B4C8-88C5FDD1E700}] SEQPACKET 2
            Provider Version   :2
        Description: MSAFD NetBIOS [\Device\NetBT_Tcpip_{17D6F603-363E-45B1-B4C8-88C5FDD1E700}] DATAGRAM 2
            Provider Version   :2


The command completed successfully
0
briancassinCommented:
Go here and download the latest versin of this and post the logfile it will tell me what your winsock says among other things
http://www.silentrunners.org/
0
planetemAuthor Commented:
0
briancassinCommented:
The winsock is definately missing things...

Try this here ... manually remove TCP / IP and then reinstall following this here
Also try the winsock reset command again.

http://support.microsoft.com/kb/325356
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
planetemAuthor Commented:
Thanks a lot, even if the original problem was something else (trendmicro new update fucked it up) you really pulled through to help with the rest!
0
briancassinCommented:
Did that fix it ???  is it working now ???
0
planetemAuthor Commented:
The winsock reset from the ms link worked, i wasnt sure since that machine was not a DC but still did the trick.

As for why the system was hogging i think it ended up also being trendmicro client that caused the issue with the last patch. As after i removed it everything was really fast.

Thanks for all the help
0
briancassinCommented:
no problem glad I ws able to help you
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Operating Systems

From novice to tech pro — start learning today.