Link to home
Start Free TrialLog in
Avatar of sdlangers
sdlangers

asked on

Please explain the results of running netstat - do i have a security problem?

Hi,

when i run netstat -a on my win2003 server i get a lot of results similar to the following sample (i've XXXd out the real names)

 TCP    MYSERVER:ms-sql-s     ns.XXX.co.uk:5432  TIME_WAIT
 TCP    MYSERVER:ms-sql-s     ns.XXX.co.uk:5968  TIME_WAIT
 TCP    MYSERVER:ms-sql-s     ns.XXX.co.uk:5994  TIME_WAIT

where MYSERVER is my server machine name and ns.XXX.co.uk is an address that ive no idea why it would be showing up here. again, its not really XXX.co.uk - i just didnt want to type in the real name here.  there about 150 of these types of entries

i have the server monitored daily and im pretty sure the ports are locked down, but im still concerned that this might be a potential security problem - please explain or let me know your thoughts - do i need to do anything? if so, what?

thank you!
Avatar of davy999
davy999
Flag of Denmark image

Avatar of sdlangers
sdlangers

ASKER

thanks - i already had the netstat command syntax -  but i was hoping more for advice on whether those results i posted could be a potential security problem.  and if it means that that ip is constantly trying to hit the sql server or something along those lines?
Avatar of Rob Williams
Very curious. It is morelikely an outgoing connection. I don't have a lot to add, but you could do a netstat -n , or an nslookup ns.xxx.co.uk and locate the IP for ns.xxx.co.uk, and then do an IP lookup at www.dnsstuff.com to possibly locate who it is, which would help to figure out why the connection.
Also what version of SQL are you running ?  The first port number you referred to 5432 is listed as a "PostgreSQL Database" service. Could it be checking for updates software updates or similar.
No chance ns.xxx is a name server, and it is doing DNS lookups, though the port numbers should be different.
Just food for thought.
sdlangers, were you able to further determine the source of the connections?
--Rob
Sorry, i meant to reply and forgot until i got the reminder email.

no - i havent been able to figure out whats going on

rob, i now get actual ip addresses when i run the netstat -a  (as opposed to ns.xxx.co.uk)

i checked the ip address in dnsstuff.com but it only tells me the location of the server and a bunch of other stuff that i'm not sure how its meant to help me.. please elaborate

but i still dont understand how that will help me

what im really looking for is someone to explain what exactly that line means in the output - i.e. surely its either an outgoing connection or an incoming one and i should be able to tell for sure and also what the actual output line means:

TCP    MYSERVER:ms-sql-s     (ip address):5994  TIME_WAIT

etc.

so, please let me know if you can shed any light on this

i basically just need a definite answer as to whether or not i may have a security problem.

thanks!
As I understand it your server ms-sql-s made an out going connection to  (ip address) using port 5994 and the connection has been closed, but remains in a "TIME_WAIT" state for a predetermined length of time (I believe 4 minutes by default) for any slow returning packets, before freeing up the port for it's next use. On any application server such as a web or database server there can be dozens or even hundreds of these at any given time, and they are of no security concern. They are just sessions in the process of shutting down.
--Rob
HI Rob,

thanks for the quick reply.

why would my sql server be making an outgoing connection?  the only access to the database is thru web applications that are hosted on the same server - so there shouldnt be any external connections (incoming or outgoing) to the sql server (with the exception of my computer since i have query analyzer etc)... or am i not understanding something?

thanks!
It is hard to say without finding out more about who is registered to  (ip address) . It could even be a request initiated by a web query if it is on the same server. Is the (ip address) always the same ? If so do a look up at www.dnsstuff.com and see if it "rings any bells" It could even be a built in update service for a driver or application.
i did the lookup as per the earlier post.  its registered to a place in florida - i've no connections whatsoever with anyone/business in florida.  i dont understand how it could be a web query - since the only apps that access it are hosted on my same server box.  this is why im concerened.  yes, the ip/dns is always the same one out of a few- any other thoughts/suggestions?  is it outgoing or incoming - how can i tell?

thanks
If the server were waiting for an incoming connection it usually shows LISTENING. If there are current communications it shows ESTABLISHED. Netstat shows all connections so it can be to a time server, Windows update service, Java Update, Virus update, printers such as Lexmark do this at least 4 times a day, and so on. The port number can be a clue sometimes. 5994 really doesn't tell you anything, but is it always that port? Many applications choose random ports in certain ranges. If other ports you may find a clue at:
http://www.iana.org/assignments/port-numbers

If you don;t want to post the IP here send it to my e-mail address (click on RobWill) and I can see what I can find out about it if anything. Ne the IP # and frequent ports used.
Hi Rob,

OK, so TIME_WAIT means the server is waiting on an outgoing connection?

no, its not always port 5994 and theres a whole bunch of them - about 150 different ports all with the same message - this is why im confused - id understand if it was just one or two then it could be waiting on a service such as the windows update or others that you mentioned... but 150?  and also - why do they all say 'ms-sql-s' - wouldnt this indicate something to do with sql server?

thanks for your ongoing ideas..
TIME_WAIT is a closed session that is waiting on any outstanding un-returned packets from a previous session. I believe one that was initially established as an outgoing connection. Many services choose a random series of ports for outgoing connections. For example session one might be on port 5591, two on 5592, three on 5993 and so on. If a session were suddenly closed, and thus freed up, another session or service may reuse that ports. A late returning packet from a previous session could be mixed with the current session and cause chaos. So, the port goes into a TIME_WAIT state for a set time (as mentioned I believe 4 minutes), effectively blocking the port from being re-used until it is certain that are no returning stragglers. There is no concern with anything in that state. The only concern you might have is what was the previous session.

The first part of the listing under Local address shows your server and port. I was thinking 'ms-sql-s' was your server name but ti would actually be the service. Try running
netstat  -a  -b
This will take longer to run but will help to isolate the application using the port. It may add some additional information, though I believe it gives more information with an open/ESTABLISHED session. See if there are any sessions showing as established with that same IP.
ok thanks - now im getting worried - i've found a about 5 entries like these:

  TCP   (my server name):ms-sql-s     (remote server ip):3305  ESTABLISHED     1820
  [sqlservr.exe]

doesnt this mean that they are connected to the sql server ?  is 3305 the port number they are connected on?

>>"doesn't this mean that they are connected to the sql server ? "
Yes, or more likely your SQL server is connected to them.
Next step would be to find out more information about the remote IP. If you need a hand with that let me know.

You could run Ethereal and analyze the packets being transmitted back and forth, but it is not an easy program to use and understand.
http://www.ethereal.com/
ok - i found out the 3 remote ips that are connected and none of them should be - one is in holland, one in iran and the other in jordan.

what can i do to kick them off ?  and how can i prevent others from connecting?

i dont understand how "more likely your SQL server is connected to them" - why would that be the case? and if it is, wouldnt that imply there was some type of program running on my server thats connecting them - like a trojan or something?

thanks for your help!
>>"some type of program running on my server thats connecting them - like a trojan or something?"
Very possible.
Is the server available to external users? If not and you have a properly configured firewall it is unlikely they are connecting to you. However, as you suggested it could be Malware initiating the outgoing connection, or as mentioned before a service of some sort looking for updates.

I would make sure all Windows patches are up to date,  run a full virus scan with all updates and then a spyware check. I find http://www.ewido.com is one of the most thorough spyware checkers.
Another great little tool is XTeq which will allow you to see any hidden services that are starting up automatically:
http://www.x-setup.net/downloads/
thanks - ok i think we're getting somewhere now - i downloaded the ewido and it didnt find anything (apart from 4 cookies) - but on the other tabs - analysis->processes - it showed a list similar to netstat-ab and i was able to end the connection to those established connections.  

but then i went back into it and it no longer shows any connections now to the sql server 1433 ???? even though im connected remotely.  i ran the netstat-ab and its still showing time_waits and others (like my connection) and other similar connections to those that i terminated.  can i do anything to block them all as a rule or somehting?  there are also ones that are FIN_WAIT_1 and FIN_WAIT_2 and LAST_ACK

i also noticed on the startup tab - an application called RootInstaller in Reg\HKLM\Run and it says the path is c:\rootinstallerNET.exe - but i checked in the c:\ drive and there is no such file - whats all this about?  i tried googling on it but couldnt find anything.  here is the startup report.  please tell me if anything looks wrong:

Reg\HKLM\Run         HotKeysCmds                              C:\WINDOWS\system32\hkcmd.exe                                                                      
Reg\HKLM\Run         IgfxTray                                 C:\WINDOWS\system32\igfxtray.exe                                                                    
Reg\HKLM\Run         RootInstaller                            C:\RootInstallerNET.exe                                                                            
Reg\HKLM\Run         Windows Defender                         "C:\Program Files\Windows Defender\MSASCui.exe" -hide                                              
Shell\CommonStartup  BGInfo.lnk                               C:\Documents and Settings\All Users\Start Menu\Programs\Startup\BGInfo.lnk                          
Shell\CommonStartup  Service Manager.lnk                      C:\Documents and Settings\All Users\Start Menu\Programs\Startup\Service Manager.lnk                

also, here is the full processes report - again let me know if you spot anything  (what are the ones that start with \??\c:\)

       0: System Process
       4: System Process
     288: c:\Program Files\Persits Software\AspEmail\BIN\EmailAgent.exe
     400: C:\WINDOWS\system32\svchost.exe
     432: C:\WINDOWS\System32\snmp.exe
     556: \??\C:\WINDOWS\system32\csrss.exe
     560: d:\Program Files\OpenSSH\usr\sbin\sshd.exe
     596: C:\Program Files\Microsoft SQL Server\MSSQL\Binn\sqlagent.EXE
     608: C:\WINDOWS\system32\serverappliance\srvcsurg.exe
     636: \SystemRoot\System32\smss.exe
     700: \??\C:\WINDOWS\system32\csrss.exe
     732: \??\C:\WINDOWS\system32\winlogon.exe
     752: C:\WINDOWS\system32\POP3Server\pop3svc.exe
     776: C:\WINDOWS\system32\services.exe
     788: C:\WINDOWS\system32\lsass.exe
     856: C:\WINDOWS\System32\svchost.exe
     988: C:\WINDOWS\system32\svchost.exe
    1024: C:\WINDOWS\system32\logon.scr
    1068: C:\Program Files\ewido anti-malware\SecuritySuite.exe
    1072: C:\WINDOWS\system32\svchost.exe
    1112: C:\Program Files\Windows Defender\MsMpEng.exe
    1168: C:\WINDOWS\system32\svchost.exe
    1232: C:\WINDOWS\system32\svchost.exe
    1264: C:\WINDOWS\System32\svchost.exe
    1408: \??\C:\WINDOWS\system32\winlogon.exe
    1480: C:\WINDOWS\system32\msdtc.exe
    1664: C:\WINDOWS\system32\serverappliance\appmgr.exe
    1684: C:\WINDOWS\system32\serverappliance\elementmgr.exe
    1732: C:\WINDOWS\System32\svchost.exe
    1792: C:\WINDOWS\system32\inetsrv\inetinfo.exe
    1816: C:\WINDOWS\System32\svchost.exe
    1820: C:\Program Files\Microsoft SQL Server\MSSQL\Binn\sqlservr.exe
    1908: d:\Program Files\OpenSSH\bin\cygrunsrv.exe
    2012: C:\WINDOWS\System32\svchost.exe
    2088: C:\WINDOWS\system32\wbem\wmiprvse.exe
    2328: C:\WINDOWS\Explorer.EXE
    2352: C:\Program Files\Windows Defender\MSASCui.exe
    2484: C:\WINDOWS\system32\mmc.exe
    2796: C:\Program Files\Microsoft SQL Server\80\Tools\Binn\sqlmangr.exe
    2864: C:\WINDOWS\system32\mmc.exe
    2988: C:\Program Files\ewido anti-malware\ewidoctrl.exe
    3392: C:\WINDOWS\system32\igfxtray.exe
    3396: C:\Program Files\ewido anti-malware\ewidoguard.exe
    3464: C:\WINDOWS\system32\cmd.exe
    3484: C:\WINDOWS\system32\rdpclip.exe
    3576: c:\windows\system32\inetsrv\w3wp.exe
    3724: C:\WINDOWS\regedit.exe
    4016: C:\WINDOWS\system32\netstat.exe

thanks for all your help - i hope we're getting to the root of it.  

do you think it would make any difference getting the full version of ewido?

also, server is not (or rather should not) be accessible to anyone else.  i have SFTP setup on it with only a couple of usernames and i know where they are and its not any of the countries that the ips are showin up from.

what anti-virus do you recommend for windows server 2003?  i have all the windows patches up to date and have windows defender installed

thanks!
That list looks pretty clean but this one worries me: C:\RootInstallerNET.exe, because of where it is in the registry and the name. I don't know much of anything about "root Kits" but they are one of the newest threats. They are also very hard to find so I am doubtful that is what this is but there is a free utility for discovering them located at:
http://www.sysinternals.com/Utilities/RootkitRevealer.html
If for no other reason than you have done a complete system scan, it would be worth running this as well.

As for virus protection, I like McAfee. Norton/Symantec has gone down hill the last couple of years in my opinion. Trendmicro and Panda are also quite popular. If you are running Small Business Server, Trend Micro has the only one I know that is specifically approved, and it is possible SQL has special requirements, I don't work with it.

What do you have for a firewall. You could set it up to block out going traffic on all ports but what you need for your applications.
Just realized I didn't answer all of above.
FIN_WAIT_1 and FIN_WAIT_2 and LAST_ACK Are all closed session states. When a session is ended, the connection goes through these states until CLOSED_WAIT and then disappears.

I don't think full version of Ewido is any more thorough, it just allows for active spyware monitoring. Personally I don't like to bog the system down with too many services. It has added 2 as it is (ewidoctrl.exe and ewidoguard.exe). You might even want to consider uninstalling once resolved.



Hi rob

thanks for your continued efforts.  the server is meant to have all outgoing traffic blocked, except on port 1433 - perhaps i should just block that port too and only open it when i need to connect thru sql server - although this is a pain having to do this every time i want to use the database.  however, that would not really be solving the root of the problem - i should be able to have this port open

i just contacted ewido support and its not fully supported on windows server 2003 - although it seemed to work, so perhaps it didnt pick up everything - so you recommend mcafee for the server - but which version/product?

also, one other thing from above - why does this process show up with the \??\ at the start?

   556: \??\C:\WINDOWS\system32\csrss.exe

i'll try the rootkitrevealer and let you know.  that one startup file definitely looks suspect - the fact it points to a file that i cant see.
Normally outgoing traffic is not blocked. Though it would be an added precaution, I agree fix the basic problem first.

I don't know why the \??\ . All the ones with it are standard Windows processes. Maybe one of the results of Server O/S not being supported.

Most of my customers that are using McAfee have the older version 7 or 8.0i
http://www.mcafee.com/us/smb/products/anti_virus/file_servers_desktops/virusscan_80i.html
However, I don't know if this will work with SQL. Databases have to be handled differently nd none of my clients, surprisingly, use SQL. (actually have to set up 1st one next week). You could contact one of their partners:
https://secure.nai.com/us/partners/channel/find/default.asp

See if the rootkit revealer shows anything. Sysinternals is a great site for excellent utilities. All can be trusted, and work extremely well. On top of that most are free. http://www.winternals.com/  has their "bigger" paid tools.
Root kits can be very difficult to find without the appropriate tools as they do not show up in a normal process list.
http://en.wikipedia.org/wiki/Root_kit
Hi Rob,

rootkitRevealer wont run - i get the error message - 'must be run from the console'  - i looked up this error and apparently you cant run it via a remote desktop connection? have you experienced this? do you know any workarounds?

thanks!
You could try logging into the console session with remote desktop. To do so  in the start/run box enter  
  mstsc  /console
It will start the remote desktop connection screen as usual, but it connects to the console session rather than 1 of the 2 terminal server sessions.  Good thing to know if ever you are locked out of server access because 2 sessions are tied up.
Might work.
thanks - that worked

i ran the rootkit revealer and it didnt find anything

im thinking at this stage just to re-image and start over - unless you have any other suggestions?
ASKER CERTIFIED SOLUTION
Avatar of Rob Williams
Rob Williams
Flag of Canada image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial