Troubleshoot suspicious traffic from workstation to AD over 1025
I'm at my wits end on this one & am hoping the EE community could lend a helping hand!
We started receiving intrusion prevention alerts on our SonicWall E5500 NSA after a recent firmware update. Of ~10 workstations, only two were triggering the alert: "IPS Prevention Alert: P2P eMule -- Obfuscated Protocol, SID: 4, Priority: Low" Source: workstation, random port; Dest: either of our DCs, port 1025
I took one of the machines offline & built a new machine for the user using a custom winXP image on different hardware. We kept the default machine name & have not had any alerts from the device, so the user is up and running for the time being.
The odd part here is that I wiped the user's old machine, reimage using the same image as above & flashed the BIOS. I removed the object from AD, renamed the machine to its original, and joined the domain. The alerts started again. The only commonality here (that i can think of ) is the machine name. I haven't even attempted to work on the second machine in question. (Malware & rootkit scans have come up clean)
I have since reimaged the machine from a base XP image, with no bells and/or whistles. Installed/updated A/V, ran all appropriate Win Updates, etc. The alerts started again as soon as I joined the domain - inbound to the AD server(s) over 1025.
Any thoughts/help would be appreciated. Our ultimate goal is to eradicate this & apply the same fix to the second device.
We started receiving intrusion prevention alerts on our SonicWall E5500 NSA after a recent firmware update. Of ~10 workstations, only two were triggering the alert: "IPS Prevention Alert: P2P eMule -- Obfuscated Protocol, SID: 4, Priority: Low" Source: workstation, random port; Dest: either of our DCs, port 1025
I took one of the machines offline & built a new machine for the user using a custom winXP image on different hardware. We kept the default machine name & have not had any alerts from the device, so the user is up and running for the time being.
The odd part here is that I wiped the user's old machine, reimage using the same image as above & flashed the BIOS. I removed the object from AD, renamed the machine to its original, and joined the domain. The alerts started again. The only commonality here (that i can think of ) is the machine name. I haven't even attempted to work on the second machine in question. (Malware & rootkit scans have come up clean)
I have since reimaged the machine from a base XP image, with no bells and/or whistles. Installed/updated A/V, ran all appropriate Win Updates, etc. The alerts started again as soon as I joined the domain - inbound to the AD server(s) over 1025.
Any thoughts/help would be appreciated. Our ultimate goal is to eradicate this & apply the same fix to the second device.
Not familiar with nSonicwalls, but Ill take a guess... Maybe the NIC is failing and sending out corrupted packets?
ASKER
Thanks for responding, John. The packets are definitley coming from the two workstations. If we rename the workstation (ie: from WS1 to WS_1), the packets stop - almost as if they are sending the packets after receiving an instruction from elsewhere; however, we have been unable to find anything that would indicate that. If I leave both WS1 and WS_1 in DNS w/the same IP, the alerts continue, so it appears that the 'ghost' instructions may be resolving via DNS...
Any chance they are trying to force browser elections?
Description of the Microsoft Computer Browser Service
http://support.microsoft.com/kb/188001
ASKER
John -
What brings you to this train of thought? Why would port 1025 be involved?
Looking at my fw logs, there may be some weight to you hypothesis - but I'd like to try to hone in on your thoughts. We've been chasing so many possibilities, I don't want to start barking up yet another tree w/out just cause. :)
Feed me some more!
What brings you to this train of thought? Why would port 1025 be involved?
Looking at my fw logs, there may be some weight to you hypothesis - but I'd like to try to hone in on your thoughts. We've been chasing so many possibilities, I don't want to start barking up yet another tree w/out just cause. :)
Feed me some more!
I think I was just throwing it out there, more than anything..... It really boggles me to not have a clkue as to why it having a specific name does this......
Have you isolated what proces is making the connection/. Lets go a different approach....
TCPView for Windows
http://technet.microsoft.c
This is a little less of a stab in the dark......
Have you isolated what proces is making the connection/. Lets go a different approach....
TCPView for Windows
http://technet.microsoft.c
This is a little less of a stab in the dark......
ASKER
John -
Thanks for responding. Sorry it took so long to get back to you.
I'm afraid I don't have the machine readily available, so I can't run TCPView on it at the moment. From what I remember, it was svchost that was making the connection to port 1025. I looked in the system event log & saw an error regarding the master browser (although I don't recall the exact error right now), which is why I thought you might be on the right track.
As soon as I have the machine in my hands, I'll run TCPView and see what we find
Thanks for responding. Sorry it took so long to get back to you.
I'm afraid I don't have the machine readily available, so I can't run TCPView on it at the moment. From what I remember, it was svchost that was making the connection to port 1025. I looked in the system event log & saw an error regarding the master browser (although I don't recall the exact error right now), which is why I thought you might be on the right track.
As soon as I have the machine in my hands, I'll run TCPView and see what we find
Ok. If it is a SVCHost, we need to get more specific.
Process Explorer
http://live.sysinternals.c
Double click the SVCHost.exe in question, and list the services it is hosting....
Process Explorer
http://live.sysinternals.c
Double click the SVCHost.exe in question, and list the services it is hosting....
ASKER
John -
Sorry about the delay in getting back to you. I got in touch with Microsoft's security team and have had them poking around in our logs, packet captures, etc. for the past few days. I just got word back of some suspicious activity on the following ports:
1335 tcp/udp
digital-notary
Digital Notary Protocol
1336 tcp/udp
ischat
Instant Service Chat
(ischat.exe??)
1337 tcp/udp
menandmice-dns
menandmice DNS
Know anything about these?
Sorry about the delay in getting back to you. I got in touch with Microsoft's security team and have had them poking around in our logs, packet captures, etc. for the past few days. I just got word back of some suspicious activity on the following ports:
1335 tcp/udp
digital-notary
Digital Notary Protocol
1336 tcp/udp
ischat
Instant Service Chat
(ischat.exe??)
1337 tcp/udp
menandmice-dns
menandmice DNS
Know anything about these?
Unfortunately no......
ASKER
John -
I'm going to keep the thread open until I get a resolution from Microsoft (or other source) so we can add this to the paq. If your input is in any way related to the resolution, I'll be sure to award point accordingly.
If you can think of anything else in the meantime, please feel free to add to the thread.
Thx
I'm going to keep the thread open until I get a resolution from Microsoft (or other source) so we can add this to the paq. If your input is in any way related to the resolution, I'll be sure to award point accordingly.
If you can think of anything else in the meantime, please feel free to add to the thread.
Thx
ASKER
Just to keep the thread open - Microsoft PSS is taking a closer look at the systems via their forensics tool, WOLF. Meanwhile, SonicWall Support is taking a closer look at packet captures across the firewall. I'll update as we get closer to a resolution... hopefully soon.
I found this tread because I was getting the same log entries for one particular computer.
The only thing in the log for that computer were multiple attempts to contact 'ardownload.adobe.com', but 0 bytes downloaded in every case.
This occurred Friday afternoon 7-31 from noon till 4:00 PM CDT. I ended up with over 250 warnings logged for this particular computer.
From log:
07/31/2009 15:56:14.704 - Alert - Intrusion Prevention - IPS Prevention Alert: P2P eMule -- Obfuscated Protocol, SID: 4, Priority: Low - 192.168.2.2, 3479, X0 - 208.19.38.73, 80, X1 -
This computer in question is Windows 2000 and has Adobe 9.1 installed. There is apparently a more recent update from Adobe that we are installing now.
Is it possible that the Adobe downloader looks like eMule to the Sonicwall?
The only thing in the log for that computer were multiple attempts to contact 'ardownload.adobe.com', but 0 bytes downloaded in every case.
This occurred Friday afternoon 7-31 from noon till 4:00 PM CDT. I ended up with over 250 warnings logged for this particular computer.
From log:
07/31/2009 15:56:14.704 - Alert - Intrusion Prevention - IPS Prevention Alert: P2P eMule -- Obfuscated Protocol, SID: 4, Priority: Low - 192.168.2.2, 3479, X0 - 208.19.38.73, 80, X1 -
This computer in question is Windows 2000 and has Adobe 9.1 installed. There is apparently a more recent update from Adobe that we are installing now.
Is it possible that the Adobe downloader looks like eMule to the Sonicwall?
Sorry I didn't specify Adobe READER above. I'm sure you knew what I meant!
Additional info:
We tried running the update from Adobe Reader and are getting the same eMule errors from the Sonicwall and the Adobe updater is reporting that it can't connect.
Will now uninstall the current version and download the latest version of Reader direct from Adobes website.
Additional info:
We tried running the update from Adobe Reader and are getting the same eMule errors from the Sonicwall and the Adobe updater is reporting that it can't connect.
Will now uninstall the current version and download the latest version of Reader direct from Adobes website.
Well, downloaded the "latest" version of Adobe Reader from Adobe's website. Unfortunately, the latest version that is on the download page is the version we already had, 9.1.0. Updates would still not work, even after a reinstall.
But I manually download the 9.1.1, 9.1.2, and 9.1.3 update files from the Adobe Reader Web page. Installed those (this does not use the updater feature) and then, after getting to version 9.1.3 we can run the Adobe Updater successfully.
So, to summarize.
In our case, it seems that the Adobe Updater that is installed with Adobe Reader 9.1.0 is the culprit. It must look like eMule to the Sonicwall. Manually downloading the version 9.1.3 release and installing it corrected the issue.
But I manually download the 9.1.1, 9.1.2, and 9.1.3 update files from the Adobe Reader Web page. Installed those (this does not use the updater feature) and then, after getting to version 9.1.3 we can run the Adobe Updater successfully.
So, to summarize.
In our case, it seems that the Adobe Updater that is installed with Adobe Reader 9.1.0 is the culprit. It must look like eMule to the Sonicwall. Manually downloading the version 9.1.3 release and installing it corrected the issue.
ASKER
We finally made some headway here...
After working with Microsoft PSS for over a month & not finding anything on the affected machines, we turned back to SonicWall for assistance. After providing a series of packet captures from affected machines, non-affected machines, the AD servers & the firewall, the SW R&D team found something in the firmware which they believe to be the cause. We will be applying a patch on 9/12 to test their hypothesis, as they were unable to recreate the issue internally.
Unfortunately, JimInKS resolution did not apply to us, as we were seeing detections from machines that were not running any Adobe products.
I will gladly update as soon as we have some results from the test.
After working with Microsoft PSS for over a month & not finding anything on the affected machines, we turned back to SonicWall for assistance. After providing a series of packet captures from affected machines, non-affected machines, the AD servers & the firewall, the SW R&D team found something in the firmware which they believe to be the cause. We will be applying a patch on 9/12 to test their hypothesis, as they were unable to recreate the issue internally.
Unfortunately, JimInKS resolution did not apply to us, as we were seeing detections from machines that were not running any Adobe products.
I will gladly update as soon as we have some results from the test.
terradmin,
Thanks for getting to the bottom of this. I am seeing a few, "P2P eMule -- Obfuscated Protocol" alerts showing up on my logs for what appear to be "random" reasons. I hope Sonicwall finds a solution.
Thanks for getting to the bottom of this. I am seeing a few, "P2P eMule -- Obfuscated Protocol" alerts showing up on my logs for what appear to be "random" reasons. I hope Sonicwall finds a solution.
ASKER
@JimInKS - in all the alerts we've seen, the destination port was 1025. In your earlier post, your log indicates port 80 - is this the case on all fw entries?
A few notes for you:
1. We found we were able to force the IPS Prevention Alert by attempting a 'gpupdate /force' from an affected machine
2. We removed an affected machine name from AD and DNS & disconnected it from the network. After renaming a non-affected machine to the affected machine's name, alerts began again. Alerts stopped after renaming the 2nd machine back to its original name.
The alerts began on our network after upgrading the firmware to SonicOS Enhanced 5.4.0.0 (feature upgrade). We rolled back to 5.2.0.3 w/SW's help (not a public release - I believe the latest is 5.2.0.1), although this did not resolve the issue.
A few notes for you:
1. We found we were able to force the IPS Prevention Alert by attempting a 'gpupdate /force' from an affected machine
2. We removed an affected machine name from AD and DNS & disconnected it from the network. After renaming a non-affected machine to the affected machine's name, alerts began again. Alerts stopped after renaming the 2nd machine back to its original name.
The alerts began on our network after upgrading the firmware to SonicOS Enhanced 5.4.0.0 (feature upgrade). We rolled back to 5.2.0.3 w/SW's help (not a public release - I believe the latest is 5.2.0.1), although this did not resolve the issue.
All the alerts I have looked at have been port 80.
I have been getting around 10 a day, but we are small, around 100 pc's with internet access.
We are running a NSA2400 firmware 5.4.0.0-20o which we installed recently (middle of July).
Did not see these on our old PRO 2040 SonicOS Enhanced 4.0.0.5-1e.
Oddly destination ip's (recently) are all 199.7.X.190, where the 3rd octet can be any of 48, 51, 52, 71
I have been getting around 10 a day, but we are small, around 100 pc's with internet access.
We are running a NSA2400 firmware 5.4.0.0-20o which we installed recently (middle of July).
Did not see these on our old PRO 2040 SonicOS Enhanced 4.0.0.5-1e.
Oddly destination ip's (recently) are all 199.7.X.190, where the 3rd octet can be any of 48, 51, 52, 71
ASKER
Those IPs belong to CRL.VERISIGN.NET, so I'd venture a guess that you're seeing false detections. Still only happening from the one machine?
Right now it is about 7 machines on a kind of sporadic basis.
I haven't done a lot of investigation. After I got slammed by that 1 machine and seemingly fixed it with the Adobe update I haven't seen a huge number of these alerts.
Since it seems to be a Sonicwall issue I wasn't overly concerned and was hoping someone with more expertise than I would find a definitive solution or Sonicwall would issue a firmware update that solved this issue.
I am a bit of novice at this. There are just 2 of us in IT. However, if an answer is not forthcoming I would be willing to do what I can to help find a solution.
I haven't done a lot of investigation. After I got slammed by that 1 machine and seemingly fixed it with the Adobe update I haven't seen a huge number of these alerts.
Since it seems to be a Sonicwall issue I wasn't overly concerned and was hoping someone with more expertise than I would find a definitive solution or Sonicwall would issue a firmware update that solved this issue.
I am a bit of novice at this. There are just 2 of us in IT. However, if an answer is not forthcoming I would be willing to do what I can to help find a solution.
ASKER
@JimInKS
We'll be applying SW's patch tomorrow morning. I'll post the results once I'm sure I haven't killed my network. :)
If all goes well, my suggestion would be that you reach out to Michael Hopper at SonicWall - he was handling my case (support [at] sonicwall [dot] com) & simply reference eMule in the subject.
Looking forward to closing this question once and for all....
:)
We'll be applying SW's patch tomorrow morning. I'll post the results once I'm sure I haven't killed my network. :)
If all goes well, my suggestion would be that you reach out to Michael Hopper at SonicWall - he was handling my case (support [at] sonicwall [dot] com) & simply reference eMule in the subject.
Looking forward to closing this question once and for all....
:)
ASKER
We upgraded to SonicOS Enhanced 5.5.0.0-m1_chestnut over the weekend & have not had any alerts for approximately 36 hours. It looks like the firmware update resolved the issue.
Good news. Thanks for all your work on this.
ASKER
@Moderator - please close this question & refund points as the solution was provided externally.
I'd like to ask that this be added to the PAQ if possible, in case similar issues crop up for other users.
I'd like to ask that this be added to the PAQ if possible, in case similar issues crop up for other users.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Anyone??