ISA 2006 windows cannot determine the user or computer name (the RPC server is unavailable.)

Open the ISA system policy rules and check that you have enabled the rpc from ISA to internal rule. (This is different to the firewall policy rule which is also needed.

Highlight the firewall policy in the left panel of the gui, a set of iconds will appear along the top of the window. The top-right icon is a toggle on/off to display/hide the System Policy.

ASKER

I have the system policy for RPC from ISA Server to trusted servers enabled to "Internal". The DC is in the internal network. The traffic on the way out (port 135)gets the error "0x8007274c WSAETIMEDOUT" on the way back in it comes in on a dynamic port with the error "0xc0040034 FWX_E_SEQ_ACK_MISMATCH". This setup must be common to have an ISA server a member of a domain that is internal. Any other thoughts?

Weird. All my ISA installs have ISA as a domain member and there is no special setup required.

I take it you have the internal LAT tables set correctly? Nothing stupid like a default gateway on both ISA interfaces?

ASKER

Good idea to check, but no gateway on internal, no dns on external and yes the subnet is in the LAT

satxusa

drbbton:

Make sure that in the System Policy under Active Directory on the To tab you have Internal selected. Also, make sure that the "Enable this configuration group" box is checked. You may want to try unchecking "Enforce strict RPC compliance." Unchecking that should not be necessary, but sometimes it solves things.

Also, check your Firewall rules to make sure none of them block communications from Localhost to Internal. In addition, you must allow PING to Localhost from Internal and vice versa.

Look at your TCP/IP settings on BOTH interfaces on the ISA box. Make sure that the DNS server entries on BOTH interfaces point to your internal DNS server. Configure the internal DNS server for recursive lookups by removing any IP addresses from the forwarders list on the "All other domains" item. Make sure the Root Name Servers list is populated on the DNS server also.

Finally, check your Network Rules. A mismatched or missing network route/NAT rule can easily cause communications headaches.

If all else fails, temporarily create an Allow All Outbound Traffic rule between the Localhost and Internal and vice versa. Make it the first rule in the list. If you have a rule that is causing necessary traffic to be blocked, this will allow you to see that.

ASKER

satxusa:

I ironically, everything in this list has been tried. I eventually opened up all ports to and from. After doing so i could ping both ways, but when I attempted to establish a port 135 there were return trip dynamic ports that were blocked. I think i resolved them, but still the computer will not authenticate.

satxusa

Try disjoining and rejoining the domain. It's possible that the box may have a partially trusted connection. Just grasping for straws on that one, but anything is worth a try, right?

Also, it may be a misinstall. Export your firewall configuration (backup) and reinstall ISA, then import (restore) them back into the configuration storage server for the array.

I probably won't be checking this for a couple days -- gonna be out fo the office. Feel free to email me directly at AaronPage@sadc.biz and I'll reply. I get emails on my handheld.

As someone else has taken on the case, I'll move on to other questions.

Keith

Hi,

You don't mention what platform you have installed ISA on, but I believe this is a problem with Windows 2003 Server SP2 and ISA. The fix is to add the following to the registry, and reboot:

Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters]
"EnableRSS"=dword:00000000

I have had this issue with five or six ISAs now, and this has fixed all of them. Just for reference, the relevant KB article is here:

http://support.microsoft.com/kb/927695

I'm guessing you've also had loads of WSAETIMEDOUT errors on RPC traffic entries in the ISA logs?

Hope this helps.

Simon.

ASKER

Simon-

I also have 7 ISA servers that this is have this same issue. They all do have win 2003 with SP2. I changed the registry key that you specified above and i get the same issue. For good measure, i reinstalled ISA Server and still got the same error. I am getting a great deal of WSAETIMEDOUT errors inn the ISA logs... So i am not exactly sure were to go. I am thinking of just uninstalling Service Pack 2.

Blake

Hmm, very odd.

I had never seen this error before installing SP2, and I've been working with ISA since 2000, so if that didn't fix it I'm not sure what else to say.

I did also have one non-ISA server which was running on Dell hardware fail post-SP2, and that seemed to be down to the extra NIC that had been installed, (it was an Intel card if I remember correctly), I removed the NIC, and the server recovered. Is your server running on Dell hardware? (all of the other servers have been HP, so I've only seen this the one time)

Just for the record, un-installing SP2 didn't work for me, although I know from the message boards that it did for some people. I had to completely reinstall 2k3 server from scratch the first time... :o(

Sorry I can't be of more assistance,

Simon.

ASKER

I removed sp2 and it works perfectly.. No Errors. As some forums suggested, i tried to install sp2 after installing ISA 2006, and that did not work. ALL the hardware is Dell PE 2950 w broadcom internal & two dual intel cards in each server, so pretty much the same setup as you have. If anyone has any more information on what specifically can be done to have SP2 installed & make ISA work, that would be helpful. Otherwise, windows 2003 R2 STD w SP1 is the best OS to run ISA 2006 on.

Yes, that sounds almost identical to the hardware I saw fail. My feeling would be that it is something specific to the Dell hardware that is causing the failure. Since SP2 came out I've installed a pair of ISA 2006 Ent boxes and five other separate ISA 2006 Std servers on HP hardware now, all running Windows 2003 R2 Std w/ SP2, and all of them are fine.

If it was me I would try taking the Dual Intel card(s) out, re-install SP2, and see how that goes. (Or you might be fed up with all this un-installing, and re-installing by now... I know it did!)

Either way I would be interested to know what you find,

Simon.

ASKER

Simon-
I think i will try removing the Intel cards and see what happens, then try to disable the broadcoms and see what happens. It has to be one of the two cards. I will let you know.
Blake

scoraymo

I have same issue with a Dell server w2k3 with isa 2006.. did you ever find a solution to this problem?

Scottt

ASKER

I ended up uninstalling SP2 and it worked perfect. When i get a chance i am going to test removing the intel cards and just use the broadcom to see if that works. I will post my findings when i get them. Do you have intel nic's as well?

scoraymo

No, I have the integrated broadcom BCM5708C. If i turn off the ISA Microsoft firewall it fixes the RPC issue but the server becomes unusable. There us a rule to allow RPC but it doesn't seem to matter. RPC fails when using telnet or portqry while the firewall is running. Turning off strict RPC doesn't work.

I guess i'll see about removing SP2.

thanks

ASKER

ok, so its not an intel nic problem then. Its something called RSS that is causing the problem in SP2. SP2 Enables rss by default. You could try do disable rss in the registry. I have tried this to no avail. Have you contacted dell or Microsoft about this? I would hope that someone is working on a fix.

scoraymo

I have not contacted Dell or Microsoft... I agree I would hope so too..

This still running?

Try the ISA best practice analyser
http://www.microsoft.com/downloads/details.aspx?FamilyId=D22EC2B9-4CD3-4BB6-91EC-0829E5F84063&displaylang=en

perry57123

I have run into the same problem with a brand new box. took me a week but i have solved my issue.

Our domain controllers are built with Windows Server 2003 Standard edition.
The new ISA 2006 box has Windows Server 2003 R2 Standard with ISA 2006

On a clean install from scratch - once the firewall is operational - the Active Directory Requests die.

Conclusion.

Installed a fresh copy of 2003 Standard with ISA 2006 - Once the firewall is enabled - Authitication requests are functional. There is some issue with the way packets/sessions are returned from the Domain controller and how 2003 R2 interprets them.

ASKER

Perry57123:

What service pack are you running on your isa server boxes?

ASKER

ok, found the solution:

http://support.microsoft.com/default.aspx?scid=kb;EN-US;936594

Steps:
1. Update to Windows 2003 Service Pack 2
2. Install hotfix listed on website
3. Modify the registry to disable Receive Side Scaling
4.Disable offloading support

I did all of these steps in this order and the problem was resolved. Let me know if you have the same results

I wonder what was the difference between the servers you were using and the ones I had installed?

For my servers it worked with just the RSS disabled, for yours it obviously required Offloading to be disabled also. Maybe the NICs in my server didn't support Offloading anyway, so I just got lucky?!

Anyway, glad you got it fixed in the end - only took 3 months for the hotfix to come out!! I shall be bookmarking this page just in case I get another server with the same problem...

Simon.

I am a Microsoft Certified Trainer for ISA, an ISA Server MVP and have been a consultant for the product since the first design versions back in 99 and it makes no sense to me either. Sometimes it is easier to just the question slide rather than dig....

ASKER

orangeunderpants:
The servers I have are as follows:
Dell PE 2950, 4gb, 1.6 single quad core.
2 onboard broadcom nics
2 intel dual port pci-e
windows 2003 R2 std x86 (started with SP1 obviously)
raid 1 - 73gb x2
dual PS

I really think it was the broadcoms that were the cause, just because i took out the intels to see if that would help and it didnt. Does your setup differ from this?

qikacha

This is a case of ISA2006, Y can try to download a update.

I have a server,it install windows2003 bit 32 R2 SP1,update to SP2,I found same case under install ISA2006.

The update is for Publishing Microsoft® Exchange Server 2007 for Microsoft® Internet Security and Acceleration (ISA) Server 2006.But same fix the case of can't connect the DC

http://www.microsoft.com/downloads/details.aspx?FamilyID=82b717ce-5b63-4098-8425-bbf4a5b7e09c&DisplayLang=en

drbbton:

Of the seven ISA servers I mentioned, six would have been on HP DL360 or DL380 hardware, with only the onboard HP NICs installed, the company I work for is a HP reseller, so I deal mainly with HP hardware. (in fact since this thread started I've installed another couple of HP servers, and they both worked too once I'd put the RSS fix in place!)

However there was one Dell server in that lot, and from memory it was also a 2950, the same sort of spec. as the one you have (it's been a while now, so I've not got access to the server any more unfortunately) but with only the onboard Broadcom NICs installed.

I could believe that the HP NICs only support RSS, and not TCP offloading, and that would explain how they were unaffected, but I don't understand how I ever got the Dell server to work! I think we'll just have to chalk this one up as an anomaly... :o(

Simon.

ASKER CERTIFIED SOLUTION

Computer101

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

ckratsch

Fought with ISA all day, this worked for me:

http://support.microsoft.com/kb/927695

ckscratch - please review the member comment I have added to your profile.

keith_alabaster
Networking Zones Advisor

ckratsch

Don't worry, you've singlehandedly and successfully alienated me from returning to EE.