Connection issues with Upgrade

Hello everyone,

Okay, this is going to be a bit long, but this is the issue I'm having.

Our company just upgraded our ISA 2004 server to a new, better one with Forefront TMG 2010. Before we upgraded, everything worked absolutely fine. Everything was running smoothly, connecting without issues, etc. After we upgraded, we can now not make any connections with external SQL Databases (port 1433) like we were able too before the upgrade. As a note, we imported all firewall policy rules from ISA 2004. Obviously, some won't work anymore, but this one is set up how I thought it should be correctly.

Now we have a rule to allow multiple protocols from internal users to external. Two of them is Telnet and the Microsoft SQL (TCP) on port 1433. If we try to connect with Logs & Reports query monitoring the computer trying to connect, you get two messages as followed:

First one (This is caused when you try to telnet to the IP Address with port 1433): "A connection was closed because no SYN/ACK reply was received from the server."

Second one (This is caused when you try to connect to the database through the Microsoft SQL Server Management Studio): Microsoft CIFS (TCP) The Policy rules do not allow the user request. Denied Connection. Default Rule. Port 445. Internal to External IPs.
NetBios Session The policy rules do not allow the user request. Denied Connection. Default Rule. Port 139. Internal to External IPs.
NetBios Name Service The policy rules do not allow the user request. Denied Connection. Default Rule. Port 137. Internal to External IPs.

The first message you get is an Initiated Connection message saying the operation was completed successfully on port 1433 for both scenarios.

Now, obviously I thought, okay, open those ports. Well, it didn't work. so I allowed ALL outgoing traffic outbound. Still nothing. So, again, I thought maybe inbound. So I did the VERY risky rule of allow all outbound and inbound traffic (left this for 5 minutes then reverted all changes) and got Somewhat closer. Telnet still didn't work,  but the database connection through the SQL studio almost got there. It connected to the database, then failed. Only one of the messages continued coming up, and that was the Microsoft CIFS (TCP) message. Which doesn't make sense if all outbound and inbound is allowed.

Telnet WILL work on the TMG server, but I don't have the SQL Studio to test.

So now, I'm rightfully confused on what to do. Been on the phone with Microsoft Support and they still haven't been able to help me either, saying it's something on our clients end. Well, I definitely tried multiple databases. Nothing. On all of them. Same messages. And again, it DID work with ISA 2004. So I highly doubt it's anything wrong on our clients end anyways. Just our TMG server.

Thanks for all of your help guys.
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Keith AlabasterEnterprise ArchitectCommented:
Not dealing with this on a bloody iPhone. Will reply when I get home from work.

Keith AlabasterEnterprise ArchitectCommented:
Really surprised your instructor did not cover this off when you attended the training course for TMG (and ISA come to think of it) but need more info from you.
Do you have TMG SP1 and the four SP1 updates applied?
As you are using those ports I'll assume this is an FTMG installed as a firewall/proxy, not just a proxy server.
Are you using the TMG firewall client or Securenat (default gateway pointing to TMG and web browser proxy settings pointing at TMG?

Confirm all traffic is being INITIATED from the inside to contact services on the outside somewhere?
Provide an ipconfig /all from the TMG box please.
ITGateAuthor Commented:

We actually didn't have an instructor teach us anything about Forefront. My boss is extremely experienced with ISA, but unfortunately he isn't in at all this week and when we need him, we can't get ahold of him.

I did have TMG SP1 installed, but before we had this issue we had to uninstall FTMG and reinstall it for a different issue. I forgot to install the service pack after that, so it's not installed currently. Is that one of the changes they made in to help with firewall policy?

TMG, yeah, it's set up as a firewall.

I'm not sure the difference between the firewall client and securenat, but right now it's the FTMG Managemenet. We have the computers on the internal network pointing to the TMG server as their default gateway, as you mentioned in the ()'s.

All traffic is being initiated inside to contact services outside. We tried it on an internal computer.
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

Keith AlabasterEnterprise ArchitectCommented:
the service pack and the additional rollups fix a number of issues - service orientated and functional ones. No point trying to move on with something that is not up to date. The SP is easy to get and the rollups are on request - you'll find tyhem and see what I mean.

FTMG firewall client is a small bit of client-side software that can be installed. SecureNAT is where you just allow default gateways/routing to ensure all traffic gets to the TMG box in the first place.

For reference, allow all traffic does not mean what it sounds like, it means allow all traffic that is defined in the TMG protocol list AND, where appropriate, is authorised based on the settings with in the FTMG System Policy (NOT the firewall Policy).

Still need to see the ipconfig /all please - just want to check the obvious.

if all the traffic is from inside to outside then there is no need to create any inbound rules at all for this - when the outbound rule is triggered it creates a session and this will enable return traffic as part of the session control.

ITGateAuthor Commented:
Hey Keith,

For the SP I just installed it, but I can't get the rollups yet as of yet since we cannot restart again in the middle of the day.

From the description you gave, we would be using SecureNAt then. We do not have a TMG firewall client installed on the clients internally.

I do understand what you mean with the SystemPolicy, also.

I did not see that part of the message before, otherwise I would've had it attached already. It is attached now.

I didn't think there was a need to configure an inbound rule, either. I just wanted to cover all basis and make sure it was good.
Keith AlabasterEnterprise ArchitectCommented:
The external nic should not have any DNS entries - should be blank. Only the internal TMG nic should have dns entries and these shoupld point to the INTERNAL dns servers only. Absolutely no server or client machine should have ANY reference to the external DNS ip addresses at all. The only place where the external DNS addresses should be held is in the forwarders tabs of your internal dns servers.
(It may be the way you have it but it is hard to tell from your ipconfig output as you have masked the outputs)

Never seen the single external nic have two different subnets assigned due to address selection. I have several hundred ISA & TMG installs under my belt and that is a first. When there are multiple external nics involved such as when ISP-R is being used then yes but not with a single external nic.

I'll assume you are natting between the internal and external rather than having a routed relationship?
Looking at the system policy, are you allowing netbios-type traffic to pass?

ITGateAuthor Commented:
If I'm not mistaken in my reading of this ipconfig output (and I'm pretty sure I'm not because even if I don't remember a LOT from it, I did take Cisco classes for 6 years), there is no DNS entry for the External. Only Internal. There's the Ethernet adapter External which has all of the IP Addresses and a default gateway, but no DNS. The one right below, Ethernet adapter Internal, has the IP Address for it internally and has the DNS servers. Not external, though.

Yeah, this is how my boss had it set up. It worked perfectly on ISA 2004, which I would have thought it would've worked now. But evidently it's giving trouble.

Yes, we are NATing between internal and external instead of routed. This is one thing I was actually looking into, also. We have it set to default IP instead of specific of multiple IP NAT Addresses.

System Policy, going into edit it I did not see anything on Netbios.
ITGateAuthor Commented:
Boss came in once we were able to get a hold of him and we were actually able to figure it out. NAT was actually the cause here with the new FTMG. Instead of picking the primary IP Address that's on the External adapter, it instead supposedly uses an "algorithm" to pick the default IP. We had to change it from Default IP Address in Networking to Specified. After we switched it, it seems to have fixed it.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
ITGateAuthor Commented:
Accepting mine as the solution.
Keith AlabasterEnterprise ArchitectCommented:
Selects in ip address order - horrid isn't it
ITGateAuthor Commented:
That it is. That it is. Now the only issue is one set of Cisco VPN going out because of NetBios Name Service. But that should be fairly easy to fix.

Thanks for your help Keith. Greatly appreciate it.
Keith AlabasterEnterprise ArchitectCommented:
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft Forefront ISA Server

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.