Okay, this is going to be a bit long, but this is the issue I'm having.
Our company just upgraded our ISA 2004 server to a new, better one with Forefront TMG 2010. Before we upgraded, everything worked absolutely fine. Everything was running smoothly, connecting without issues, etc. After we upgraded, we can now not make any connections with external SQL Databases (port 1433) like we were able too before the upgrade. As a note, we imported all firewall policy rules from ISA 2004. Obviously, some won't work anymore, but this one is set up how I thought it should be correctly.
Now we have a rule to allow multiple protocols from internal users to external. Two of them is Telnet and the Microsoft SQL (TCP) on port 1433. If we try to connect with Logs & Reports query monitoring the computer trying to connect, you get two messages as followed:
First one (This is caused when you try to telnet to the IP Address with port 1433): "A connection was closed because no SYN/ACK reply was received from the server."
Second one (This is caused when you try to connect to the database through the Microsoft SQL Server Management Studio): Microsoft CIFS (TCP) The Policy rules do not allow the user request. Denied Connection. Default Rule. Port 445. Internal to External IPs.
NetBios Session The policy rules do not allow the user request. Denied Connection. Default Rule. Port 139. Internal to External IPs.
NetBios Name Service The policy rules do not allow the user request. Denied Connection. Default Rule. Port 137. Internal to External IPs.
The first message you get is an Initiated Connection message saying the operation was completed successfully on port 1433 for both scenarios.
Now, obviously I thought, okay, open those ports. Well, it didn't work. so I allowed ALL outgoing traffic outbound. Still nothing. So, again, I thought maybe inbound. So I did the VERY risky rule of allow all outbound and inbound traffic (left this for 5 minutes then reverted all changes) and got Somewhat closer. Telnet still didn't work, but the database connection through the SQL studio almost got there. It connected to the database, then failed. Only one of the messages continued coming up, and that was the Microsoft CIFS (TCP) message. Which doesn't make sense if all outbound and inbound is allowed.
Telnet WILL work on the TMG server, but I don't have the SQL Studio to test.
So now, I'm rightfully confused on what to do. Been on the phone with Microsoft Support and they still haven't been able to help me either, saying it's something on our clients end. Well, I definitely tried multiple databases. Nothing. On all of them. Same messages. And again, it DID work with ISA 2004. So I highly doubt it's anything wrong on our clients end anyways. Just our TMG server.
Thanks for all of your help guys.