TMG 2010 undefined error when attempting to access site

Alright so I may have TMG setup incorrect. I'm going to give you an idea of how I have this setup and you can let me know if this is wrong. The current design may or may not have something to do with the error that I am currently getting.

The topology for how TMG is setup should be in the bottom of this post. So the TMG server is setup as a single leg firewall (wish i could just do reverse proxy and not firewall), it has a private IP associated to it and I currently have that IP setup in the internal network range which I don't think should be right? It should work using only local host but it doesn't, I have to setup the range that the TMG server is using on its interface within the internal network range for communication to function between TMG and our domain or anything else.

So the only reason we use TMG is as a reverse proxy for web sites. On the firewall coming in from the internet, I perform all nat functions. I also setup a static nat translation there to see anything coming in over a particular public IP to our TMG server if on port http or https. TMG does see these requests so I know that is functioning properly. The problem that I am having right now is with the one site I have published on TMG, an FTP site. When I try and access the site from external the request hits TMG and I get the below error message. So the message occurs when I try and login to the ftp site so you see the source and destination of what is going on but TMG blocks the connection as "Unidentified IP Traffic (TCP:47778)"?


 
Denied Connection IRPV-FFTMG 10/3/2011 11:19:49 AM
Log type: Firewall service
Status: A non-SYN packet was dropped because it was sent by a source that does not have an established connection with the Forefront TMG computer.  
Rule: None - see Result Code
Source: Local Host (X.X.X.X:443) (This is the TMG host)
Destination: External (X.X.X.X:47778) (This is the host that requested the ftp site)
Protocol: Unidentified IP Traffic (TCP:47778)
 Additional information  
 
 TMG topology
justin0104Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

justin0104Author Commented:
I have yet to receive an answer for this question?
0
pwindellCommented:
It requires patients.
Hang on an give me time to absorb this.
0
pwindellCommented:
Ok.

1. A single-nic ISA/TMG can only be a "web proxy".  It can never be anything else.  It cannot be a firewall, it can not be a Winsock Proxy (what's called the Firewall Service in ISA/TMG).  It can only be a "web proxy" and that is all.  A single-nic ISA/TMG, at least in my opinion is a waste of time and money,...why spend all that money for a high-level industrial firewall product and then throw away almost 70% of it's abilities.  But no problem,..to each his own.  Here's are article about the limitations of using a single-nic setup. It focuses on ISA2004 but ISA2006  and TMG are still the same way.
The features and limitations of a single-homed ISA Server 2004 computer
http://support.microsoft.com/kb/838364/en-us
http://www.microsoft.com/technet/prodtechnol/isa/2004/plan/single_adapter.mspx


2. Yes, the IP# of the ISA/TMG is going to be part of the Internal Network.  Everything on the LAN is going to be part of Internal,...if there are multiple IP Segmetns to the LAN then they are all part of internal but you need a Static Route added to the ISA/TMG telling it which router to use to reach those subnets.

3. The "A non-SYN packet" error means you have a topology or routing design error,...also refered to as Asynchronous Routing (which is bad).  Every connection starts with a SYN packet follow by an ACK packet, so the error means the ISA/TMG is only seeing the second half of the transaction,..that is,..it is seeing the ACK (or some other subsequent packet) without ever seeing the original SYN packet so from the ISA/TMG's perspective it is seeing an incomplete, forged, or broken session and is dropping it.

3A.  Your drawing is not valid (at least visually), also it may just be the way it was drawn.  You have to draw the logical topology (not the physical topology) in order for it to make sense.  If an L3 Switch is in use then it needs to be shown as two different physical devices (a router and a switch) because it is doing two distinct jobs.  Give me a little bit to get it done and I'll draw up something that shows how it should be lain out.

0
How do you know if your security is working?

Protecting your business doesn’t have to mean sifting through endless alerts and notifications. With WatchGuard Total Security Suite, you can feel confident that your business is secure, meaning you can get back to the things that have been sitting on your to-do list.

pwindellCommented:
Ok.
Here is a drawing of a single-nig ISA/TMG on a multi-segment LAN. The single-nic ISA/TMG can be place anywhere in any segment of the LAN,...it really does not matter.  If the LAN Router is an L3 Switch then the Router Icon and the three Switch Icons could potentially all be the same physical device,...but because the logical design is what is important they are shown as separate devices.  Even if a regular L2 Switch was in the diagram,... if it was split in half via VLANing,...it would be shown as two or more separate switches (one per segment) even though it was physically just one device.

 Single nic ISA
0
justin0104Author Commented:
Yes I agree with your topology and this is pretty much the way I have it setup. I didn't draw it out the way you did simply because I have many VLANs and I bridge between vlans using the ASA. So the TMG box is on a l2 vlan 70 and the l3 vlan is 2000, the firewall bridges 70 to 2000 for any l3 work that needs to be performed. I would agree with you regarding the reasoning for the syn and the ack and how tmg must be hearing the conversation out of sync but it only happens occasionally but when it does occur then obviously the TMG box denies it.

As for wasting much of the potential that TMG has, we are a MS gold partner so we get it for free :).

I know you said in a 1 nic setup that TMG should not act as a firewall but it does. I had to create rules to allow TMG to communicate with our domain controllers and to allow tmg out to the web. Should this be necessary?
0
pwindellCommented:
As for wasting much of the potential that TMG has, we are a MS gold partner so we get it for free :).

Ok,...that's cheating :-)

I know you said in a 1 nic setup that TMG should not act as a firewall but it does. I had to create rules to allow TMG to communicate with our domain controllers and to allow tmg out to the web. Should this be necessary?

It is not acting as a Firewall,..just not possible.  But a Web Proxy can also "allow" or "deny",..so don't confuse that with it being a firewall.  A web proxy can be bypassed by simply "not using it",...a firewall cannot be bypassed that way.

On the other things you asked.....

1. Domain Controller communication,...that is supposed to be done with System Policies,..not Access Rules.  ISA/TMG is supposed to have the machine it is installed on to already be a domain member before the ISA/TMG software is installed. In which case the Installation routines detect that the machine is a Domain Member and will create the proper System Policies during the installation.  If you did not do it in that order then you would have to manually create the system policies.  In my experience it is usually more dependable to let the Installation do that for you.  

2. The ISA/TMG is also not part of the Internal Network,...it is only part of LocalHost and nothing else no matter what IP#s are use and no matter where you use them,...keep that in mind,...this has nothing to do with IP#s,...yes the Nic has an IP from the Internal Network, however functionally, the ISA/TMG does not trust the Internal LAN,...it doesn't trust anything but itself.  By default the ISA/TMG will not communicate with the Internal LAN except for what is required for it to do its job (hence the System Policies).  Also,..by default,...the ISA/TMG will not communicate with the Internet itself, meaning it is perfectly normal,...and expected that you cannot sit at the ISA console and open Internet Explorer and browse the Internet.  Both of these aspects are by design and intentional, but you can over-ride them by creating Access Rules to do what you want.
0
pwindellCommented:
I know you said in a 1 nic setup that TMG should not act as a firewall but it does. I had to create rules to allow TMG to communicate with our domain controllers and to allow tmg out to the web. Should this be necessary?

ISA/TMG is always a "firewall to itself".  It will always protect itself,..in both directions, in to it  or out from it.  Even the Firewall Service works in that manner because it has two logical interfaces (LocalHost -vs- Internal),...however it cannot be used as a firewall for Clients because nothing can ever be part of LocalHost but itself.
0
justin0104Author Commented:
So is it normal then at if I remove the local subnet that TMG is on from the internal network then I lose network connectivity? I have an access rule created that says all outbound traffic to all protected networks is allowed but if I remove that subnet from the internal network then it looses connectivity.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
pwindellCommented:
Every IP# on the LAN needs to be in the Internal Network, including the IP# of the ISA/TMG.  Usually you give it the whole range,..for example if your LAN has three segments,...192.168.10.0/24, 192.168.11.0/24, and 192.168.12.0/24,...then you would give the Internal Network a range of 192.168.10.0 through 192.168.12.255.  You could also give it the three separate ranges, but in any case it still needs to include the full ranges total.  The ISA/TMG's Internal Network is a "logical entity" and may include multiple IP segments,...even if they might be remote locations over private WAN Links,...basically it comes down to it has to include all IP ranges that are "reachable" from the Internal Nic regardless of where they are physically located

If you remove the address as you did then that will break it, yes.  I suspect the TMG is all working as it should, you just have to get used to how it works and what is normal and what it not normal.
0
justin0104Author Commented:
i'm awesome
0
justin0104Author Commented:
ahhh
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft Forefront ISA Server

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.