OK, I'll try and explain this as best I can,
Some months ago I set up an Anti-spam/Anti-virus gateway on my network. I placed it in a DMZ. When originally set up checkpoint was at version NG FP3. My problem is the following;
As part of the AS/AV gateways spam detection I was trying to allow it communicate with two spamcop servers, vmx1 and vmx2.spamcop.net. The Reporting call goes out on TCP port 587. I also expect a reply over this connection.
I set up two security rules ;
1. my_gateway to any over any for tcp 587 accept log
when this rule failed to produce the goods I later added
2. vmx1 and vmx2.spamcop.net to my_gateway over any for any accept log
The call out to vmx1 or vmx2.spamcop.net:587 goes out, no problems. However, subsequent replies are dropped by the firewal, for example;
vmx2.spamcop.net (tcp 587) to my_gateway (origin outbound source port) dropped ...
... reason 'TCP port out of state: first packet isn't SYN tcp_flags: FIN-ACK
OK, so I thought,
1. the call is going out
2. the reply is being sent
3. the firewall is stopping the reply
So then I thought that the session was timing out in the state table. I then went to the customised TCP service object for port 587 and changed it from the default timeout of 3600 seconds to 7200 seconds.
No change, packets still going out and replies still being stopped.
The firewall has been upgraded to NGX R65 in the meantime and still no change.
I believe there may be a delay in the reply by necessity, hence my efforts to extend the timeout. Is there any way I can exempt this specific type of call from the usual 'State' checks. I'm GUESSING SmartDefence is at the core of this.
Thankfully, the AS/AV gateway performs quite well despite this because it uses other checks, but I'd like to see if it improves with these calls working, and besides, it's one of those nagging problems!
I'd love to know if anyone has come across this specific issue or indeed, something akin to it.
cheers
This Question has been solved and asker verified All Experts Exchange premium technology solutions are available to subscription members.
Experts Exchange has been collecting answers to technology questions since 1996…3 million and counting! If you have a question, chances are we already have your answer.
If you can't find the exact answer you're looking for, ask our exclusive community of 50,000 experts. You’ll get a personalized answer from a trusted professional.
Thousands of free tech tips, tricks, how-to’s and tutorials are available in our peer reviewed articles section. See for yourself how smart our experts are, no login required.
Save solutions to your questions, answers you’ve discovered through searching plus helpful articles in your personal knowledgebase for easy future access.
Access the answers to your technology questions today.
30-day free trial. Register in 60 seconds.
Members of the expert community talk about why the experience at Experts Exchange is different than what you will find anywhere else.
Try it out and discover for yourself.
30-day free trial. Register in 60 seconds.
Join the community of experts here and help other tech pros by answering question in your area of expertise. You can earn FREE access to all Experts Exchange's premium features and resources.
Hi grimkin,
I haven't had a chance to try this stuff yet, but i'll get to it as soon as possible. One thing I noticed on the log viewer is that if i look at the predifined SmartDefence logs, there is no mention or these dropped calls. Can I therefore say that smartdefence is not the culprit and we're talking about straight old fashioned stateful inspection?
Thanks
Thanks again grimkin,
I'm looking for details on the use of fw monitor(I'm more of a tcpdump man myself), i'll follow that up.
By the way, the firewall is sitting on W2K Server SP4.
To complicate things, my access is via secure remote because the box is in london and i'm in ireland (via a KVM over IP which has been restricted to admin SR and specific intrenal addresses). Therefore, I can't do anything too risky on the box which may cut my access.
Cheers
To see traffic dropped but not logged:
fw ctl zdebug + drop | grep <host ip or port number>
I know this is a linux command but you can run it without the grep (tedious) or you can download a freeware port of grep for windows (dead handy).
FW monitor user guide is here: http://www.checkpoint.com/
grimkin,
I sorry about delays in getting back but I've been having other issues more visible to the users which have to be attended to as priority.
Here's what I've done so far,
I captured abuot 170K packets through fw monitor,
moved inside the network and used ethereal to decode capture file,
filtered by ip and port and saved subset,
despite the expect duplicate entries, i.e. hitting outside, then DMZ, there seemed to be quite a number of packets listed in their info as either
[TCP Dup Ack] or [TCP Retransmission], somewhat worrying.
One thing that strikes me a strange is that there appears to be somewhat of a 3 to 1 pattern in the packets. What I mean is that there appears to be instances where, for example, I get three [TCP Retransmission ] packets coming from my mail gateways internal DMZ address followed by 1 from it's external address. Similarly, I may get 1 [TCP Retransmission] packets from the spamcop.net server to the gateways extrenal address followed by 3 to the DMZ address. The behaviour is not 100 percent consistent, but prevalent enough to warrent attention or better explanation.
In the meantime, instead of disabling SmartDefence, as suggested, I disabled 'Drop out of state TCP packets' under 'Stateful Inspection' tab of Global properties. Sure enough, the packets are no longer appearing in the logs as dropped but indeed accepted. However, I'm not yet in a position to see if this has had any effect of the gateway itself.
Either way, I don't see this as a real answer. Any ideas on the greater implications of the action taken?
also for info, I was watching the logs after pushing the change and noticed that the behaviour seems to be that several calls, maybe 10 or 20, will go out to the spamcop servers, then there may be a delay of several minutes, I'd say about 5, then a torrent of replies come back. Presumabley this is the most efficient use of bandwidth for spamcop, but just to let you know the observations.
Cheers
Just as an update, i'm starting to see spamassassin rules being triggered which I don't recall seeing before ;
1.558 received via a relay in bl.spamcop.net
I'm hoping this is confirmation that the replies are no getting in.
However, doesn't it seem a bit general to have to disable the above mentioned setting (disabling 'Drop out of state TCP packets'), just to enable this improved service.
Hi Des,
If you run the fw monitor with the "-p all" switch you will get one capture entry per step in the chain *per packet* - this will give you roughly 12-16 entries per packet in the capture log and this will account for the duplicates you can see, its actually just 1 or 2 packets.
I've just noticed that I omitted something in the wireshark setup:
Edit -> preferences -> columns -> New -> Title: FW Chain, Format: FW-1 monitor if/direction
This will give you another column in the capture and if you have set up the other settings as i explained in the first post then you should be able to see entries beside this e.g. i0,i1,i2 .. I6,I7,I8 ... o1,o2 ...O5,O6 etc etc.
This is the position of the packet in the chain as described by the output of fw ctl chain.
Here is an example of a firewall chain:
in chain (11):
0: -7f800000 (a0973cd8) IP Options Strip (ipopt_strip)
1: - 2000000 (a0e4cf80) vpn decrypt (vpn)
2: - 1fffff8 (a0e50d54) l2tp inbound (l2tp)
3: - 1fffff6 (a097488c) Stateless verifications (asm)
4: - 1fffff0 (a0e4ca90) vpn decrypt verify (vpn_ver)
5: - 1000000 (a098a66c) SecureXL conn sync (secxl_sync)
6: 0 (a093eae8) fw VM inbound (fw)
7: 2000000 (a0e4dfe0) vpn policy inbound (vpn_pol)
8: 10000000 (a098a800) SecureXL inbound (secxl)
9: 7f600000 (a096f950) fw SCV inbound (scv)
10: 7f800000 (a0973f24) IP Options Restore (ipopt_res)
out chain (9):
0: -7f800000 (a0973cd8) IP Options Strip (ipopt_strip)
1: - 1ffffff (a0e4c99c) vpn nat outbound (vpn_nat)
2: - 1f00000 (a097488c) Stateless verifications (asm)
3: 0 (a093eae8) fw VM outbound (fw)
4: 2000000 (a0e4db70) vpn policy outbound (vpn_pol)
5: 10000000 (a098a800) SecureXL outbound (secxl)
6: 1ffffff0 (a0e512fc) l2tp outbound (l2tp)
7: 20000000 (a0e4d494) vpn encrypt (vpn)
8: 7f800000 (a0973f24) IP Options Restore (ipopt_res)
On the packet capture in Wireshark:
i = pre-inbound
I = post-inbound
o = pre-outbound
O = post outbound
When the packet is with the little "ix", this is it being checked by the kernel at stage x in the chain e.g. AV or vpn decrypt. Big Ix is after the kernel has inspected it then little "ox" is when it's being inspected as it leaves the interface again and so on.
SO .. e.g. you can see the packets (port 587) coming in but only reaching e.g. i3 and then no more entries, you can look at your fw ctl chain output:
If for example stage 3 on the inbound chain is "stateless verifications" and stage 4 is "vpn decrypt verify" and your packet is getting no further than i3 then your packet is being dropped at stage 4, in this case "vpn decrypt verify" and you have a better idea of where to start looking.
As a side note, I also found this:
"The most common problem is asynchronous routing, meaning that the path that any given packet takes has multiple routes either inbound or outbound. It may also happen if SmartDefense drops a packet due to a SmartDefense violation, and removes the connection from the connection table. Once the connection has been removed from the connection table, any packet other than a syn will be dropped with a TCP packet out of state."
Grimkin,
Sorry for the delay, but I've had a number of other issues arise which have been swallowing my time.
As it stands, I've effectively disabled the TCP out of state drops. As I said the calls are now appearing in the logs as permitted, as one woold expect. What are the implications of this action. Does it pose a serious risk, given my limited number of externally accessible ip addresses, (2, 1 x fw address, 1 x inbound mx).
I hope to perform more analysis with fw monitor, and will get back asap. Thanks for your patience and continued support.
Hi Keith and grimkin,
Apologies to all. I find myself in a position unable to follow through on Grimkins request for log analysis, due to circumstances beyond my control.
I therefore suggest, subject to your collective approval, that the question be closed. I would however like to award the points to Grimkin for his efforts.
Please let me know if this is acceptable, and the best way to achieve this.
Thanks and once again, my apologies for this particular outcome.
Business Accounts
Answer for Membership
by: grimkinPosted on 2007-09-17 at 02:00:25ID: 19903600
HI,
If you can disable SD for a short time to test then that would be ideal :)
Otherwise you can:
1 run the "fw ctl chain" to get the inand outbound chains
2 set up a "fw monitor" to capture all comms on port 587 with the "-p all" switch
3 debug in wireshark to see at which stage in the chain the packet is being dropped (see below). IF you see your packet constantly reaching only a certain step in the chain then the likelihood is that the one after it will be the culprit.
Set up Wireshark to interpret FW-1 captures:
1 Edit -> Preferences -> Protocols -> FW-1 -> tick all the boxes
2 Edit -> Preferences -> Protocols -> Ethernet -> Tick the box for "Attempt to interpret as FW-1 monitor file"
Let me know if you need more info on this,
CHeers