Solved

Cisco NAT port 25 and Barracuda Spam filter.

Posted on 2008-06-22
97
2,756 Views
Last Modified: 2012-08-13
Hi,

Im using Cisco1840 with NAT rules to forward mail to my exchange. Im facing a problem now, once, sometimes twice a week my Cisco router stops answering for port 25 and I cant figure out why. Internet is working as well is all other port forwarding rules. Can someone tell me if there is any problem with my config file?

Any help will be appreciated.


!
version 12.4
!
!
!
!
interface Loopback1
 description NAT Outside
 ip address 2xx.xx6.xx.x 255.255.255.255
 ip nat outside
 ip virtual-reassembly
!
interface Loopback2
 ip address 2xx.xx6.xx.x 255.255.255.255
 ip nat outside
 ip virtual-reassembly
 ip tcp adjust-mss 1452
!
interface FastEthernet0/0
 description LAN
 ip address 10.1.1.18 255.255.255.0
 ip nat inside
 ip virtual-reassembly max-fragments 16 max-reassemblies 64 timeout 5
 duplex auto
 speed auto
!
interface FastEthernet0/1
 no ip address
 shutdown
 duplex auto
 speed auto
!
interface Serial0/0/0
 description WAN / Internet
 ip address xx6.xx6.xx.22 255.255.255.252
 ip nat outside
 ip virtual-reassembly
!
interface Virtual-Template1 type tunnel
 ip unnumbered Loopback2
 tunnel mode ipsec ipv4
 tunnel protection ipsec profile SDM_Profile1
!
ip local pool SDM_POOL_1 10.1.1.220 10.1.1.225
ip local pool SDM_POOL_2 10.1.1.230 10.1.1.240
ip route 0.0.0.0 0.0.0.0 2xx.xx6.xx.21
!
!
ip http server
ip http secure-server
ip nat inside source list 98 interface Loopback2 overload
ip nat inside source list 99 interface Loopback1 overload
ip nat inside source static tcp 10.1.1.3 25 2xx.xx6.x.1 25 extendable
ip nat inside source static tcp 10.1.1.3 110 2xx.xx6.x.1 110 extendable
ip nat inside source static tcp 10.1.1.3 143 2xx.xx6.x.1 143 extendable
ip nat inside source static tcp 10.1.1.15 25 2xx.xx6.x.2 25 extendable
ip nat inside source static tcp 10.1.1.3 110 2xx.xx6.x.2 110 extendable
ip nat inside source static tcp 10.1.1.3 143 2xx.xx6.x.2 143 extendable
!
logging 10.1.1.40
access-list 98 permit 10.1.1.11
access-list 98 permit 10.1.1.15
access-list 98 permit 10.1.1.40
access-list 99 deny   10.1.1.11
access-list 99 deny   10.1.1.15
access-list 99 deny   10.1.1.40
access-list 99 permit 10.1.1.0 0.0.0.255
access-list 100 remark SDM_ACL Category=4
access-list 100 permit ip 10.1.1.0 0.0.0.255 any
access-list 101 remark SDM_ACL Category=4
access-list 101 permit ip 10.1.1.0 0.0.0.255 any
access-list 102 remark SDM_ACL Category=4
access-list 102 permit ip 10.1.1.0 0.0.0.255 any



10.1.1.15 is my Barracuda SPAM Box
10.1.1.3  is my Exchange server

My MX  record 2xx.xx6.x.2 so all emails gets to my spam box first and then forward to exchange.
0
Comment
Question by:stasila2010
  • 58
  • 39
97 Comments
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
Are you sure that it's the Cisco that's not accepting the packets and forwarding them on as opposed to the Barracuda?

When this problem occurs -- clear your access-list counters on the Cisco, start up syslog on the Barracuda and send a test email.

Check the access-list counters on the cisco and review the syslog data on the Barracuda.

And, if all mail relays through the Barracuda before reaching the exchange server, I wonder why you allow port 25 connections direct to exchange?
0
 

Author Comment

by:stasila2010
Comment Utility
Jesper,

thanks for quick respond. I'm not 100% sure if it Cisco router or Barracuda. I know when I can't reach port 25 from outside my network I can always telnet to port 25 on Barracuda internally.

Is this is a right command to clear counters "clear access-list <ACL_name> counters" ?

our external users can access Exchange using a second IP address by passing Barracuda box. They using POP3 and IMAP.

Should I clear counters and turn syslog on Barracuda only when it happens again, right? If I do it right now it will be pointless?

Thanks.
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
clear access-list counters ?

specify the number word or clear all counters.

yes, wait until the problem occurs.

i may be tired, but i don't see the access-list allowing inbound TCP to port 25 and applied to the internet interface.
0
 

Author Comment

by:stasila2010
Comment Utility
isn't this line is all I need ? ip nat inside source static tcp 10.1.1.15 25 2xx.xx6.x.2 25 extendable

0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
That sets up the static translation.  And yes, this is all that you need if you are not running a firewall (my apologies).
0
 

Author Comment

by:stasila2010
Comment Utility
thanks, I will try what you suggested and post updates.
0
 

Author Comment

by:stasila2010
Comment Utility
I tried what you suggested, clear access-list counters and turn syslog on.

here is my syslog from Barracuda

Jun 24 08:54:10 spam outbound/smtp[3997]: 127.0.0.1 1214311928-0f8800000000-1X4NT9 0 0 SEND - 3 9851F3BF8E connect to gsmtp183.google.com[64.233.183.27]: Connection timed out
ýJun 24 08:54:51 spam scan[21742]: UNKNOWN[10.1.1.3] 1214312090-0fdf00000000-9EWdOI 1214312091 1214312091 SCAN - mike@xxx.com mike@gmail.com - 0 0 - SZ:19002 SUBJ:test
ýJun 24 08:56:51 spam outbound/smtp[3997]: 127.0.0.1 1214312090-0fdf00000000-9EWdOI 0 0 SEND - 3 C71F23BF9A connect to gsmtp183.google.com[64.233.183.27]: Connection timed out


I was able to connect to barracuda from inside my network but not from outside.

Please advice.
thanks



0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
The email made it all the way to the Barracuda and is trying to deliver to Google domains.

Some questions that occur to me are:

1) is the connection problem always with Google?
2) when you see an error trying to connect or deliver to a domain from the Barracuda, do you experience problems elsewhere and can you duplicate the problem manually right after it happens?
0
 

Author Comment

by:stasila2010
Comment Utility
I can connect to Barracuda internally all emails are getting to Barracuda from Exchange and from Exchange to Barracuda without any problems. Internally is no problem port 25 is always answering on Exchange and Barracuda. It happens randomly that port 25 stop responding from outside of my network. As soon as I reboot Cisco route everything back to normal. I think it has something to do with my NAT.

1) I was trying to send email from/to my Gmail account, that's why there is a google domain.  Its a same problem for all domains.

2) No, I can duplicate this problem.

Thanks
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
Ok.  So this is an internal send.  The router won't just randomly block data to a specific port.

It sounds like the connection between the Barracuda and the router is a problem.  Are they directly connected?  Do you see errors on the ethernet interface of the router?  If you use a switch, do you see errors on the switch port?  Have you swapped out the ethernet cables for the router and the Barracuda?

When mail quits flowing out, does anything else break?
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
I would also like you to change the inside interface (though I can't see this as a problem with a small file on a test send):

from this ->  ip virtual-reassembly max-fragments 16 max-reassemblies 64 timeout 5
to this     ->  ip virtual-reassembly

And to determine whether the packets are even making it to the router, please apply the following access list.  This needs to be removed after we determine on the next failed run the status of the packets:

config t
access-list 199 permit tcp 10.1.1.0 0.0.0.255 any eq 25 log
access-list 199 permit ip any any
interface FastEthernet0/0
 ip access-group 199 in

I've had problems with NIC cards in Barracuda products.  We need to rule out problems with the NICs on any other connected devices.
0
 

Author Comment

by:stasila2010
Comment Utility
No, Barracuda directly connected to 4 port switch which is not manageable and then switch is connected to Cisco router so I can check for errors on switch.

below is output from "show logging" command on Cisco router


show logging
Syslog logging: enabled (1 messages dropped, 1 messages rate-limited,
                0 flushes, 0 overruns, xml disabled, filtering disabled)

No Active Message Discriminator.



No Inactive Message Discriminator.


    Console logging: level debugging, 37 messages logged, xml disabled,
                     filtering disabled
    Monitor logging: level debugging, 0 messages logged, xml disabled,
                     filtering disabled
    Buffer logging:  level warnings, 8 messages logged, xml disabled,
                     filtering disabled
    Logging Exception size (4096 bytes)
    Count and timestamp logging messages: disabled

No active filter modules.

    Trap logging: level informational, 39 message lines logged
        Logging to 10.1.1.40  (udp port 514,  audit disabled,
              authentication disabled, encryption disabled, link up),
              39 message lines logged,
              0 message lines rate-limited,
              0 message lines dropped-by-MD,
              xml disabled, sequence number disabled
              filtering disabled

Log Buffer (4659 bytes):

*Jun 24 12:52:03.083: %LINK-3-UPDOWN: Interface Serial0/0/0, changed state to down
*Jun 24 12:52:20.983: %LINK-3-UPDOWN: Interface Serial0/0/0, changed state to up
*Jun 24 13:10:02.979: %ALIGN-3-CORRECT: Alignment correction made at 0x602932D0  reading 0xE75480CE
*Jun 24 13:10:02.983: %ALIGN-3-TRACE: -Traceback= 0x602932D0 0x60296474 0x6003C21C 0x6002A12C 0x62190B94 0x62197FC0 0x621947A0 0x621949F8
*Jun 24 13:10:02.983: %ALIGN-3-CORRECT: Alignment correction made at 0x602932D0  reading 0xE75543CE
*Jun 24 13:10:02.983: %ALIGN-3-TRACE: -Traceback= 0x602932D0 0x60295EA0 0x6003C21C 0x6002A12C 0x62190B94 0x62197FC0 0x621947A0 0x621949F8
*Jun 24 13:10:02.983: %ALIGN-3-CORRECT: Alignment correction made at 0x602932D0  reading 0xE7557E4E
*Jun 24 13:10:02.983: %ALIGN-3-TRACE: -Traceback= 0x602932D0 0x60295EA0 0x6003C21C 0x6002A12C 0x62190B94 0x62197FC0 0x621947A0 0x62194A68


is there any other command to display loggin on interface?
0
 

Author Comment

by:stasila2010
Comment Utility
are we applying this to internal interface, correct?

config t
access-list 199 permit tcp 10.1.1.0 0.0.0.255 any eq 25 log
access-list 199 permit ip any any
interface FastEthernet0/0
 ip access-group 199 in


0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
Yes, it lists f0/0 which is the private interface.

However, the problem may be with the serial interface.  What does a "show interface serial0/0/0" give you?  You may have a problem with your circuit, CSU or smartjack.  I would have been interested in the log info on the router at the time of the email send.  
0
 

Author Comment

by:stasila2010
Comment Utility
OK I just applied access-list to internal interface

interface FastEthernet0/0
 description LAN
 ip address 10.1.1.18 255.255.255.0
 ip access-group 199 in
 ip nat inside
 ip virtual-reassembly
 duplex auto
 speed auto



access-list 199 permit tcp 10.1.1.0 0.0.0.255 any eq smtp log
access-list 199 permit ip any any


0
 

Author Comment

by:stasila2010
Comment Utility
output from serial interface

Serial0/0/0 is up, line protocol is up
  Hardware is GT96K with integrated T1 CSU/DSU
  Description: WAN / Rogers Internet
  Internet address is 206.186.248.22/30
  MTU 1500 bytes, BW 1544 Kbit, DLY 20000 usec,
     reliability 255/255, txload 9/255, rxload 28/255
  Encapsulation HDLC, loopback not set
  Keepalive set (10 sec)
  Last input 00:00:00, output 00:00:00, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/109/0 (size/max/drops/flushes); Total output drops: 2
  Queueing strategy: weighted fair
  Output queue: 0/1000/64/2 (size/max total/threshold/drops)
     Conversations  0/30/256 (active/max active/max total)
     Reserved Conversations 0/0 (allocated/max allocated)
     Available Bandwidth 1158 kilobits/sec
  5 minute input rate 175000 bits/sec, 16 packets/sec
  5 minute output rate 56000 bits/sec, 14 packets/sec
     183529 packets input, 168068693 bytes, 0 no buffer
     Received 528 broadcasts, 0 runts, 0 giants, 0 throttles
     1 input errors, 1 CRC, 0 frame, 1 overrun, 0 ignored, 0 abort
     152661 packets output, 48512957 bytes, 0 underruns
     0 output errors, 0 collisions, 4 interface resets
     0 output buffer failures, 0 output buffers swapped out
     1 carrier transitions
     DCD=up  DSR=up  DTR=up  RTS=up  CTS=up
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
That's pretty clean.  Let's see if we can duplicate the problem again.  When it happens:

1) turn syslog on the Barracuda
2) clear access-list counters 199
3) clear count s0/0/0
4) send a test email
5) sh log (on the router)
6) sh int f0/0
7) sh int s0/0/0

If we have to, we can turn on debugging.
0
 

Author Comment

by:stasila2010
Comment Utility
thanks for your help I will post updates when in happens again.
0
 

Author Comment

by:stasila2010
Comment Utility
Hi,

here is a syslog from Barracuda

Jun 25 08:08:44 spam outbound/smtp[13218]: 127.0.0.1 1214256754-6a3900190000-Sd1QNx 0 0 SEND - 3 92F3C3BB68 connect to strand.ca[208.73.212.12]: Connection timed out
Jun 25 08:08:48 spam scan[6695]: UNKNOWN[10.1.1.3] 1214395727-33a400000000-9EWdOI 1214395727 1214395728 SCAN - mike@xxx.com mike@gmail.com - 0 0 - SZ:19002 SUBJ:test
Jun 25 08:09:26 spam scan[6695]: UNKNOWN[10.1.1.3] 1214395765-33a400010000-A2ipsz 1214395765 1214395766 SCAN - mike@xxx.com mike@yahoo.com - 0 0 - SZ:18990 SUBJ:test

sh log (on router)

Router1-Internet#sh log
Syslog logging: enabled (1 messages dropped, 1 messages rate-limited,
                0 flushes, 0 overruns, xml disabled, filtering disabled)

No Active Message Discriminator.



No Inactive Message Discriminator.


    Console logging: level debugging, 188 messages logged, xml disabled,
                     filtering disabled
    Monitor logging: level debugging, 0 messages logged, xml disabled,
                     filtering disabled
    Buffer logging:  level warnings, 12 messages logged, xml disabled,
                     filtering disabled
    Logging Exception size (4096 bytes)
    Count and timestamp logging messages: disabled

No active filter modules.

    Trap logging: level informational, 190 message lines logged
        Logging to 10.1.1.40  (udp port 514,  audit disabled,
              authentication disabled, encryption disabled, link up),
              190 message lines logged,
              0 message lines rate-limited,
              0 message lines dropped-by-MD,
              xml disabled, sequence number disabled
              filtering disabled

Log Buffer (4659 bytes):

*Jun 24 21:02:41.055: %LINK-3-UPDOWN: Interface Serial0/0/0, changed state to down
*Jun 24 21:02:59.491: %LINK-3-UPDOWN: Interface Serial0/0/0, changed state to up
*Jun 24 21:39:40.983: %ALIGN-3-CORRECT: Alignment correction made at 0x602932D0 reading 0xE75591CE
*Jun 24 21:39:40.983: %ALIGN-3-TRACE: -Traceback= 0x602932D0 0x60296474 0x6003C21C 0x6002A12C 0x62190B94 0x62197FC0 0x621947A0 0x621949F8
*Jun 24 21:39:40.983: %ALIGN-3-CORRECT: Alignment correction made at 0x602932D0 reading 0xE74EF14E
*Jun 24 21:39:40.983: %ALIGN-3-TRACE: -Traceback= 0x602932D0 0x60295EA0 0x6003C21C 0x6002A12C 0x62190B94 0x62197FC0 0x621947A0 0x62194A68
*Jun 24 23:29:40.983: %ALIGN-3-CORRECT: Alignment correction made at 0x602932D0 reading 0xE74EF7CE
*Jun 24 23:29:40.983: %ALIGN-3-TRACE: -Traceback= 0x602932D0 0x60295EA0 0x6003C21C 0x6002A12C 0x62190B94 0x62197FC0 0x621947A0 0x621949F8
*Jun 24 23:37:40.983: %ALIGN-3-CORRECT: Alignment correction made at 0x602932D0 reading 0xE754874E
*Jun 24 23:37:40.983: %ALIGN-3-TRACE: -Traceback= 0x602932D0 0x60296474 0x6003C21C 0x6002A12C 0x62190B94 0x62197FC0 0x621947A0 0x62194A68
*Jun 25 11:18:40.980: %ALIGN-3-CORRECT: Alignment correction made at 0x602932D0 reading 0xE754B4CE
*Jun 25 11:18:40.980: %ALIGN-3-TRACE: -Traceback= 0x602932D0 0x60295EA0 0x60020C80 0x600424D0 0x6002A12C 0x62190B94 0x62197FC0 0x621947A0


sh int f0/0


Router#sh int f0/0
FastEthernet0/0 is up, line protocol is up
  Hardware is Gt96k FE, address is 001c.5847.3414 (bia 001c.5847.3414)
  Description: LAN
  Internet address is 10.1.1.18/24
  MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Full-duplex, 100Mb/s, 100BaseTX/FX
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 00:00:00, output 00:00:00, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 19000 bits/sec, 17 packets/sec
  5 minute output rate 11000 bits/sec, 13 packets/sec
     678207 packets input, 167865091 bytes
     Received 91501 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog
     0 input packets with dribble condition detected
     761254 packets output, 759828776 bytes, 0 underruns
     0 output errors, 0 collisions, 2 interface resets
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier
     0 output buffer failures, 0 output buffers swapped out

sh int s0/0/0

Router#sh int s0/0/0
Serial0/0/0 is up, line protocol is up
  Hardware is GT96K with integrated T1 CSU/DSU
  Description: WAN /  Internet
  Internet address is 206.1xx.2xx.22/30
  MTU 1500 bytes, BW 1544 Kbit, DLY 20000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation HDLC, loopback not set
  Keepalive set (10 sec)
  Last input 00:00:01, output 00:00:00, output hang never
  Last clearing of "show interface" counters 00:14:15
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: weighted fair
  Output queue: 0/1000/64/0 (size/max total/threshold/drops)
     Conversations  0/22/256 (active/max active/max total)
     Reserved Conversations 0/0 (allocated/max allocated)
     Available Bandwidth 1158 kilobits/sec
  5 minute input rate 7000 bits/sec, 10 packets/sec
  5 minute output rate 7000 bits/sec, 9 packets/sec
     10429 packets input, 1234104 bytes, 0 no buffer
     Received 85 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     10002 packets output, 2365271 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 output buffer failures, 0 output buffers swapped out
     0 carrier transitions
     DCD=up  DSR=up  DTR=up  RTS=up  CTS=up


0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
What was the "show access-list 199" when you had the problem, cleared the counters and ran a test message through.

I dimissed the traceback messages earlier since the date/timestamp did not match up.  Reconsidering, it appears that the clock on the router is incorrect.  It appears that you have a problem with the memory (it may be the cause).  Please, still let me know the ACL from above.
0
 

Author Comment

by:stasila2010
Comment Utility
I didn't check when it happened. I guess I will have to wait till it happens one more time.

this is how it looks after I rebooted it

Extended IP access list 199
    10 permit tcp 10.1.1.0 0.0.0.255 any eq smtp log (285 matches)
    20 permit ip any any (8999 matches)

how can we check if time is correct or not? and what about memory?

thanks,
0
 

Author Comment

by:stasila2010
Comment Utility
next time when it happens I should do this:

1) turn syslog on the Barracuda
2) clear access-list counters 199
3)sh access-list 199
4) clear count s0/0/0
5) send a test email
6) sh log (on the router)
7) sh int f0/0
8) sh int s0/0/0

right?
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
yes.

this way acl 199 does not reflect traffic that has passed before the problem.

this acl will tell us whether the traffic is successfully leaving the barracuda.

please also add this acl:

access-list 198 permit tcp 10.1.1.0 0.0.0.255 any eq smtp log
access-list 198 permit ip any any
int s0/0/0
 ip access-group 198 out

when you clear acl 199 please also clear acl 198.

after testing the problem, i'd like to see "sh access-list 198" andd "sh access-list 199"
0
 

Author Comment

by:stasila2010
Comment Utility
Extended IP access list 199
    10 permit tcp 10.1.1.0 0.0.0.255 any eq smtp log
    20 permit ip any any (2299 matches)

Extended IP access list 198
    10 permit tcp 10.1.1.0 0.0.0.255 any eq smtp log
    20 permit ip any any (2477 matches)


sh log

Syslog logging: enabled (1 messages dropped, 2 messages rate-limited,
                0 flushes, 0 overruns, xml disabled, filtering disabled)

No Active Message Discriminator.



No Inactive Message Discriminator.


    Console logging: level debugging, 1565 messages logged, xml disabled,
                     filtering disabled
    Monitor logging: level debugging, 0 messages logged, xml disabled,
                     filtering disabled
    Buffer logging:  level warnings, 11 messages logged, xml disabled,
                     filtering disabled
    Logging Exception size (4096 bytes)
    Count and timestamp logging messages: disabled

No active filter modules.

    Trap logging: level informational, 1566 message lines logged
        Logging to 10.1.1.40  (udp port 514,  audit disabled,
              authentication disabled, encryption disabled, link up),
              1566 message lines logged,
              0 message lines rate-limited,
              0 message lines dropped-by-MD,
              xml disabled, sequence number disabled
              filtering disabled

Log Buffer (4659 bytes):

*Jun 25 12:18:48.067: %LINK-3-UPDOWN: Interface Serial0/0/0, changed state to do                                                                                                 wn
*Jun 25 12:19:06.103: %LINK-3-UPDOWN: Interface Serial0/0/0, changed state to up
*Jun 25 13:03:47.967: %ALIGN-3-CORRECT: Alignment correction made at 0x602932D0                                                                                                   reading 0xE74DB94E
*Jun 25 13:03:47.971: %ALIGN-3-TRACE: -Traceback= 0x602932D0 0x60296474 0x6003C2                                                                                                 1C 0x6002A12C 0x62190B94 0x62197FC0 0x621947A0 0x621949F8
*Jun 25 13:03:47.971: %ALIGN-3-CORRECT: Alignment correction made at 0x602932D0                                                                                                   reading 0xE74E074E
*Jun 25 13:03:47.971: %ALIGN-3-TRACE: -Traceback= 0x602932D0 0x60295EA0 0x6003C2                                                                                                 1C 0x6002A12C 0x62190B94 0x62197FC0 0x621947A0 0x62194A68
*Jun 25 13:03:47.971: %ALIGN-3-CORRECT: Alignment correction made at 0x602932D0                                                                                                   reading 0xE74F04CE
*Jun 25 13:03:47.971: %ALIGN-3-TRACE: -Traceback= 0x602932D0 0x60295EA0 0x6003C2                                                                                                 1C 0x6002A12C 0x62190B94 0x62197FC0 0x621947A0 0x621949F8
*Jun 25 13:06:47.967: %ALIGN-3-CORRECT: Alignment correction made at 0x602932D0                                                                                                   reading 0xE74F2BCE
*Jun 25 13:06:47.967: %ALIGN-3-TRACE: -Traceback= 0x602932D0 0x60296474 0x6003C2                                                                                                 1C 0x6002A12C 0x62190B94 0x62197FC0 0x621947A0 0x62194A68
Jun 26 09:53:14.018: %CRYPTO-4-IKMP_NO_SA: IKE message from 76.18.184.160 has no                                                                                                  SA and is not an initialization offer





FastEthernet0/0 is up, line protocol is up
  Hardware is Gt96k FE, address is 001c.5847.3414 (bia 001c.5847.3414)
  Description: LAN
  Internet address is 10.1.1.18/24
  MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Full-duplex, 100Mb/s, 100BaseTX/FX
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 00:00:00, output 00:00:00, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 46000 bits/sec, 13 packets/sec
  5 minute output rate 8000 bits/sec, 11 packets/sec
     5606937 packets input, 1256095740 bytes
     Received 304717 broadcasts, 0 runts, 0 giants, 0 throttles
     1 input errors, 0 CRC, 0 frame, 1 overrun, 0 ignored
     0 watchdog
     0 input packets with dribble condition detected
     7318467 packets output, 3735627768 bytes, 0 underruns
     0 output errors, 0 collisions, 2 interface resets
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier
     0 output buffer failures, 0 output buffers swapped out





int s0/0/0



Serial0/0/0 is up, line protocol is up
  Hardware is GT96K with integrated T1 CSU/DSU
  Description: WAN /  Internet
  Internet address is 206.xx.248.22/30
  MTU 1500 bytes, BW 1544 Kbit, DLY 20000 usec,
     reliability 255/255, txload 5/255, rxload 1/255
  Encapsulation HDLC, loopback not set
  Keepalive set (10 sec)
  Last input 00:00:09, output 00:00:05, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/8320/0 (size/max/drops/flushes); Total output drops: 174
  Queueing strategy: weighted fair
  Output queue: 0/1000/64/174 (size/max total/threshold/drops)
     Conversations  0/35/256 (active/max active/max total)
     Reserved Conversations 0/0 (allocated/max allocated)
     Available Bandwidth 1158 kilobits/sec
  5 minute input rate 7000 bits/sec, 11 packets/sec
  5 minute output rate 35000 bits/sec, 11 packets/sec
     7386235 packets input, 3684272716 bytes, 0 no buffer
     Received 17203 broadcasts, 0 runts, 0 giants, 0 throttles
     289 input errors, 289 CRC, 169 frame, 48 overrun, 0 ignored, 142 abort
     5390992 packets output, 1180985313 bytes, 0 underruns
     0 output errors, 0 collisions, 4 interface resets
     0 output buffer failures, 0 output buffers swapped out
     1 carrier transitions
     DCD=up  DSR=up  DTR=up  RTS=up  CTS=up



0
 

Author Comment

by:stasila2010
Comment Utility
Hi Jesper,

I realized its always happening on a same time. 12:00 AM
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
Do you have a reload scheduled on the router at that time every day?
0
 

Author Comment

by:stasila2010
Comment Utility
no I didn't scheduled any jobs. It always happens at 12:00AM. port 25 stop responding but everything else is working OK.

sh kron schedule, returns nothing.

Is there is any way we can check what happens on router at 12AM?
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
Is the barracuda performing any firmware or AV updates or does it do a restart at that time?
0
 

Author Comment

by:stasila2010
Comment Utility
No, nothing that I'm aware of. do you think Barracuda is the source of this problem? not a Cisco router?
0
 

Author Comment

by:stasila2010
Comment Utility
I can replicate this problem if I unplug network cable from Barracuda box and plug it back. port 25 from outside stop's responding right the way.
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
The only reason that the Cisco would stop accepting traffic at a specified time is if there were a reload event scheduled on your router or your direct upstream (something to which I advise against) or if your interface is bouncing.

To determine if the problem is with the Barracuda or the Cisco, you really do need to clear both ACLs 198 and 199, send a test message and *immediately* show both access-lists.  

I think that it would help to go into the Barracuda administrative interface and check to see what time updates are scheduled.  I'm also curious if, when sending a test message, the message remains in the Barracuda outbound queue during the failure.  

How long does the outage last?  Is the time period consistent?
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
I missed your last statement.  That goes back to the Barracuda ethernet card.  Is it plugged directly into the router or does it go into a switch?  I've seen Barracuda ethernet cards go back -- sometimes multiple times.

It's the bit about it always being at 12:00am that's puzzling unless the Barracuda has a configuration item that resets the ethernet interface and which does not bring the interface back up.
0
 

Author Comment

by:stasila2010
Comment Utility
I did clear access-list counters for both 198and 199 lists, sent test and show both access-lists

this is before I clear them
sh access-lists 199
Extended IP access list 199
    10 permit tcp 10.1.1.0 0.0.0.255 any eq smtp log (163450 matches)
    20 permit ip any any (5360543 matches)

Extended IP access list 198
    10 permit tcp 10.1.1.0 0.0.0.255 any eq smtp log
    20 permit ip any any (4642914 matches)

and this is after I clear counters and sent a test message.

Extended IP access list 199
    10 permit tcp 10.1.1.0 0.0.0.255 any eq smtp log
    20 permit ip any any (2299 matches)

Extended IP access list 198
    10 permit tcp 10.1.1.0 0.0.0.255 any eq smtp log
    20 permit ip any any (2477 matches)


here is a schedules in Barracuda

Spam Definition Updates : hourly
Virus Definition Updates: hourly
Security Definition Updates: hourly


outage last till I reboot the Cisco router.

Is there is any debugging we can turn on and see in more detail whats happening?
thank you.







0
 

Author Comment

by:stasila2010
Comment Utility
Barracuda plugs into switch and then to Cisco router. Yes, it happens at same time at least for past 4 times that I have noticed and as I said before it very strange be course Barracuda are still responding to port 25 internally so I don't think network card is the issue. It looks like something happening on Barracuda at 12:00AM and then NAT rules that forward port 25 to Barracuda stops working on Cisco router.
0
 

Author Comment

by:stasila2010
Comment Utility
I'm thinking is this somehow can case this issue?

ip nat inside source list 98 interface Loopback2 overload
ip nat inside source list 99 interface Loopback1 overload


access-list 98 permit 10.1.1.15
access-list 98 permit 10.1.1.40
access-list 99 deny   10.1.1.15
access-list 99 deny   10.1.1.40
access-list 99 permit 10.1.1.0 0.0.0.255


Barracuda box is going outside using a different loopback interface then all my local network.
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
So this is a single ethernet unit?

What are the translation timeouts for your NAT rules?
0
 

Author Comment

by:stasila2010
Comment Utility
Yes. its single unit.

#sh ip nat statistics
Total active translations: 227 (55 static, 172 dynamic; 227 extended)
Outside interfaces:
  Serial0/0/0, Loopback1, Loopback2
Inside interfaces:
  FastEthernet0/0
Hits: 812444  Misses: 10137
CEF Translated packets: 805037, CEF Punted packets: 27140
Expired translations: 12045
Dynamic mappings:
-- Inside Source
[Id: 1] access-list 98 interface Loopback2 refcount 27
[Id: 2] access-list 99 interface Loopback1 refcount 139
Queued Packets: 0

I'm not sure if there is any translation timeouts set on my NAT rules.

0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
I'm guessing not.  And they wouldn't pertain to static entries anyway.

And access-lists either work or they don't.

This failure at exactly the same time sounds like a scheduled job (sorry to be so persistent).

To summarize:

1) Mail destined to the exchange server sometimes stops.
2) Mail destined to the exchange server is: Internet -> Cisco -> Barracuda -> Exchange

What IP address do you have for mail deliver in the Barracuda?  The exchange's public IP or the private IP?
0
 

Author Comment

by:stasila2010
Comment Utility
1) I would say mail destined to Barracuda server sometimes stops.
2)that's correct.

Private IP of my Exchange serer.
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
We're back to determining whether the router or the Barracuda is the problem.

The easy way to test when it goes down is to bypass the Barracuda:

telnet atlmail3.turner.com 25

You should see:
Connected to atlmail3.turner.com (64.236.240.74).
Escape character is '^]'.
220 atlmail3 ESMTP

Type "quit" with no quotes to disconnect.
0
 

Author Comment

by:stasila2010
Comment Utility
I'm a bit confused, should I telnet from my router? then how can I by pass Barracuda?
0
 

Author Comment

by:stasila2010
Comment Utility
next time when it happens I will use troubleshooting from Barracuda, telnet/tcpdump/traceroute this will give us some information.
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
The next time you experience an SMTP failure, telnet atlmail3.turner.com 25, from a machine behind the firewall.
0
 

Author Comment

by:stasila2010
Comment Utility
tried to telnet from barracuda box yesterday without any luck.
0
 

Author Comment

by:stasila2010
Comment Utility
Hi jesper,

last weekend it happened  more then 3 times in a same day. is there any other way we can test it to find what casing this problem?
thanks,

0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
One more time (for my clarification):

Is the problem with mail have to do with mail leaving the Cisco router or coming into the Cisco router?

Is it still happening at 00:00 hours every time consistently?
0
 

Author Comment

by:stasila2010
Comment Utility
both, when it happens port 25 stop responding and no mail coming into Cisco router and leaving Cisco router. No, it happening at random time.
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
If all other traffic works except inbound and outbound mail *and* all traffic flows through the Barracuda, I'm inclined to believe the problem is with the Barracuda.

If all other traffic works and mail fails, from a machine behind the router but *not* from the Barracuda, try the telnet to port 25 -> telnet atlmail3.turner.com 25

I cannot see where the router would fail to pass port 25 traffic in and out but pass all other traffic.
0
Windows Server 2016: All you need to know

Learn about Hyper-V features that increase functionality and usability of Microsoft Windows Server 2016. Also, throughout this eBook, you’ll find some basic PowerShell examples that will help you leverage the scripts in your environments!

 

Author Comment

by:stasila2010
Comment Utility
hello jesper,

I talk to Barracuda support and they of course  said that all the logs are OK and nothing wrong with Barracuda itself.
any advice would be appreciated.
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
i hate this new formatting box.  answer is attached via code snippet.

Let's reverse the access list (remove 198 and 199 from the interfaces):
 

--------------------

config t
 

no access-list 198

access-list 198 permit tcp any any eq 25 log

access-list 198 permit ip any any
 

int f0/0

 ip access-group 198 out

 end

-------------------
 

We need to test email coming from the outside in.
 

1) Problem occurs

2) clear access-list count 198

3) Send an email to your domain from some outside email address

4) On the router -> show access-list 198

   save the results

5) sh ip nat trans | i 10.1.1.15

   save the results

6) disconnect and reconnect the ethernet cable to get traffic flowing

again
 

I'm not saying that the Barracuda is definitely the problem -- just that

I've seen enough problems with Barracuda ethernet interfaces to give me

pause.  Which Barracuda model number is this and how many email messages

either per day or per hour on average do you receive?

Open in new window

0
 

Author Comment

by:stasila2010
Comment Utility
thanks for reply.  I'm using 300 model with average of 500-700 emails per day. I will post updates when it happens again.

0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
That's the smaller model but it can easily handle that number of messages.
0
 

Author Comment

by:stasila2010
Comment Utility
access-list 198 after

Internet#sh access-lists 198
Extended IP access list 198
    10 permit tcp any any eq smtp log (763 matches)
    20 permit ip any any (1408 matches)


sh ip nat translation | i 10.1.1.15


Internet#sh ip nat translations | i 10.1.1.15
tcp ---                   ---                   206.xx.x.2:25       10.1.1.15:25
tcp ---                   ---                   206.xx.x.2:29       10.1.1.15:29
tcp 206.xx.x.2:25       10.1.1.15:25          68.119.107.176:1431   68.119.107.176:1431
tcp 206.xx.x.2:25       10.1.1.15:25          68.119.107.176:1846   68.119.107.176:1846
tcp 206.xx.x.2:25       10.1.1.15:25          91.77.194.72:1622     91.77.194.72:1622
tcp 206.xx.x.2:25       10.1.1.15:25          91.77.194.72:3018     91.77.194.72:3018
tcp 206.xx.x.2:25       10.1.1.15:25          91.77.194.72:4268     91.77.194.72:4268
tcp 206.xx.x.2:25       10.1.1.15:25          91.77.194.72:4698     91.77.194.72:4698
tcp 206.xx.x.2:25       10.1.1.15:25          189.6.175.22:59452    189.6.175.22:59452
tcp 206.xx.x.2:25       10.1.1.15:25          190.139.107.26:2587   190.139.107.26:2587
tcp 206.xx.x.2:25       10.1.1.15:25          201.76.132.36:9524    201.76.132.36:9524
tcp 206.xx.x.2:25       10.1.1.15:25          209.85.198.245:35108  209.85.198.245:35108
tcp 206.xx.x.2:25       10.1.1.15:25          ---                   ---
tcp 206.xx.x.2:29       10.1.1.15:29          ---                   ---
tcp 206.xx.x.2:37645    10.1.1.15:37645       209.62.72.173:25      209.62.72.173:25
tcp 206.xx.x.2:37839    10.1.1.15:37839       209.62.72.173:25      209.62.72.173:25
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
If these occurred while mail stopped, it suggests that the Cisco device is working properly.

If you can do without your Barracuda for 24 or 48 hours, I'd be curious to see if the problem manifests itself again.
0
 

Author Comment

by:stasila2010
Comment Utility
When it happened I reboot barracuda but it didn't resolve this problem. It seems that rebooting router only helps.
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
And if you disconnect and reconnect the ethernet cable on the Cisco -- does that work, as well?
0
 

Author Comment

by:stasila2010
Comment Utility
No it doesn't. Only rebooting Cisco router fixing this problem.
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
One more question:

If mail inbound flows through the Barracuda prior to hitting the exchance server, why do you have a static NAT rule for the Exchange server port 25?
0
 

Author Comment

by:stasila2010
Comment Utility
our 2 external users using POP3/SMTP to connecting straight to Exchange server by passing Barracuda box.
0
 

Author Comment

by:stasila2010
Comment Utility
should I get another router for testing purpose and make a NAT rule for Barracuda box on it?
0
 

Author Comment

by:stasila2010
Comment Utility
it happend again at exactly same time as a last time 12:09AM, here is the output from access-list 198

2008-07-16 00:12:08      Local7.Info      10.1.1.18      7417: *Jul 16 04:05:16.968: %SEC-6-IPACCESSLOGP: list 198 permitted tcp 10.1.1.15(35384) -> 62.232.75.226(25), 2 packets

2008-07-16 00:12:08      Local7.Info      10.1.1.18      7418: *Jul 16 04:05:16.968: %SEC-6-IPACCESSLOGP: list 198 permitted tcp 10.1.1.15(35383) -> 206.186.26.2(25), 1 packet

2008-07-16 00:12:09      Local7.Info      10.1.1.18      7419: *Jul 16 04:05:16.968: %SEC-6-IPACCESSLOGP: list 198 permitted tcp 10.1.1.15(35378) -> 72.50.22.144(25), 2 packets

2008-07-16 00:12:09      Local7.Info      10.1.1.18      7420: *Jul 16 04:05:16.968: %SEC-6-IPACCESSLOGP: list 198 permitted tcp 10.1.1.15(35379) -> 62.232.75.226(25), 3 packets

2008-07-16 00:12:09      Local7.Info      10.1.1.18      7421: *Jul 16 04:05:16.972: %SEC-6-IPACCESSLOGP: list 198 permitted tcp 10.1.1.15(35382) -> 206.186.26.2(25), 1 packet

2008-07-16 05:30:44      Local7.Info      10.1.1.18      7422: *Jul 16 09:23:53.870: %SEC-6-IPACCESSLOGP: list 198 permitted tcp 10.1.1.40(55261) -> 0.1.1.11(25), 1 packet

reload the router and everything starts working again.

2008-07-16 07:43:25      Local7.Notice      10.1.1.18      7423: *Jul 16 11:36:33.866: %SYS-5-RELOAD: Reload requested by admin on vty3 (10.1.1.x). Reload Reason: Reload Command.
2008-07-16 07:44:58      Local7.Info      10.1.1.18      1: *Jul 16 11:37:34.391: %VPN_HW-6-INFO_LOC: Crypto engine: onboard 0  State changed to: Initialized
2008-07-16 07:44:58      Local7.Info      10.1.1.18      2: *Jul 16 11:37:34.391: %VPN_HW-6-INFO_LOC: Crypto engine: onboard 0  State changed to: Enabled
2008-07-16 07:44:58      Local7.Error      10.1.1.18      3: *Jul 16 11:37:35.075: %LINK-3-UPDOWN: Interface Serial0/0/0, changed state to down
2008-07-16 07:44:58      Local7.Notice      10.1.1.18      4: *Jul 16 11:37:36.075: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0/0/0, changed state to down
2008-07-16 07:44:58      Local7.Notice      10.1.1.18      5: *Jul 16 11:37:37.043: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/0, changed state to up
2008-07-16 07:44:58      Local7.Notice      10.1.1.18      6: *Jul 16 11:37:38.395: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback1, changed state to up
2008-07-16 07:44:58      Local7.Notice      10.1.1.18      7: *Jul 16 11:37:38.435: %LINEPROTO-5-UPDOWN: Line protocol on Interface NVI0, changed state to up
2008-07-16 07:44:58      Local7.Notice      10.1.1.18      8: *Jul 16 11:37:38.519: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
2008-07-16 07:44:58      Local7.Notice      10.1.1.18      9: *Jul 16 11:37:52.615: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback2, changed state to up
2008-07-16 07:44:58      Local7.Notice      10.1.1.18      10: *Jul 16 11:37:52.615: %LINEPROTO-5-UPDOWN: Line protocol on Interface Virtual-Template1, changed state to down
2008-07-16 07:44:58      Local7.Notice      10.1.1.18      11: *Jul 16 11:37:52.667: %LINK-5-CHANGED: Interface FastEthernet0/1, changed state to administratively down
2008-07-16 07:44:58      Local7.Notice      10.1.1.18      12: *Jul 16 11:37:53.059: %SYS-5-CONFIG_I: Configured from memory by console
2008-07-16 07:44:58      Local7.Error      10.1.1.18      13: *Jul 16 11:37:53.411: %LINK-3-UPDOWN: Interface Serial0/0/0, changed state to up
2008-07-16 07:44:58      Local7.Notice      10.1.1.18      14: *Jul 16 11:37:53.567: %SYS-5-RESTART: System restarted --
2008-07-16 07:44:58      Local7.Notice      10.1.1.18      15: Cisco IOS Software, 1841 Software (C1841-ADVSECURITYK9-M), Version 12.4(11)T, RELEASE SOFTWARE (fc2)
2008-07-16 07:44:58      Local7.Notice      10.1.1.18      16: Technical Support: http://www.cisco.com/techsupport
2008-07-16 07:44:58      Local7.Notice      10.1.1.18      17: Copyright (c) 1986-2006 by Cisco Systems, Inc.
2008-07-16 07:44:58      Local7.Notice      10.1.1.18      18: Compiled Sat 18-Nov-06 15:46 by prod_rel_team
2008-07-16 07:44:58      Local7.Notice      10.1.1.18      19: *Jul 16 11:37:53.571: %SNMP-5-COLDSTART: SNMP agent on host Internet is undergoing a cold start
2008-07-16 07:44:58      Local7.Notice      10.1.1.18      20: *Jul 16 11:37:53.591: %SSH-5-ENABLED: SSH 1.99 has been enabled
2008-07-16 07:44:59      Local7.Info      10.1.1.18      21: *Jul 16 11:37:53.743: %SYS-6-BOOTTIME: Time taken to reboot after reload =   91 seconds
2008-07-16 07:44:59      Local7.Info      10.1.1.18      22: *Jul 16 11:37:54.019: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is OFF
2008-07-16 07:44:59      Local7.Info      10.1.1.18      23: *Jul 16 11:37:54.019: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is OFF
2008-07-16 07:44:59      Local7.Info      10.1.1.18      24: *Jul 16 11:37:54.019: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is OFF
2008-07-16 07:44:59      Local7.Info      10.1.1.18      25: *Jul 16 11:37:54.019: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is ON
2008-07-16 07:44:59      Local7.Notice      10.1.1.18      26: *Jul 16 11:37:54.643: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0/0/0, changed state to up
2008-07-16 07:45:00      Local7.Info      10.1.1.18      27: *Jul 16 11:37:55.523: %PKI-6-AUTOSAVE: Running configuration saved to NVRAM
2008-07-16 07:45:00      Local7.Info      10.1.1.18      28: *Jul 16 11:37:57.959: %SYS-6-LOGGINGHOST_STARTSTOP: Logging to host 10.1.1.x Port 514 started - CLI initiated
2008-07-16 07:45:42      Local7.Notice      10.1.1.18      29: *Jul 16 11:38:39.447: %LINEPROTO-5-UPDOWN: Line protocol on Interface Virtual-Access2, changed state to up
2008-07-16 07:46:47      Local7.Info      10.1.1.18      30: *Jul 16 11:39:44.143: %SEC-6-IPACCESSLOGP: list 198 permitted tcp 10.1.1.15(36547) -> 212.27.48.6(25), 1 packet
2008-07-16 07:46:50      Local7.Info      10.1.1.18      31: *Jul 16 11:39:47.907: %SEC-6-IPACCESSLOGP: list 198 permitted tcp 10.1.1.15(36560) -> 217.196.160.4(25), 1 packet
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
The router shows packets coming from the Barracuda.  Now let's look at packets coming from the router and going to the Barracuda

no access-list 199
access-list 199 permit tcp any 10.1.10 0.0.0.255 eq smtp log
access-list 199 permit ip any any

int f0/0
description LAN
ip access-group 199 out

When the problem occurs: clear access-list counter 199 and post the log data.

We still need to determine whether it's a bug in the IOS, the ethernet port on the router or the ethernet card on the Barracuda.

If you have an identical unit (same IOS), a swap of the router would be my first choice.

When I'm working in a critical environment and I think the problem may be hardware and I cannot determine within a very reasonable short period of time what is the problem, I start swapping hardware out.
0
 

Author Comment

by:stasila2010
Comment Utility
I was replicating this problem today. By unplugging network cable from Barracuda for 5 minutes and plug in back port 25 stop responding on Cisco router.
0
 

Author Comment

by:stasila2010
Comment Utility
this is a debugging

07-18-2008      09:20:05      Local7.Error      10.1.1.18      34: *Jul 18 13:13:00.963: %ALIGN-3-CORRECT: Alignment correction made at 0x602932D0  reading 0xE74EBD4E

07-18-2008      09:20:05      Local7.Error      10.1.1.18      35: *Jul 18 13:13:00.963: %ALIGN-3-TRACE: -Traceback= 0x602932D0 0x60296474 0x6003C21C 0x6002A12C 0x62190B94 0x62197FC0 0x621947A0 0x621949F8


07-18-2008      09:20:05      Local7.Error      10.1.1.18      37: *Jul 18 13:13:00.963: %ALIGN-3-TRACE: -Traceback= 0x602932D0 0x60295EA0 0x6003C21C 0x6002A12C 0x62190B94 0x62197FC0 0x621947A0 0x621949F8
07-18-2008      09:20:05      Local7.Error      10.1.1.18      36: *Jul 18 13:13:00.963: %ALIGN-3-CORRECT: Alignment correction made at 0x602932D0  reading 0xE7549ACE
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
It sounds like IOS bug.

If you can do a "show alignment" and "show stack", I see if I can find out whether this is a bug and if it has been fixed.
0
 

Author Comment

by:stasila2010
Comment Utility
#show alignment

No alignment data has been recorded.

No spurious memory references have been recorded.

Internet#show stack
Minimum process stacks:
 Free/Size   Name
 5336/6000   Inspect Init Msg
 5328/6000   SPAN Subsystem
 2272/3000   config_verify
 5144/6000   DIB error message
 5332/6000   SASL MAIN
 5472/12000  Init
 5336/6000   vidb clone Process
 5140/6000   RADIUS INITCONFIG
 2220/3000   Rom Random Update Process

Interrupt level stacks:
Level    Called Unused/Size  Name
  1       64932   5796/9000  Network interfaces
  2        4644   8572/9000  DMA/Timer Interrupt
  3           0   9000/9000  PA Management Int Handler
  4        1568   8556/9000  Console MPSC
  5           0   9000/9000  External Interrupt
  7      192433   8572/9000  NMI Interrupt Handler




thanks
0
 

Author Comment

by:stasila2010
Comment Utility
Internet#show alignment
Alignment data for:
1841 Software (C1841-ADVSECURITYK9-M), Version 12.4(11)T, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Compiled Sat 18-Nov-06 15:46 by prod_rel_team

Total Corrections 10, Recorded 4, Reads 10, Writes 0

 Initial         Initial
 Address  Count  Access Type   Traceback
E7546D4E      1  32bit read    0x602932D0  0x60296474  0x6003C21C  0x6002A12C
                               0x62190B94  0x62197FC0  0x621947A0  0x62194A68
E74E8FCE      6  32bit read    0x602932D0  0x60295EA0  0x6003C21C  0x6002A12C
                               0x62190B94  0x62197FC0  0x621947A0  0x621949F8
E74E484E      2  32bit read    0x602932D0  0x60295EA0  0x6003C21C  0x6002A12C
                               0x62190B94  0x62197FC0  0x621947A0  0x62194A68
E754394E      1  32bit read    0x602932D0  0x60296474  0x6003C21C  0x6002A12C
                               0x62190B94  0x62197FC0  0x621947A0  0x621949F8
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
The only data that I could search with was the log data.  Here are the results:

"%ALIGN-3-CORRECT (x0): Alignment correction made at [hex] [chars]ing
[hex]

Explanation: An alignment error has been detected and corrected.  Alignment errors
are due to  misaligned reads and writes caused by a software failure. Correcting
alignment errors consumes  processor resources and may result in a performance
penalty. If there are recurring alignment errors,  CPU utilization may be seriously
affected.

Recommended Action: To take advantage of recent fixes, upgrade your system to
the latest Cisco  IOS  software release in your release train. If this message
recurs, enter the show log, show alignment  and show tech-support commands, contact
your Cisco technical support representative, and  provide the representative with
the gathered information."

This is pretty generic since there are no alignment data and no spurios memory records.

My suggestion:  first try another router (same model, same IOS).  See if you can duplicate the problem.  If so, it's software.  If not, it's hardware (memory?).
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
Your alignment data didn't post when I checked it.  Hold on while I verify the alignment data you just posted.
0
 

Author Comment

by:stasila2010
Comment Utility
Thanks for reply. I unfortunately I can't try different router with same IOS. I will have to download and try latest IOS and see if it resolves my problem.

0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
Upgrade your IOS (and hopefully this is fixed):

c1841-advsecurityk9-mz.124-19b.bin
0
 

Author Comment

by:stasila2010
Comment Utility
I did upgraded IOS but problem still exist. I found this link and it seems to be the same problem as mine.

http://www.experts-exchange.com/Hardware/Networking_Hardware/Routers/Q_23374992.html

0
 

Author Comment

by:stasila2010
Comment Utility
Hi Jesper,

I'm getting this when I do "debug ip nat translation"
5 03:56:14.316: NAT: expiring 206.186.26.2 (10.1.1.15) tcp 25 (25)

any idea how to resolve this?
thanks in advance
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
It shouldn't be expiring.

Can you post the timeouts for all of the services?
0
 

Author Comment

by:stasila2010
Comment Utility
Jesper,

I think I found the problem but it need's more troubleshooting. when the problem occurs I realize that I can't ping barracuda from my cisco router.

i have to "clear arp-cache" and everything starts working again.


0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
I wonder if something is arping for that internal IP when it shouldn't be.

Try hardcoding the private IP of the mail server to the MAC address in the config.
0
 

Author Comment

by:stasila2010
Comment Utility
Can you please help me with this? I'm not sure how to do this in my config.

thanks
0
 

Author Comment

by:stasila2010
Comment Utility
is this what I need

mac-address-table secure MAC_ADDRESS_ BARRACUDA IP_ADDRESS_BARRACUDA?

0
 

Author Comment

by:stasila2010
Comment Utility
is this what I need

mac-address-table secure MAC_ADDRESS_ BARRACUDA IP_ADDRESS_BARRACUDA?

0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
It should be something similar to:

arp f0/0 10.1.1.15 <mac address>

Where <mac address> is in the format shown in "sho arp"  for that IP -- abcd.1234.ffff
0
 

Author Comment

by:stasila2010
Comment Utility
it doesn't take this command.

I can only do this:

-Internet(config)#arp 10.1.1.15 001d.9240.11d0 ?
  arpa   ARP type ARPA
  sap    ARP type SAP (HP's ARP type)
  smds   ARP type SMDS
  snap   ARP type SNAP (FDDI and TokenRing)
  srp-a  ARP type SRP (side A)
  srp-b  ARP type SRP (side B)


what should I choose here?
0
 

Author Comment

by:stasila2010
Comment Utility
It looks like my cisco model or IOS version does not support this command:
arp f0/0 10.1.1.15 <mac address>
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
No that's perfect.  The last option is "arpa".
0
 

Author Comment

by:stasila2010
Comment Utility
thanks a lot for your help. I will have to wait and see if it will happens again.
0
 

Author Comment

by:stasila2010
Comment Utility
jesper,

its just happened again. can't ping barracuda from router until I clear arp-cache. very strange
0
 

Author Comment

by:stasila2010
Comment Utility
this is how my arp looks right now

Internet  10.1.1.11               0   00b0.d0e1.ad37  ARPA   FastEthernet0/0
Internet  10.1.1.15               -   001d.9240.11d0  ARPA
Internet  10.1.1.17               0   001b.d4c5.1eed  ARPA   FastEthernet0/0
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
k.  when it happens again, the first thing that we want is a picture of the arp cache before it's cleared.
0
 

Author Comment

by:stasila2010
Comment Utility
if I ping my broadcast address 10.1.1.255 it returns with my barracuda IP. is this normal?

Pinging 10.1.1.255 with 32 bytes of data:

Reply from 10.1.1.15: bytes=32 time<1ms TTL=64
Reply from 10.1.1.15: bytes=32 time<1ms TTL=64
Reply from 10.1.1.15: bytes=32 time<1ms TTL=64
Reply from 10.1.1.15: bytes=32 time<1ms TTL=64
Reply from 10.1.1.15: bytes=32 time<1ms TTL=64
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
Odd.  Let's wait until it times out again and get the arp cache.
0
 

Author Comment

by:stasila2010
Comment Utility
jesper,

I attached 3 files with arp cache.
1) before it happend
2)after it happend
3)after I clear arp-cache

thanks,
sh-arp-before-in-happend.txt
sh-arp-after-it-happend.txt
after-clear-arp-cache.txt
0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
Let's remove the static arp entry and do before and after snapshots of the IP translation table.

sh run | i 10.1.1.15
sh ip nat trans | i 10.1.1.15
0
 

Author Comment

by:stasila2010
Comment Utility
Ok, I removed static arp

sh run | i 10.1.1.15
ip nat inside source static tcp 10.1.1.15 25 206.186.26.2 25 extendable
access-list 98 permit 10.1.1.15
access-list 99 deny   10.1.1.15
access-list 99 deny   10.1.1.152



sh ip nat translations | i 10.1.1.15
tcp 206.186.26.2:25    10.1.1.15:25       58.8.239.193:1095  58.8.239.193:1095
tcp 206.186.26.2:25    10.1.1.15:25       78.97.214.234:5943 78.97.214.234:5943
tcp 206.186.26.2:25    10.1.1.15:25       88.69.0.51:4567    88.69.0.51:4567
tcp 206.186.26.2:25    10.1.1.15:25       190.188.66.210:53054 190.188.66.210:53054
tcp 206.186.26.2:25    10.1.1.15:25       211.247.38.142:49976 211.247.38.142:49976
tcp 206.186.26.2:25    10.1.1.15:25       ---                ---


0
 

Author Comment

by:stasila2010
Comment Utility
hi jesper,

I found a temporary solution be setting arp timeout 300. so it does clear arp cache for me every 5 minutes.


0
 
LVL 28

Expert Comment

by:Jan Springer
Comment Utility
It individually expires arp entries but I sure wish we could find out why this is necessary.
0
 

Author Comment

by:stasila2010
Comment Utility
Yes, me too. Can it be a bad port on the router? it very strange that its happening only between Barracuda and my router.
0
 
LVL 28

Accepted Solution

by:
Jan Springer earned 500 total points
Comment Utility
If it was the ethernet interface of the router, it seems to me that there would be a problem with other devices on the network.  

If it were a problem with a poor implementation of arp cache, it seems to me that other devices would periodically be affected.

I would still ask to have the ethernet card on the Barracuda replaced.  If I hadn't had a few experiences with failing NICs on Barracudas, that thought wouldn't have come to mind to begin with.
0
 

Author Closing Comment

by:stasila2010
Comment Utility
Thanks a lot for your time and assistance and patience.
0

Featured Post

Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

Join & Write a Comment

Exchange server is not supported in any cloud-hosted platform (other than Azure with Azure Premium Storage).
Follow this checklist to learn more about the 15 things you should never include in an email signature from personal quotes, animated gifs and out-of-date marketing content.
In this video we show how to create a Shared Mailbox in Exchange 2013. We show this process by using the Exchange Admin Center. Log into Exchange Admin Center.: First we need to log into the Exchange Admin Center. Navigate to the Recipients >> Sha…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now