URGENT: high severity problem with the PIX 525 at Osaka conference
Dear all,
Bellow you will find the description my colleagues sent. A memory increasing problem risk to provoke a general outage at the plenary (Client coference we run) that is running in Japan today.
- SEVERITY HIGH : PIX FW version 8.0.4 memory increasing problem
The upgrade of the IOS on the PIX from 6.3 to 8.0.4 has fixed the problem on the VPN Client which used IPSEC. However, this has introduced a new problem. The memory usage on the PIX is continuously increasing until the max. When it reaches the limit of the capacity, there are no Internet Access here.
It has happened around 10:00 this morning. We have decided to failover the Standby unit.
But, The standby unit has a copy of the memory content thus we were in the same situation.
Finally, We have shut down the Standby unit and start the Primary one.
After the start-up we had only 84MB used now it is increasing by 25MB every hours.
We suspect 2 processes on the PIX:
Dispatch Unit and Unicorn Admin Handler are 2 processes that used the biggest amount of memory and they are increasing.
Thanks in advance for anything you can do to help us to resolve this critical issue.
If you need more information, you can reach me at :
miguel.paton@etsi.org
Best regards
PIX-125MB-AFTER-2H.TXT
PIX-171MB-AFTER-6H.TXT
Bellow you will find the description my colleagues sent. A memory increasing problem risk to provoke a general outage at the plenary (Client coference we run) that is running in Japan today.
- SEVERITY HIGH : PIX FW version 8.0.4 memory increasing problem
The upgrade of the IOS on the PIX from 6.3 to 8.0.4 has fixed the problem on the VPN Client which used IPSEC. However, this has introduced a new problem. The memory usage on the PIX is continuously increasing until the max. When it reaches the limit of the capacity, there are no Internet Access here.
It has happened around 10:00 this morning. We have decided to failover the Standby unit.
But, The standby unit has a copy of the memory content thus we were in the same situation.
Finally, We have shut down the Standby unit and start the Primary one.
After the start-up we had only 84MB used now it is increasing by 25MB every hours.
We suspect 2 processes on the PIX:
Dispatch Unit and Unicorn Admin Handler are 2 processes that used the biggest amount of memory and they are increasing.
Thanks in advance for anything you can do to help us to resolve this critical issue.
If you need more information, you can reach me at :
miguel.paton@etsi.org
Best regards
PIX-125MB-AFTER-2H.TXT
PIX-171MB-AFTER-6H.TXT
ASKER
Hello,
Seams to me the origin of the memory leak is on a bug quite well known : CSCsj84640 : Memory leak on CRYPTO_malloc.
There is no available higher version than 8.0.4. The version bellow which doesn't have this bug is the the Version 7.1(2).
Before instruct my colleagues to downgrade to the this version I'd like to know if there is any other solution. The PIX is in production at this moment.
Best regards
Seams to me the origin of the memory leak is on a bug quite well known : CSCsj84640 : Memory leak on CRYPTO_malloc.
There is no available higher version than 8.0.4. The version bellow which doesn't have this bug is the the Version 7.1(2).
Before instruct my colleagues to downgrade to the this version I'd like to know if there is any other solution. The PIX is in production at this moment.
Best regards
Hard to believe someone has other options here - problem started clearly after upgrade, the 1st
solution that TAC would ask to try - upgrade/downgrade , the only question asked here is to what version.
I 've had one client with VPn clients problem - more than 5 connections were freezing the ASA 5510 on memory( it was FOS 7.0) , after a search on cisco.com I found 7.2(4) to be least buggy , without using 8.x that would for sure introduce new bugs.So It has already passed about 2 months - havent heard from this client yet.
solution that TAC would ask to try - upgrade/downgrade , the only question asked here is to what version.
I 've had one client with VPn clients problem - more than 5 connections were freezing the ASA 5510 on memory( it was FOS 7.0) , after a search on cisco.com I found 7.2(4) to be least buggy , without using 8.x that would for sure introduce new bugs.So It has already passed about 2 months - havent heard from this client yet.
Open a TAC case with Cisco. They may have an unreleased bug fix for you.
any progress on this subject/post?
i've wrote the other post you referenced to...
These last days we are having increased fallouts, i'm runnung os 8.0.(4) and adsm 6.1.(5)51 on our Pix 515E / 64MB. we have 1 site2site vpn and use the device for remote access for 5 / 10 employees..
Mostly it begins with the pix not being to able to accept incomming VPN connects and since yesterday it block all incomming connections for remote control when the memory is full. So i have to do hardware reboot. Earlier i rebooted the machine remotely by cli.
Since the PIX support is dead in june, should i even buy a ASA?? it uses the same software and probebly has the same errors...
Regards, Rick
i've wrote the other post you referenced to...
These last days we are having increased fallouts, i'm runnung os 8.0.(4) and adsm 6.1.(5)51 on our Pix 515E / 64MB. we have 1 site2site vpn and use the device for remote access for 5 / 10 employees..
Mostly it begins with the pix not being to able to accept incomming VPN connects and since yesterday it block all incomming connections for remote control when the memory is full. So i have to do hardware reboot. Earlier i rebooted the machine remotely by cli.
Since the PIX support is dead in june, should i even buy a ASA?? it uses the same software and probebly has the same errors...
Regards, Rick
>should i even buy a ASA??
Absolutely, without question. It does not use the same hardware and the software is not exactly the same. Better memory management on ASA. More development is going into the ASA because the PIX is dead.
You can also try upgrade to a bug fix release 8.0(4)16 or 8.0(4)23
Absolutely, without question. It does not use the same hardware and the software is not exactly the same. Better memory management on ASA. More development is going into the ASA because the PIX is dead.
You can also try upgrade to a bug fix release 8.0(4)16 or 8.0(4)23
haha, this is funny :-S
I've listed 4 OS versions, check out the bugs.. listed in each (and solved or not...)
taken from cisco release notes for the asa 5500 series
ASA 5500 series version 8.0.(3)
CSCsj84640 - Memory leak on CRYPTO_malloc (Open Caveats)
CSCso64944 - (doesn't excist)
CSCsj25896 - (doesn't excist)
ASA 5500 series version 8.0.(4)
CSCsj84640 - (doesn't excist? / not solved or open!)
CSCso64944 - ASA memory leak due to IPSEC (Open Caveats) (huh?? new name?)
CSCsj25896 - ASA may reload with traceback in Thread name: CTM Message Handler (Resolved Caveats)
(i don't know if i would apply a 5580 to a 5500 series, but ok.. just to show u)
ASA 5580 version 8.1.(1)
non of the above ceveats! (not open or closed)
ASA 5580 version 8.1.(2)
CSCsj25896 Crypto Accelerator Memory Leak (re-opened under a new name??)
Very strange all!
A company who does some IT projects for us suggested that we buy a Zyxcel firewall/router, 1/3 cost of a new 5510
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Here are some possible causes and resolutions for high memory utilization. It sounds like a memory leak and may have to contact Cisco TAC for an upgrade to the OS, or downgrade to 8.0(3)19 or something..
Event logging: Event logging can consume large amounts of memory. In order to resolve this issue, install and log all events to an external server, such as a syslog server.
Memory Leakage: A known issue in the security appliance software can lead to high memory consumption. In order to resolve this issue, upgrade the security appliance software.
Debugging Enabled: Debugging can consume large amounts of memory. In order to resolve this issue, disable debugging with the undebug all command.