Is Apache ports unstable or my CA Unicentre monitoring giving false alerts


I configured my Apache to listen on tcp 83, 446 & 86.

Our monitoring team uses CA Unicentre (awservices) to monitor the
Apache's ports & I often get the alerts below which always recover
within 2-5 minutes.

Our monitoring team asserts that there's nothing wrong with their
monitoring tool (CA) as it polls the port(s) for response once every
30 seconds for 10 times & only if 10 out of 10 polls fail to respond,
then only the alert (like the ones shown below) are sent out.


13/12/10 06:26 E5NKK1S-ABCDWEB8 ABCDWEB8 port iProject_Apache (83) port OK
13/12/10 06:23 E5NKK1S-ABCDWEB8 ABCDWEB8 port iProject_Apache (83) is not responding
13/12/10 06:26 E5NKK1S-ABCDWEB8 ABCDWEB8 port Micro Focus (tcp 446) port OK
13/12/10 06:23 E5NKK1S-ABCDWEB8 ABCDWEB8 port Micro Focus (tcp 446) is not responding

Question is how do I determine if it's my Apache that's not responding
or it's the CA tool not being configured well or not fine-tuned?  I'm sure
our network has no issue.

For some other Apach webservers (in another project), they subscribe
to Websitepulse to monitor the website's availability & they faced similar
issue too : often getting alert that the website is down & within 5 minutes
they will get another message that it has recovered.  No user complaints
though for such phenomenon, so I guess it's sort of false alert.

However, we need to cut down on the number of such alerts as they
also get sent via SMS to support guys' mobiles.

Just this morning I got the following alert followed by a
recovery message at 07:56:57
 
03/12/10 07:53 E5NKK1S-ABCDWEB8  ABCDWEB8 port iProject_Apache (83) is not responding
03/12/10 07:56 E5NKK1S-ABCDWEB8  ABCDWEB8 port iProject_Apache (83) port OK
 

I thought if extra pairs of eyes can have a look at the Apache
  Access logs below just to be sure I'm not missing anything:
 
So did my Apache stops responding or it's just the CA monitoring
tool?
 
172.18.30.3 - - [26/Dec/2010:07:52:50 +0800] "GET /" 302 226 "-" "-" "0" "177"
172.18.30.2 - - [26/Dec/2010:07:52:53 +0800] "GET /" 302 226 "-" "-" "0" "184"
172.18.30.3 - - [26/Dec/2010:07:52:55 +0800] "GET /" 302 226 "-" "-" "0" "175"
172.18.30.2 - - [26/Dec/2010:07:52:58 +0800] "GET /" 302 226 "-" "-" "0" "171"
228.74.172.12 - - [26/Dec/2010:07:52:58 +0800] "GET /arc/accntings/transport/warrant.html HTTP/1.1" 200 1643 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "65620"
228.74.172.12 - - [26/Dec/2010:07:52:58 +0800] "GET /arc/images/icons/wo_careerScheme.png HTTP/1.1" 200 13439 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "55621"
160.96.2.106 - - [26/Dec/2010:07:52:49 +0800] "GET /content/ibankng/bankng_websites/atozlistings/fsd/infohub/_jcr_content/ibankngPars/0025/file.res/ikc2_all.pdf HTTP/1.0" 200 5708469 "-" "sgms-prod-crawler (Enterprise; T2-RZFWVYN9FQSJA; shine_sgms@ncs.com.sg)" "10" "10072258"
172.18.30.3 - - [26/Dec/2010:07:53:00 +0800] "GET /" 302 226 "-" "-" "0" "206"
206.53.155.227 - - [26/Dec/2010:07:53:02 +0800] "GET /ibankng/rss/newsvideos.html HTTP/1.1" 302 113 "-" "Carpathia Server 1.0" "0" "179015"
172.18.30.2 - - [26/Dec/2010:07:53:03 +0800] "GET /" 302 226 "-" "-" "0" "194"
206.53.155.227 - - [26/Dec/2010:07:53:03 +0800] "GET /content/ibankng/rss/newsvideos.xml HTTP/1.1" 200 1709 "-" "Carpathia Server 1.0" "0" "138542"
172.18.30.3 - - [26/Dec/2010:07:53:05 +0800] "GET /" 302 226 "-" "-" "0" "179"
172.18.30.2 - - [26/Dec/2010:07:53:08 +0800] "GET /" 302 226 "-" "-" "0" "179"
160.96.2.106 - - [26/Dec/2010:07:52:46 +0800] "GET /content/ibankng/bankng_websites/topics/aarms09/news/news_letter/_jcr_content/ibankngPars/000191/file.res/AT5%20(PG2&3).pdf HTTP/1.0" 200 14594474 "-" "sgms-prod-crawler (Enterprise; T2-RZFWVYN9FQSJA; shine_sgms@ncs.com.sg)" "24" "24297319"
172.18.30.3 - - [26/Dec/2010:07:53:10 +0800] "GET /" 302 226 "-" "-" "0" "168"
172.18.30.2 - - [26/Dec/2010:07:53:13 +0800] "GET /" 302 226 "-" "-" "0" "181"
172.18.30.3 - - [26/Dec/2010:07:53:15 +0800] "GET /" 302 226 "-" "-" "0" "178"
172.18.30.2 - - [26/Dec/2010:07:53:18 +0800] "GET /" 302 226 "-" "-" "0" "172"
228.74.172.12 - - [26/Dec/2010:07:53:18 +0800] "GET /arc/accntings/bank-dayprocessing/banner.html HTTP/1.1" 200 440 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "69903"
228.74.172.12 - - [26/Dec/2010:07:53:18 +0800] "GET /css/accnting_layout_1.css HTTP/1.1" 404 5812 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "4295"
228.74.172.12 - - [26/Dec/2010:07:53:18 +0800] "GET /arc/accntings/bank-dayprocessing/overview.html HTTP/1.1" 200 1513 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "81333"
228.74.172.12 - - [26/Dec/2010:07:53:19 +0800] "GET /arc/images/accntings/badges/small_bankdayprocessing.png HTTP/1.1" 200 8286 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "33876"
228.74.172.12 - - [26/Dec/2010:07:53:19 +0800] "GET /arc/images/accntings/backgrounds/bank-dayprocessing/bg.png HTTP/1.1" 200 12655 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "39134"
228.74.172.12 - - [26/Dec/2010:07:53:19 +0800] "GET /arc/images/accntings/contents/bank-dayprocessing/img-01.jpg HTTP/1.1" 200 41546 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "689956"
228.74.172.12 - - [26/Dec/2010:07:53:19 +0800] "GET /arc/images/accntings/contents/bank-dayprocessing/img-02.jpg HTTP/1.1" 200 39732 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "749524"
228.74.172.12 - - [26/Dec/2010:07:53:19 +0800] "GET /arc/images/accntings/contents/bank-dayprocessing/img-03.jpg HTTP/1.1" 200 45458 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "747701"
228.74.172.12 - - [26/Dec/2010:07:53:19 +0800] "GET /arc/images/accntings/contents/bank-dayprocessing/img-04.jpg HTTP/1.1" 200 28718 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "763528"
228.74.172.12 - - [26/Dec/2010:07:53:20 +0800] "GET /arc/accntings/bank-clearnce-services/banner.html HTTP/1.1" 200 425 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "34679"
228.74.172.12 - - [26/Dec/2010:07:53:20 +0800] "GET /css/accnting_layout_1.css HTTP/1.1" 404 5812 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "3698"
228.74.172.12 - - [26/Dec/2010:07:53:20 +0800] "GET /arc/accntings/bank-clearnce-services/overview.html HTTP/1.1" 200 1777 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "48120"
228.74.172.12 - - [26/Dec/2010:07:53:20 +0800] "GET /arc/images/accntings/badges/small_bankclearnceExpert.png HTTP/1.1" 200 7054 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "76531"
228.74.172.12 - - [26/Dec/2010:07:53:20 +0800] "GET /arc/images/accntings/contents/bank-clearnce-services/img-02.jpg HTTP/1.1" 200 45468 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "291352"
172.18.30.3 - - [26/Dec/2010:07:53:20 +0800] "GET /" 302 226 "-" "-" "0" "176"
228.74.172.12 - - [26/Dec/2010:07:53:20 +0800] "GET /arc/images/accntings/contents/bank-clearnce-services/img-04.jpg HTTP/1.1" 200 49824 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "418785"
228.74.172.12 - - [26/Dec/2010:07:53:20 +0800] "GET /arc/images/accntings/backgrounds/bank-clearnce-services/bg.png HTTP/1.1" 200 21545 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "479381"
228.74.172.12 - - [26/Dec/2010:07:53:20 +0800] "GET /arc/images/accntings/contents/bank-clearnce-services/img-03.jpg HTTP/1.1" 200 47667 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "719484"
228.74.172.12 - - [26/Dec/2010:07:53:20 +0800] "GET /arc/images/accntings/contents/bank-clearnce-services/img-01.jpg HTTP/1.1" 200 72086 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "953354"
172.18.30.2 - - [26/Dec/2010:07:53:23 +0800] "GET /" 302 226 "-" "-" "0" "163"
228.74.172.12 - - [26/Dec/2010:07:53:24 +0800] "GET /arc/accntings/processing/banner.html HTTP/1.1" 200 415 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "38844"
228.74.172.12 - - [26/Dec/2010:07:53:24 +0800] "GET /css/accnting_layout_1.css HTTP/1.1" 404 5812 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "4961"
228.74.172.12 - - [26/Dec/2010:07:53:24 +0800] "GET /arc/accntings/processing/overview.html HTTP/1.1" 200 2148 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "54860"
76.13.123.157 - - [26/Dec/2010:07:53:24 +0800] "GET /ibankng/rss/pioneerfeatures.html HTTP/1.0" 302 118 "-" "Yahoo-Newscrawler/3.9 (news-search-crawler at yahoo-inc dot com)" "0" "180115"
228.74.172.12 - - [26/Dec/2010:07:53:24 +0800] "GET /arc/images/accntings/badges/small_processing.png HTTP/1.1" 200 4728 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "76141"
228.74.172.12 - - [26/Dec/2010:07:53:24 +0800] "GET /arc/images/accntings/backgrounds/processing/bg.png HTTP/1.1" 200 21415 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "70993"
160.96.2.106 - - [26/Dec/2010:07:52:59 +0800] "GET /content/dam/ibankng_media_library/pdf/hqmc/0011.res HTTP/1.0" 200 14768344 "-" "sgms-prod-crawler (Enterprise; T2-RZFWVYN9FQSJA; shine_sgms@ncs.com.sg)" "25" "25445563"
160.96.2.106 - - [26/Dec/2010:07:53:25 +0800] "GET /robots.txt HTTP/1.0" 200 243 "-" "sgms-prod-crawler (Enterprise; T2-RZFWVYN9FQSJA; shine_sgms@ncs.com.sg)" "0" "19893"
172.18.30.3 - - [26/Dec/2010:07:53:25 +0800] "GET /" 302 226 "-" "-" "0" "179"
76.13.123.157 - - [26/Dec/2010:07:53:26 +0800] "GET /content/ibankng/rss/pioneerfeatures.xml HTTP/1.0" 200 4231 "-" "Yahoo-Newscrawler/3.9 (news-search-crawler at yahoo-inc dot com)" "0" "134421"
228.74.172.12 - - [26/Dec/2010:07:53:26 +0800] "GET /arc/accntings/bank-engineers/banner.html HTTP/1.1" 200 437 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "37777"
228.74.172.12 - - [26/Dec/2010:07:53:26 +0800] "GET /css/accnting_layout_1.css HTTP/1.1" 404 5812 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "3820"
228.74.172.12 - - [26/Dec/2010:07:53:26 +0800] "GET /arc/accntings/bank-engineers/overview.html HTTP/1.1" 200 1668 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "44208"
228.74.172.12 - - [26/Dec/2010:07:53:27 +0800] "GET /arc/images/accntings/badges/small_bankEngineers.png HTTP/1.1" 200 9488 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "28136"
228.74.172.12 - - [26/Dec/2010:07:53:27 +0800] "GET /arc/images/accntings/contents/bank-engineers/img-03.jpg HTTP/1.1" 200 37007 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "146087"
228.74.172.12 - - [26/Dec/2010:07:53:27 +0800] "GET /arc/images/accntings/backgrounds/bank-engineers/bg.png HTTP/1.1" 200 24881 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "117233"
228.74.172.12 - - [26/Dec/2010:07:53:27 +0800] "GET /arc/images/accntings/contents/bank-engineers/img-04.jpg HTTP/1.1" 200 52619 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "563529"
228.74.172.12 - - [26/Dec/2010:07:53:27 +0800] "GET /arc/images/accntings/contents/bank-engineers/img-01.jpg HTTP/1.1" 200 48376 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "870758"
228.74.172.12 - - [26/Dec/2010:07:53:27 +0800] "GET /arc/images/accntings/contents/bank-engineers/img-02.jpg HTTP/1.1" 200 53881 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "967982"
172.18.30.2 - - [26/Dec/2010:07:53:28 +0800] "GET /" 302 226 "-" "-" "0" "154"
228.74.172.12 - - [26/Dec/2010:07:53:29 +0800] "GET /arc/accntings/backend-dayprocessing/banner.html HTTP/1.1" 200 424 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "42432"
228.74.172.12 - - [26/Dec/2010:07:53:29 +0800] "GET /css/accnting_layout_1.css HTTP/1.1" 404 5812 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "3639"
228.74.172.12 - - [26/Dec/2010:07:53:29 +0800] "GET /arc/accntings/backend-dayprocessing/overview.html HTTP/1.1" 200 1684 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "34929"
228.74.172.12 - - [26/Dec/2010:07:53:29 +0800] "GET /arc/images/accntings/badges/small_coventrydayprocessing.png HTTP/1.1" 200 16739 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "40161"
228.74.172.12 - - [26/Dec/2010:07:53:29 +0800] "GET /arc/images/icons/me_careerScheme.png HTTP/1.1" 200 12275 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "49504"
228.74.172.12 - - [26/Dec/2010:07:53:29 +0800] "GET /arc/images/accntings/contents/backend-dayprocessing/img-02.jpg HTTP/1.1" 200 39732 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "143208"
228.74.172.12 - - [26/Dec/2010:07:53:29 +0800] "GET /arc/images/accntings/contents/backend-dayprocessing/img-03.jpg HTTP/1.1" 200 45458 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "182175"
228.74.172.12 - - [26/Dec/2010:07:53:29 +0800] "GET /arc/images/accntings/contents/backend-dayprocessing/img-04.jpg HTTP/1.1" 200 28718 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "411444"
228.74.172.12 - - [26/Dec/2010:07:53:29 +0800] "GET /arc/images/accntings/contents/backend-dayprocessing/img-01.jpg HTTP/1.1" 200 41546 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "573612"
228.74.172.12 - - [26/Dec/2010:07:53:30 +0800] "GET /arc/images/accntings/backgrounds/backend-dayprocessing/bg.png HTTP/1.1" 200 24373 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "84365"
172.18.30.3 - - [26/Dec/2010:07:53:30 +0800] "GET /" 302 226 "-" "-" "0" "171"
172.18.30.2 - - [26/Dec/2010:07:53:33 +0800] "GET /" 302 226 "-" "-" "0" "167"
172.18.30.3 - - [26/Dec/2010:07:53:35 +0800] "GET /" 302 226 "-" "-" "0" "167"
172.18.30.2 - - [26/Dec/2010:07:53:38 +0800] "GET /" 302 226 "-" "-" "0" "179"
228.74.172.12 - - [26/Dec/2010:07:53:38 +0800] "GET /css/accnting_layout_1.css HTTP/1.1" 404 5812 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "3887"
160.96.2.106 - - [26/Dec/2010:07:53:10 +0800] "GET /content/ibankng/bankng_websites/topics/aarms09/news/news_letter/_jcr_content/ibankngPars/000171/file.res/AT4%20(PG%202&3).pdf HTTP/1.0" 200 16204570 "-" "sgms-prod-crawler (Enterprise; T2-RZFWVYN9FQSJA; shine_sgms@ncs.com.sg)" "28" "28318722"
228.74.172.12 - - [26/Dec/2010:07:53:40 +0800] "GET /css/accnting_layout_1.css HTTP/1.1" 404 5812 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "5821"
172.18.30.3 - - [26/Dec/2010:07:53:40 +0800] "GET /" 302 226 "-" "-" "0" "176"
228.74.172.12 - - [26/Dec/2010:07:53:42 +0800] "GET /css/accnting_layout_1.css HTTP/1.1" 404 5812 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "3984"
172.18.30.2 - - [26/Dec/2010:07:53:43 +0800] "GET /" 302 226 "-" "-" "0" "166"
228.74.172.12 - - [26/Dec/2010:07:53:44 +0800] "GET /css/accnting_layout_1.css HTTP/1.1" 404 5812 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "4026"
228.74.172.12 - - [26/Dec/2010:07:53:44 +0800] "GET /arc/accntings/tecnolgy/warrant.html HTTP/1.1" 200 1514 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "47745"
172.18.30.3 - - [26/Dec/2010:07:53:45 +0800] "GET /" 302 226 "-" "-" "0" "173"
172.18.30.2 - - [26/Dec/2010:07:53:48 +0800] "GET /" 302 226 "-" "-" "0" "173"
172.18.30.3 - - [26/Dec/2010:07:53:50 +0800] "GET /" 302 226 "-" "-" "0" "169"
160.96.2.106 - - [26/Dec/2010:07:53:25 +0800] "GET /content/ibankng/bankng_websites/topics/aarms09/news/news_letter/_jcr_content/ibankngPars/0001121/file.res/AT6%20(PG%202&3).pdf HTTP/1.0" 200 23010120 "-" "sgms-prod-crawler (Enterprise; T2-RZFWVYN9FQSJA; shine_sgms@ncs.com.sg)" "27" "27058254"
172.18.30.2 - - [26/Dec/2010:07:53:53 +0800] "GET /" 302 226 "-" "-" "0" "186"
228.74.172.12 - - [26/Dec/2010:07:53:55 +0800] "GET /arc/accntings/coventry/banner.html HTTP/1.1" 200 421 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "52680"
228.74.172.12 - - [26/Dec/2010:07:53:55 +0800] "GET /css/accnting_layout_1.css HTTP/1.1" 404 5812 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "4145"
228.74.172.12 - - [26/Dec/2010:07:53:55 +0800] "GET /arc/accntings/coventry/overview.html HTTP/1.1" 200 2280 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "73220"
228.74.172.12 - - [26/Dec/2010:07:53:55 +0800] "GET /arc/images/accntings/badges/small_coventry.png HTTP/1.1" 200 9954 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "80982"
172.18.30.3 - - [26/Dec/2010:07:53:55 +0800] "GET /" 302 226 "-" "-" "0" "191"
228.74.172.12 - - [26/Dec/2010:07:53:55 +0800] "GET /arc/images/accntings/contents/coventry/img-01.jpg HTTP/1.1" 200 50293 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "259328"
228.74.172.12 - - [26/Dec/2010:07:53:55 +0800] "GET /arc/images/accntings/contents/coventry/img-03.jpg HTTP/1.1" 200 32951 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "321749"
228.74.172.12 - - [26/Dec/2010:07:53:55 +0800] "GET /arc/images/accntings/contents/coventry/img-02.jpg HTTP/1.1" 200 46369 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "409386"
228.74.172.12 - - [26/Dec/2010:07:53:55 +0800] "GET /arc/images/accntings/contents/coventry/img-04.jpg HTTP/1.1" 200 61737 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "399356"
228.74.172.12 - - [26/Dec/2010:07:53:55 +0800] "GET /arc/images/accntings/backgrounds/coventry/bg.png HTTP/1.1" 200 21530 "http://www.bankng.com.au/arc/accntings.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)" "0" "417160"
172.18.30.2 - - [26/Dec/2010:07:53:58 +0800] "GET /" 302 226 "-" "-" "0" "190"
172.18.30.3 - - [26/Dec/2010:07:54:00 +0800] "GET /" 302 226 "-" "-" "0" "178"
172.18.30.2 - - [26/Dec/2010:07:54:03 +0800] "GET /" 302 226 "-" "-" "0" "138"
sunhuxAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

giltjrCommented:
What is the IP address of the CA box?

What page is it getting to test to make sure the ports are up?
sunhuxAuthor Commented:

>What is the IP address of the CA box?
For security reason, I can't reveal this but from the CA monitoring server,
I'm able to do "telnet web_server_internal_address port#".  How does it
help if I can provide the IP addr of the CA server?

>What page is it getting to test to make sure the ports are up?
It does not get a specific page as there are dozens of pages in the
webserver.  It has a certain way of testing if the webserver's Tcp
port is responding
giltjrCommented:
--> For security reason, I can't reveal this  ....

Well then you should not be posting your Apache log on the Internet.  From your log I can tell:

1) Two comptuer: 172.18.30.3 and 172.18.30.2, seem to want to get your default page ("GET /") and seem to get a HTTP 302 return code.  Which is page temporaryly moved.

2) The host 228.74.172.12 is getting something that appears to be part of a accounting system.

3) By doing a simple search on some of the information in your post I found you have also asked the same question on the site "http://www.spinics.net" and "http://osdir.com" or you have posted this question someplace else where it gets replcated to these two sites.

-->  How does it help if I can provide the IP addr of the CA server?

Because then when we look at the Apache log you posted we can tell what return code the CA server is getting.

-->It has a certain way of testing if the webserver's Tcp port is responding

Then you need to understand what this way is.  I will say that 99.9999999999999...999 % of servery HTTP test I have seen and encoutered invloved doing a HTTP GET.  If CA is doing something other than a HTTP GET, that could be your problem.

Yes you can telnet to a HTTP port, but that is because HTTP is simple text protocol, you still need to issue HTTP commands (GET/POST/HEAD, ect.).
Bootstrap 4: Exploring New Features

Learn how to use and navigate the new features included in Bootstrap 4, the most popular HTML, CSS, and JavaScript framework for developing responsive, mobile-first websites.

sunhuxAuthor Commented:

> your Apache log on the Internet.  From your log I can tell:
All addresses, directories etc are fictitious (ie I've altered them) but
you can work with those fictitious addresses.  Many other things
in the log was altered to anything even close to the actual thing, just
retaining the time & syntax


> look at the Apache log you posted we can tell what return code the CA server is getting
The webserver's IP address is not reflected (even the fictitious address) in the log

CA Unicentre is certainly not doing any GET because none of the  GETs in the Apache
log reflected the CA Unicentre monitoring server's IP address.  

I know nmap does not need to do a GET to test if a port is open
sunhuxAuthor Commented:

> 1) Two computers: 172.18.30.3 and 172.18.30.2, seem to want to get your default page
> ("GET /") and seem to get a HTTP 302 return code.  Which is page temporaryly moved.

Can we say that the "GET /" or "GET default" failed temporarily & this may cause the
CA monitoring to report a failure to respond on the port?  Certainly these 2 computers
are not the CA monitoring server
sunhuxAuthor Commented:

About 2-3 years ago, I had a few servers that do not have CA monitoring & I
wrote scripts (.cmd) to use nmap to test if a port (can be http 80 or tcp 443 or
Oracle tcp 1521) is open & if it's not open, then I'll email alerts out tt service is down.
I don't have such false alerts in those days.  So is doing "http GET" supposed to
be a more reliable way of monitoring if an Apache / web service is responding?
giltjrCommented:
If CA is not doing a GET, or any other HTTP command, then all it is checking is to see if the TCP port is open.  Which is all NMAP does, open a TCP connection and then close it.  This checks to make sure Apache is up, but it may not be fully operational.

Doing a TCP connection alone does not really test if the web server is fully working.  That is like testing a phone number by calling it and just waiting for ring and not waiting for somebody to answer.  Just because you get a ring, does not mean that the phone is actually working, or that there is somebody there to answer.

Doing a GET is a better way to test that Apache (or any other web server) is actually responding.

I don't know if the 302's could cause this type of problem.  It depends on what CA expects back.  I know I have some monitors setup that expect HTTP 200 return codes and so if they were get a 302, they would assume something is wrong and mark the server as being down.

One thing you could try is doing a packet capture on the Apache server and only capture packets that are to/from the CA servers IP address.  Then you can correlate the packet trace to when CA says the servers are down and see what is going on.

It is possible that it is taking longer than expected to complete the TCP connection and so CA assumes it is down.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
sunhuxAuthor Commented:

Our CA support said their CA tool uses a type of icmp ping to poll for the port (which I
find rather strange) : anyway, I requested for the polling to be increased from 5 secs
timeout to 30 secs
giltjrCommented:
Well that is CA for you, or more than likely that's the company whose software CA bought.

So the only way you will be able to tell what is going on is to run a packet capture on the Apache server. I would filter on the IP address of the CA box.

I personally would not (and do not) trust any product that just verifies that the port is open.

sunhuxAuthor Commented:

Ok, so on my Linux web server, I'll run " tcpdump | grep IP_addr_of_CA_box"
to see how it goes
giltjrCommented:
Actually want you will want to try is:

   nohup tcpdump -i eth0 -s 0 ip host x.x.x.x -w ca01.cap

This does:

1) Assumes that the traffic is coming on eth0, if it comes on another interface replace eth0 with that interfaces name
2) Captures the whole buffer (-s 0).
3) Only captures traffic where the IP address i x.x.x.x, put the IP address of the CA box here
4) Writes it out to the file ca01.cap

The nohup means that the capture will continue to run after you logoff of the box.  You can kill the pid later on, that way you don't have to say signed on.

You can then use wireshark (http://www.wireshark.com) to look at the ca01.cap file to see what is going on.
sunhuxAuthor Commented:
ok I'm running MS portqry from the CA box to the servers
& looks like the CA tool has been occasionally reporting false alerts but the CA monitoring team does not know
how to fix this
giltjrCommented:
It not really that abnormal to have false positives.  We get them every now and then.  Typically we end up tracing them back to a resource constrant someplace.  The sever doing the monitoring hit 100% CPU utlization for longer than normal, the server being monitored hit 100% CPU utilization for longer than norma, or some network path between the tester and the testee was congested for longer than normal.

Any of these situation can cause a delay in the sending and receiving of the test and thus be detected as a "out of service" situation.  If it happens a lot, then you need to dig in and see what resource is constrained.

It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Apache Web Server

From novice to tech pro — start learning today.