Solved

apache logFormat not working

Posted on 2014-11-25
17
230 Views
Last Modified: 2014-12-28
I'm sure I must be doing something wrong. In my httpd.conf I have:

<IfModule log_config_module>
    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
    LogFormat "%h %l %u %t \"%r\" %>s %b" common

#    CustomLog "logs/access_log" common
    CustomLog "logs/access_log" combined
</IfModule>

Open in new window


Yet I am getting log messages in logfiles/access_log according to the "common" format, not the "combined" format. Yes, I've restarted httpd.

I'm confused! Why is the "combined" format not working? Where is it getting the "common" format from? I'm also running Tomcat. Could that be overriding the format somehow?

ServerRoot "/usr/local/apache2"
Slackware 13.37.0, kernel 2.6.37.6
Apache 2.2.9
Tomcat 6.0.14
0
Comment
Question by:jmarkfoley
  • 9
  • 8
17 Comments
 
LVL 61

Expert Comment

by:gheist
ID: 40468745
Can you confirm with "apachectl -M" that log_config module is to be loaded?
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40474861
yes:
>/usr/local/apache2/bin/apachectl -M
Loaded Modules:
 core_module (static)
 authn_file_module (static)
 authn_default_module (static)
 authz_host_module (static)
 authz_groupfile_module (static)
 authz_user_module (static)
 authz_default_module (static)
 auth_basic_module (static)
 dumpio_module (static)
 include_module (static)
 filter_module (static)
 [b]log_config_module (static)[/b]
 env_module (static)
 unique_id_module (static)
 setenvif_module (static)
 ssl_module (static)
 mpm_prefork_module (static)
 http_module (static)
 mime_module (static)
 status_module (static)
 autoindex_module (static)
 asis_module (static)
 cgi_module (static)
 negotiation_module (static)
 dir_module (static)
 actions_module (static)
 speling_module (static)
 userdir_module (static)
 alias_module (static)
 rewrite_module (static)
 so_module (static)
 jk_module (shared)
Syntax OK

Open in new window

0
 
LVL 61

Accepted Solution

by:
gheist earned 500 total points
ID: 40475255
You should be using slackware-built httpd.
Maybe apachectl works with that and not in a manual build in /usr/local?

This is the latest patched version for your slackware:
ftp://ftp.slackware.com/pub/slackware/slackware-13.37/patches/packages/httpd-2.2.27-i486-1_slack13.37.txz
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40476954
I needd to do a manual built at the time because I needed some things not in the pre-built version -- can't remember what now (installed in 2008). apachectl is using /usr/local/apache2/httpd.conf. The init script has, e.g.
case "$1" in
  'start')
    /usr/local/apache2/bin/apachectl -k start
  ;;

Open in new window

and /usr/local/apache2/bin/apachectl is set to execute:
HTTPD='/usr/local/apache2/bin/httpd'

Open in new window

I have verified (/usr/local/apache2/bin/httpd -S) that /usr/local/apache2/conf/httpd.conf is being used. If I change other parameters in that file, things do change, just not the log file format!!

I'll experiment further, meanwhile, any other ideas? (This is not important enough to try re-installing apache, plus no assurance that will make any different after doing the work)
0
 
LVL 61

Expert Comment

by:gheist
ID: 40477477
20 releases later about 100 bugs are fixed, like 10 of them critical for security and relevant to your setup:
https://httpd.apache.org/security/vulnerabilities_22.html
If you havent got all the apache scripts installed manually in good places it could be viable to just install distribution httpd and merge config files (it provides threaded "worker" MPM which is like 100x more efficient than your prefork homebrew apache (some reservations apply)

Can you check if you dont redefine that LogFormat in multiple places?
1 can be in default configuration, then each virtualhost can re-define same names IF NEEDED (for all practical purposes that one should write own access log (CustomLog) to not pollute main access log with different format lines. Also dont repeat defaults. Probably define special log fromats for special statistics logs, leave at least on combined access log for future keeping.
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40479356
I've checked the various other configs. We have no vhosts. There is a CustomLog defined in httpd-ssl.conf, but it is specific to logs/ssl_request_log.

I guess the thing to do is upgrade apache and see if that fixes it. Meanwhile, I'll turn off tomcat and post back results (in case ...).

Seems strange though. The logging stuff goes way back in Apache and I know I've seen it work properly in even older versions.
0
 
LVL 61

Expert Comment

by:gheist
ID: 40480283
I can just confirm that LogFormat works on any version of apache I encountered in last 10 years. With log config module always being loaded as default.
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40482262
Well, it just started working when I stopped and started tomcat and apache for the nth time. I have no explanation. This time I stopped tomcat, waited 12 seconds and stopped httpd -- but it said "httpd: no process". I waited 10 seconds, then restarted httpd. I then restarted tomcat 3 seconds later, and I got messages WITH browser information (see lines below).

Don't know why I had "httpd: no process" when I tried to shutdown apache - I've checked and the tomcat shutdown script does not kill httpd. It's a mystery, but at least it's working now.
157.55.39.176 - - [04/Dec/2014:23:50:13 -0500] "GET /ohprs HTTP/1.1" 302 -
157.55.39.176 - - [04/Dec/2014:23:50:13 -0500] "GET /ohprs/ HTTP/1.1" 200 21830
157.55.39.176 - - [04/Dec/2014:23:50:15 -0500] "GET / HTTP/1.1" 301 235
(apache and tomcat stopped)
(apache and tomcat restarted)
123.125.68.51 - - [04/Dec/2014:23:58:05 -0500] "GET /ohprs/downloads/newsletters/September%202012.pdf HTTP/1.1" 301 278 "-" "Mozilla/5.0 (Windows NT 6.1; rv:15.0) Gecko/20100101 Firefox/15.0.1"
123.125.68.17 - - [04/Dec/2014:23:58:08 -0500] "GET /ohprs/downloads/newsletters/September%202012.pdf HTTP/1.1" 503 323
202.46.62.199 - - [05/Dec/2014:00:00:22 -0500] "GET /ohprs/downloads/forms/20%20Retirees/minutes.jsp?agendas=Audit%20Committee HTTP/1.1" 301 311 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.1.2) Firefox/3.5.2"

Open in new window

0
IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 1

Author Comment

by:jmarkfoley
ID: 40482267
Spoke too soon. The list below begins with the last message from the list above (line 8), and several lines following. You will notice at line 7 that it suddenly stops showing the client browser. I've done nothing but `tail -f` the access_log. Despite Heisenberg, I don't think my mere observation changed the outcome! What's up????? Now I'm really confused!

202.46.62.199 - - [05/Dec/2014:00:00:22 -0500] "GET /ohprs/downloads/forms/20%20Retirees/minutes.jsp?agendas=Audit%20Committee HTTP/1.1" 301 311 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.1.2) Firefox/3.5.2"
202.46.60.66 - - [05/Dec/2014:00:02:21 -0500] "GET / HTTP/1.1" 301 241 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)"
119.63.193.194 - - [05/Dec/2014:00:03:21 -0500] "GET / HTTP/1.1" 301 241 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)"
123.125.71.88 - - [05/Dec/2014:00:05:02 -0500] "GET / HTTP/1.1" 301 235 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
220.181.108.176 - - [05/Dec/2014:00:06:30 -0500] "GET / HTTP/1.1" 301 235 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
208.69.40.107 - - [05/Dec/2014:00:08:36 -0500] "GET /ohprs/minutes.jsp?minutes=Retirement%20Board HTTP/1.1" 200 7111
68.198.32.50 - - [05/Dec/2014:00:10:43 -0500] "GET / HTTP/1.1" 301 235
68.198.32.50 - - [05/Dec/2014:00:10:43 -0500] "GET /ohprs HTTP/1.1" 302 -
68.198.32.50 - - [05/Dec/2014:00:10:43 -0500] "GET /ohprs/ HTTP/1.1" 200 21830
68.198.32.50 - - [05/Dec/2014:00:10:44 -0500] "GET /ohprs/ohprs.css HTTP/1.1" 200 6556
68.198.32.50 - - [05/Dec/2014:00:10:44 -0500] "GET /ohprs/ohprs.js HTTP/1.1" 200 4773
68.198.32.50 - - [05/Dec/2014:00:10:44 -0500] "GET /ohprs/images/stripes.png HTTP/1.1" 200 5045

Open in new window

0
 
LVL 61

Expert Comment

by:gheist
ID: 40483096
You log in 2 formats to same file
apachectl graceful-stop
watch "ps -ef | grep httpd"
for minute
if all processes are out restart it
if not
killall -9 httpd
and start it back.
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40484732
Quite interesting. Indeed, after the `apachectl graceful-stop` the httpd tasks persisted. Strange that they didn't die. After killing them by-hand and restarting I got 6 httpd tasks created. When the 1st user connected I did get the browser info in the log, and another 5 httpd tasks were spawned. The next connections DID NOT show browser info. After a few minutes 3 of the original httpds are gone and 2 more appeared. Most new connections are not showing browser info in the log file.

1) Would respawns of new httpd task possibly be looking at a different httpd.conf?

2) Could it be simply that some browsers clients do not send information about themselves? (I doubt this is the case because my jsp programs can always get the User-Agent).
0
 
LVL 61

Expert Comment

by:gheist
ID: 40484870
1) NO. But if you impatiently kill master apache they are unlikely to ever exit.
2) They are not obliged.

Your httpd is compiled with almost default configuration, yes it respawns a process for every request.

Your server is vulnerable to so called slowloris attack. One can easily DOS it by opening maximum number of connections and sending one byte on connection every 29s
Please move on to system apache version which is at least patched against this. And consider moving to worker MPM once there. (will not be hard with reverse proxy config only)
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40485096
OK - this logging issue is more work than it deserves. Yes, I will move on to the current release, which I do have staged on another computer: Slackware 14.1/64bit, Apache 2.4.10. This is a completely vanilla, as-installed Apache; no tweaks, no builds.

My big problem with moving on is that I need tomcat and I've been told mod_jk for 64bit Slackware does not exist. That seems unbelievable to me. Do you agree? If you've got some insight on mod_jk for 64bits hop on over to question http://www.experts-exchange.com/Programming/Languages/Java/J2EE/JSP/Q_28574869.html and bail me out there.

Meanwhile, I've leave this question open a day longer in case you have a parting comment.
0
 
LVL 61

Expert Comment

by:gheist
ID: 40485248
Latest apache with slackware 13.37 is 2.2.29
it includes mod_proxy_ajp that is functional eqiovalent of mod_jk

why do you need mod_jk in first place?
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40499687
why do you need mod_jk in first place?

Because I'm running tomcat with Apache. I've never tried using mod_proxy_ajp - not sure how that would work with tomcat. I have staged a new 64bit 14.1 distro with Apache 2.4.10 and Tomcat 8.0.15. I have located the necessary mod_jk.so at http://tomcat.apache.org/download-connectors.cgi. All that is working. I've set the LogFormat as desired. So far, it seems to be logging just fine, but there's not a lot of traffic at the moment until I turn it loose in production. I intend to do that tomorrow, so I'll leave this open to post back results.
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40520462
After running the newer Apache 2.4.10 setup for quite a while all seems fine. Logging works as desired with UserAgent logged.

I think the new version solution solves the issue.
0
 
LVL 61

Expert Comment

by:gheist
ID: 40520518
Not the new version. You just have to overhaul apache config sometimes.
I hope you adapted to using vendor packages in the meantime.
0

Featured Post

What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

Join & Write a Comment

As Wikipedia explains 'robots.txt' as -- the robot exclusion standard, also known as the Robots Exclusion Protocol or robots.txt protocol, is a convention to prevent cooperating web spiders and other web robots from accessing all or part of a websit…
Hi, in this article I'm going to teach you how to run your own site, and how to let people in (without IP). I'll talk about and explain each step... :) By the way, everything in this Tutorial is completely free and legal. This article is for …
It is a freely distributed piece of software for such tasks as photo retouching, image composition and image authoring. It works on many operating systems, in many languages.
This video demonstrates how to create an example email signature rule for a department in a company using CodeTwo Exchange Rules. The signature will be inserted beneath users' latest emails in conversations and will be displayed in users' Sent Items…

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now