Need to exclude certain messages from Apache access_log

I am running Apache 2.2.9 and Tomcat 6.0.14 on Slackware 13.37.0, kernel 2.6.37.6-smp.

I am logging messages to /usr/local/apache2/logs/access_log as follows:
98.102.63.107 - - [21/May/2015:10:51:56 -0400] "GET /ohprs/images/memoriam/JackHolland.jpg HTTP/1.1" 200 10502
98.102.63.107 - - [21/May/2015:10:51:56 -0400] "GET /ohprs/images/memoriam/AndrewBaldridge.jpg HTTP/1.1" 200 9147
98.102.63.107 - - [21/May/2015:10:51:58 -0400] "GET /ohprs/images/stripes.png HTTP/1.1" 304 -
98.102.63.107 - - [21/May/2015:10:51:58 -0400] "GET /ohprs/images/logo.png HTTP/1.1" 304 -

Open in new window

I want to eliminate logging of any accesses to the ohprs/images folder. In my httpd.conf file I have:

    SetEnvIf Request_URI "/ohprs/images" dontlog
    CustomLog "logs/access_log" combined env=!dontlog

I've tried various combinations including using "Referer" as the attribute and with or without the leading "/" in the regex. Nothing seems to work.

Does this version of Apache support SetEnvIf? The docs seem to indicate it does. Do I need to load some module?  Perhaps I have the SetEnvIf directive in the wrong place in httpd.conf? Any ideas?

Here's my httpd.conf:
ServerRoot "/usr/local/apache2"

PidFile /var/run/apache.pid

Listen 80

LoadModule    jk_module  lib/mod_jk.so

JkWorkersFile conf/workers.properties

JkShmFile     /var/log/httpd/mod_jk.shm

JkLogFile     /var/log/httpd/mod_jk.log

JkLogLevel    info

JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "

JkMount  /ohprs/* worker1
JkMount  /ohprs worker1

JkMount  /ohprs2/* worker1
JkMount  /ohprs2 worker1

JkMount  /office/* worker1
JkMount  /office worker1

<IfModule !mpm_netware_module>
<IfModule !mpm_winnt_module>
User apache
Group apache

</IfModule>
</IfModule>

ServerAdmin mfoley@novatec-inc.com

ServerName www.ohprs.org

DocumentRoot "/usr/local/apache2/htdocs"

Rewritelog logs/rewrite.log
Rewriteloglevel 5

<Directory />
    Options FollowSymLinks
    AllowOverride None
    Order deny,allow
    Deny from all
</Directory>

<Directory "/usr/local/apache2/htdocs">
    Options Indexes FollowSymLinks ExecCGI

    AllowOverride All

    Order allow,deny
    Allow from all

</Directory>

<IfModule dir_module>
    DirectoryIndex index.html
</IfModule>

<FilesMatch "^\.ht">
    Order allow,deny
    Deny from all
    Satisfy All
</FilesMatch>

<Location /office>
RewriteEngine On
RewriteCond %{HTTPS} !on
RewriteRule ^/?(.*) https://%{SERVER_NAME}%{REQUEST_URI} [L,R=301]
</Location>

<Location /ohprs>
RewriteEngine On
RewriteCond %{HTTPS} !on
RewriteRule ^/?(.*) https://%{SERVER_NAME}%{REQUEST_URI} [L,r=301]
</Location>

<Location /ohprs2>
RewriteEngine On
RewriteCond %{HTTPS} !on
RewriteRule ^/?(.*) https://%{SERVER_NAME}%{REQUEST_URI} [L,r=301]
</Location>

<Location /ohprs/upload>
    AuthType Basic
    AuthName "Restricted"
    AuthUserFile conf/passwords
    Require valid-user
    AuthBasicProvider file
</Location>

  CheckCaseOnly on

ErrorLog "logs/error_log"

LogLevel warn

<IfModule log_config_module>
    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
    LogFormat "%h %l %u %t \"%r\" %>s %b" common

    <IfModule logio_module>
      LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
    </IfModule>

    SetEnvIf Request_URI "/ohprs/images" dontlog
    CustomLog "logs/access_log" combined env=!dontlog

</IfModule>

<IfModule alias_module>

    ScriptAlias /cgi-bin/ "/usr/local/apache2/cgi-bin/"

</IfModule>

<IfModule cgid_module>
</IfModule>

<Directory "/usr/local/apache2/cgi-bin">
    AllowOverride None
    Options None
    Order allow,deny
    Allow from all
</Directory>

DefaultType text/plain

<IfModule mime_module>
    TypesConfig conf/mime.types

    AddType application/x-compress .Z
    AddType application/x-gzip .gz .tgz

    AddHandler cgi-script .cgi

</IfModule>

Include conf/extra/httpd-ssl.conf
<IfModule ssl_module>
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
</IfModule>

Open in new window

LVL 1
jmarkfoleyAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Joseph GanSystem AdminCommented:
0
jmarkfoleyAuthor Commented:
Tried that. Added the following to my httpd.conf -- also did this on a newer Slackware64 14.1, kernel 3.10.7, Apache version 2.4.10.

    SetEnvIf Request_URI "^/calendars" dontlog
    CustomLog "/var/log/httpd/access_log" common=!dontlog

access_log:
192.168.0.56 - - [26/May/2015:20:05:23 -0400] "GET /calendars/Dennis_Smith_Calendar.ics HTTP/1.1" 304 -
192.168.0.56 - - [26/May/2015:20:05:23 -0400] "GET /calendars/Mike_Press_Calendar.ics HTTP/1.1" 304 -
192.168.0.56 - - [26/May/2015:20:05:23 -0400] "GET /calendars/Mark_Atkeson_Calendar.ics HTTP/1.1" 304 -

Open in new window


Any other ideas?
0
Joseph GanSystem AdminCommented:
Did you try:

SetEnvIf Request_URI "^/calendars" log_exclude=true
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

jmarkfoleyAuthor Commented:
Did try that on the apache 2.2.9; just added it the the Apache 2.4.10 server.

But wait! Here's wrinkle. Don't know why I didn't see this earlier unless I forgot to restart apache. On the 2.4.10 server I have in the access_log:
107.194.150.115 - - [26/May/2015:22:37:01 -0400] "GET / HTTP/1.0" 400 362
107.194.150.115 - - [26/May/2015:22:37:02 -0400] "GET / HTTP/1.0" 400 362
common=!dontlog
common=!dontlog
107.194.150.115 - - [26/May/2015:22:37:17 -0400] "GET / HTTP/1.0" 400 362
107.194.150.115 - - [26/May/2015:22:37:20 -0400] "GET / HTTP/1.0" 400 362
107.194.150.115 - - [26/May/2015:22:37:20 -0400] "GET / HTTP/1.0" 400 362
107.194.150.115 - - [26/May/2015:22:37:24 -0400] "GET / HTTP/1.0" 400 362
107.194.150.115 - - [26/May/2015:22:37:24 -0400] "GET / HTTP/1.0" 400 362
107.194.150.115 - - [26/May/2015:22:37:28 -0400] "GET / HTTP/1.0" 400 362
107.194.150.115 - - [26/May/2015:22:37:31 -0400] "GET / HTTP/1.0" 400 362
107.194.150.115 - - [26/May/2015:22:37:31 -0400] "GET / HTTP/1.0" 400 362
107.194.150.115 - - [26/May/2015:22:37:34 -0400] "GET / HTTP/1.0" 400 362
107.194.150.115 - - [26/May/2015:22:37:34 -0400] "GET / HTTP/1.0" 400 362
common=!dontlog=true
common=!dontlog=true
common=!dontlog=true
107.194.150.115 - - [26/May/2015:22:39:29 -0400] "GET / HTTP/1.0" 400 362
107.194.150.115 - - [26/May/2015:22:39:32 -0400] "GET / HTTP/1.0" 400 362
common=!dontlog=true
107.194.150.115 - - [26/May/2015:22:39:38 -0400] "GET / HTTP/1.0" 400 362
common=!dontlog=true
107.194.150.115 - - [26/May/2015:22:40:14 -0400] "GET / HTTP/1.0" 400 362
107.194.150.115 - - [26/May/2015:22:40:17 -0400] "GET / HTTP/1.0" 400 362
common=!dontlog=true
common=!dontlog=true
common=!dontlog=true
common=!dontlog=true
common=!dontlog=true

Open in new window

So with or without the "=true" it's working-ish. It's not logging the /calendars URI_Request, but instead is logging *lots* of "common=!dontlog=true". That doesn't quite solve my intended problem of not cluttering my access_log. Is there a way to not log the fact that it is not logging?
0
Joseph GanSystem AdminCommented:
CustomLog logs/access_log combined env=!log_exclude
0
jmarkfoleyAuthor Commented:
I see the error. I omitted "env="; however, still getting "/calendars" logged:
192.168.0.56 - - [27/May/2015:00:06:36 -0400] "GET /calendars/Smith_Calendar.ics HTTP/1.1" 304 -
192.168.0.56 - - [27/May/2015:00:06:36 -0400] "GET /calendars/Mike_Calendar.ics HTTP/1.1" 304 -
192.168.0.56 - - [27/May/2015:00:06:36 -0400] "GET /calendars/Mark_Calendar.ics HTTP/1.1" 304 -

Open in new window

0
Joseph GanSystem AdminCommented:
SetEnvIf Request_URI "^/calendars" log_exclude=true
CustomLog logs/access_log combined env=!log_exclude

You can have more than one SetEnvIf directive in your httpd.conf .
0
jmarkfoleyAuthor Commented:
I believe I have exactly what you have except I'm calling ENV variable "dontlog" instead of "log_exclude":

    SetEnvIf Request_URI "^/calendars" dontlog=true
    CustomLog "/var/log/httpd/access_log" common env=!dontlog

Yet it doesn't work. Yes, I've restarted httpd. Have you actually tried this?
0
jmarkfoleyAuthor Commented:
The following link indicates that this does not work if redirecting to SSL: http://www.apachelounge.com/viewtopic.php?p=22266, which I am doing. The poster in that link says his SetEnvIf did not work until he turned off the SSL redirection. Otherwise, no solution was posted. I am not in a position to turn this off and for production use don't want to. Not really sure why this wouldn't work redirecting to SSL. Any ideas?

I have the following in my DOCUMENT_ROOT/.htaccess:

RewriteEngine On
RewriteCond %{HTTPS} !^on
RewriteRule .* https://%{SERVER_NAME}%{REQUEST_URI} [L,R]
0
jmarkfoleyAuthor Commented:
More info: I turned off the http rewrite. If I reference the /calendars directory as http://hostname.com/calendars, then the nolog function works. However, even with the rewrite off, if I use https://hostname.com/calendars the request gets logged to access_log. So, using https at all causes the SetEnvIf to fail.

Why?
0
jmarkfoleyAuthor Commented:
This question seems harder than I thought. OK, let's drop back a bit. How can I turn off ALL logging in Apache? I've commented out the various CustomLog directives. That didn't do it, still logging to access_log. Next, I added "CustomLog /dev/null common". That didn't do it. Still logging to access_log. How can I turn of ALL logging? (please respond if you know and if you've successfully done this -- don't just google for a guess!)
0
jmarkfoleyAuthor Commented:
I figured out the problem. Logging was also set for SSL in httpd-ssl.conf. Specifically, it had:

TransferLog "/var/log/httpd/access_log"

So, this was logging messages to access_log regardless of what I had in http.conf. Not sure why that is the default, seems wrong. The solution was to comment out ALL logging directives in httpd-ssl.conf and have only the following in httpd.conf:

LogLevel warn
LogFormat "%h %l %u %t \"%r\" %>s %b" common
SetEnvIf Request_URI "^/ohprs/images" dontlog
CustomLog "logs/access_log" combined env=!dontlog

That seems to do the trick. Again, the as-shipped Apache settings seem wrong and confusing, but what do I know.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
jmarkfoleyAuthor Commented:
I figured out the problem, but points to ganjos for hanging in there - I don't often get a lot of participation on Apache questions.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Apache Web Server

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.