Block directory listing when browsing by IP

Suggestion:  Create the server as a named vhost.  Then it is impossible to access it by IP.

This is prudent considering the many bots, scrapers and script kiddies which go through entire IP blocks looking for vulnerable servers and accessing them by IP address.

This would probably be the way to go. Right now I've disabled indexing server-wide and there don't seem to be any problems, but disabling IP access altogether would probably be the best solution.

Do you have any pointers on the quickest way to do this?

Here's a chunk of httpd.conf that should give you a jumping-off point.

Note particularly the segment of the main config file that intercepts any request coming into the "base" server (which only receives IP-address-based requests), and turns it around with a standard rejection page.

# ============= APACHE2 SERVER CONFIGURATION FILE ============
#
#               For all web sites on this system
#
# ============= APACHE2 SERVER CONFIGURATION FILE ============

# ========================================================
#
#                APACHE GLOBAL ENVIRONMENT
#
# ========================================================

# Group, User: Group/user to run as after switching from root.
User httpd
Group httpd

# ServerRoot: The top of the directory tree under which the
#  server's configuration, error, and log files are kept.
ServerRoot "/usr/local/apache2"

# PidFile: Where the process ID number is stored when starting.
PidFile /var/log/apache/httpd.pid

# Listen: Bind Apache to specific IP addresses and/or ports.
#  See also the <VirtualHost> # directive.
Listen 80

# StartServers:  Number of child/servers to start with
StartServers 1

# ========================================================
#
#              DEFAULT/"MAIN" SERVER DIRECTIVES
#
# ========================================================

#
# Where server problems should be emailed.
#
ServerAdmin root@127.0.0.1

#
# The default hostname sent back to clients.
#
ServerName localhost

#
# The default document base directory.
#
DocumentRoot /www/localhost

#
# System root directory: Deny all access
#
<Directory />
    Options +FollowSymLinks -Indexes
    AllowOverride None
    Order deny,allow
    Deny from all
</Directory>

#
# CoreDumpDirectory: Where to write core dump in case of a crash
#
CoreDumpDirectory log

#
# Log the sitenames of clients or just IP addresses
#
HostnameLookups Off

#
# Append server info to internally generated pages
#
ServerSignature Off

# ========================================================
#
#                 DEFAULT VIRTUAL HOST
#
# ========================================================

#
# IP address[:port] for name-based virtual hosts
#
NameVirtualHost *:80

#
# This "Virtual"Host is used for IP-addressed requests, i.e., no server name.
#
<VirtualHost *:80>

#
# Set default root and access allowed
#
DocumentRoot /www/localhost
<Directory /www/localhost>
  Order allow,deny
  Allow from all
  Options none
  AllowOverride none
</Directory>

#
# Configure error logging
#
CustomLog /dev/null locallog

# Redirect everything to a blocking page
RewriteEngine On
RewriteLogLevel 0
RewriteCond %{REQUEST_URI} !NoWai\.jpg$            [NC]
RewriteCond %{REQUEST_URI} !standard\.html$        [NC]
RewriteRule .* /standard.html [L]

</VirtualHost>


# ========================================================
#
#                 VIRTUAL HOST INCLUDES
#
# ========================================================

#
# 1: website1.com
#
Include conf/website1.conf

#
# 2: website2.biz
#
Include conf/website2.conf

Select allOpen in new window

And the included config file for website1, the "real" server.

# ====================== VIRTUAL HOST  ======================
#
#                 www.website1.com
#
# ====================== VIRTUAL HOST  ======================

#
# VirtualHost begin: Define a new virtual host
#
<VirtualHost *:80>

#
# ServerName: The primary name for this virtual host
# ServerAlias: Other acceptable names for this virtual host
# UseCanonicalName:  Use ServerName to build URLs referring to itself
#
ServerName www.website1.com
ServerAlias website1.com
UseCanonicalName on

#
# DocumentRoot: This server's base directory.
#
DocumentRoot /www/ws1

#
# This server's base directory: Permit all access
#
<Directory /www/ws1>
  Order deny,allow
  Allow from all
  Options +FollowSymLinks -Indexes
  AllowOverride none
</Directory>

#
# Security stuff
#

# Remove the ETAG from all response headers
Header unset ETag
FileETag None

# Add the X-ROBOTS-TAG to all response headers
Header set x-robots-tag "noarchive,noimageindex,noodp"

# Add the X-XSS-PROTETCION tag to all response headers
Header set X-XSS-Protection "1; mode=block"

# Disable framing our site from any other site
Header set X-Frame-Options SAMEORIGIN

# Disable content from any other site
Header set Content-Security-Policy "default-src 'none'; script-src 'self';  img-src 'self'; style-src 'self';"

# Catch anything with invalid (on this site) characters
RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D|\?|\+|\%|\&|=).* [NC]
RewriteRule .* - [F,L]

# Rewrite a root level URI of "/" to "/index.html"
#  ... but no other URIs.  Let them throw a 404.
RewriteRule ^/$ /index.html [R=301]

# Always allow access to robots.txt, no matter who or what
RewriteCond %{REQUEST_URI} robots\.txt$            [NC]
RewriteRule .* /robots\.txt                         [L]

#
# VirtualHost end: End of definitions for this virtual host
#
</VirtualHost>

Select allOpen in new window

@Daniel, in your original question you don't mention why you're trying to block IP. My guess is some bad behavior.

IP blocking is best managed by fail2ban, which allows recipes/rules to be written to flag IPs producing bad behavior.

For example, I have a rule which says any IP with 5 failed login attempts over a 1 hour period, gets blocked for 1 hour.

If they repeat this behavior 5 times in 24 hours, then they're blocked for 1 day.

This might be what you're looking for to address blocking/unblocking IPs on a regular basis, with no intervention on your part.

Thanks David, actually that's now what I meant in my question, remember I'm italian so I'm not writing in my mother tongue.

I already use fail2ban for blocking misbehaving external IPs. What I need to do here is keeping people from browsing my server using the server's own IP.

I think previous suggestions are on the money but I still need to try them out and test the results. Will let you know as soon as I do.

Ah... Now I understand.

I run a private hosting company for high traffic + high speed WordPress hosting.

Here's how I handle this.

1) Whatever server you're running will have a default web page. I'll use Apache as an example, which uses /var/www/html, if you leave the 000-default.conf site configured.

2) At this point all traffic visiting your site by IP will land on /var/www/html pages.

3) All you have to do to deal with these miscreants, is write a fail2ban rule targeting any visit these visits.

So remove the /var/www/html/index.html file + replace with foo.txt which will only be visible via directory scan + you know 100% of the time any IP visiting foo.txt is a bot.

Then just use fail2ban to block an foo.txt visit for an hour.

This is the dirt simple way to catch Bots.

I try opening up holes like this so I can catch Bots via stupid requests, like foo.txt, to avoid trying to catch them with more complex countermeasures.

You can even have some serious fun with Bots by installing the TARPIT module.

Tarpitting is way more fun then using the normal fail2ban reject-with icmp-host-prohibited approach.

Tarpitting catches connections at the Kernel + using minuscule resources, holds every Bot connection open... pretty much forever...

The net effect is no measurable CPU or memory usage on your end.

For the Bot, its Armageddon, because as the Bot opens more + more connections to your site, your site siphons off more + more Bot resources.

Usually Bots figure this out quickly + abandon hitting your site.

And sometimes, really stupid Bots will continually hit your site + then stall, so once they hit your site, they can't attack anyone else's site.

I consider Tarpitting a universal service.

If every Linux machine ran Tarpitting, the Bot universe would implode.

Right now I've solved the IP address access problem adding this configuration file:

<VirtualHost *:80>
    ServerName <THE IP ADDRESS OF MY SERVER>
    Redirect 403 /
    ErrorDocument 403 "Sorry, no IP address access allowed"
    DocumentRoot /var/www/html/null/
    UseCanonicalName Off
    UserDir disabled
</VirtualHost>

Select allOpen in new window

It doesn't cover https access though, and if I use a similar configuration for port 443 I get an error when loading Apache. It simply says error with no explanation, which is quite unusual.

But are we sure a bot can't access stuff on my server using https and IP address? If I try, I get a certificate error, BUT if I allow an exception I can see stuff I now can't see if I access the IP address with http. So couldn't a bot do the same?

1) But are we sure a bot can't access stuff on my server using https and IP address?

Depends on the Bot.

If a Bot implements HTTPS (more do every day), then they can access HTTPS content.

2) If I try, I get a certificate error, BUT if I allow an exception I can see stuff I now can't see if I access the IP address with http. So couldn't a bot do the same?

Correct.

Anything you can do, a Bot can do.

Suggestion: Open a new question about blocking Bots verses Humans.... or maybe about Bot/Human differentiation... as this can be a long discussion.

And, the answer to your original question appears to be...

Options -Indexes

Select allOpen in new window