Link to home
Create AccountLog in
Linux

Linux

--

Questions

--

Followers

Top Experts

Avatar of molerner
molerner🇺🇸

Get Squid Cache back on its feet! (Linux RedHat9, Squidcache,)
Intro:
I just took over a server in an educational environment, running Red Hat 9. The networked computers run Windows XP. It's been maintained by many different people (all "uncontactable") over the years, with the result that it's been a bit "Frankensteined", and went down every couple of months. I wasn't working here when it happened, and nobody but the sysadmin knew anything about computers, so no word on what the problem was when it went down in the past, but I know most of the time a simple reboot wasn't the fix. I don't know very much about Linux, but I'm learning fast.

Functions:
Right now my main goal is to get the server back on its feet for about 3 months. The only thing that needs to stay working is Squid, which has been heavily customized (I believe that all the customizations only affected the squid conf, but I'm not positive). The way the network is supposed to work is that users log in to the server through telnet. Normally, any computer that tries to access the internet during "off hours" is blocked. The staff uses their assigned linux login to telnet into the server, which automatically launches an app called "kpine" (it somehow integrates with squid and pine- looks custom), and unlocks internet access for that computer. The only sites allowed are those on the whitelist. There was also a subadmin (called a "monitors" account) that made new users and managed things like expiration dates and passwords. == All of that works fine == There was also a feature within the monitors account to disable the "off-hours" (for when there was a vacation) and turn it back on (assumedly it ran some script, the same way the "mail" option in kpine literally just launched pine). This feature was purposely disabled at some point.

Problems:
1. The previously mentioned disabling of "off hours". What script was created/run through this option? (Less important)
2. (This is the most important) page requests for the past 10 months have been coming up sour- it would happen across multiple sites for multiple hours, then inexplicably starts working again. Error page is:

The requested URL could not be retrieved
---------------------------------------------------
While trying to retrieve the URL: http://www.example.com
The following error was encountered:
.........Unable to determine IP address from host name for www.example.com
The dnsserver returned:
.........No address records
This means that:
The cache was not able to resolve the hostname presented in the URL.
Check if the address is correct.
Your cache administrator is webmaster
---------------------------------------------------
(based on Squid)

I'm not sure what to do at this point. At the time that the problem is happening on the workstation (XP) computers, if I use Lynx on the server, the internet works perfectly.

Questions:
Is it that the DNS nameserver set up for the proxy isn't good enough? I could add in the nameservers for OpenDNS, I suppose, but that would entail updating restarting Squid with the new settings, which because of the Frankensteining, no longer occurs (except for the changes made in the squid-conf, that is- and even that only if I specify it in an argument). Also, typing "squid" in the console isn't goot enough (even though for lynx and pine and kpine, it is), I have to go to the exact folder for it. I'd be OK with upgrading Squid (we're only using 2.3*), but it might be more work than necessary right now. I plan to move it up to CentOS 4.5 (too scared to use 5) in Mid-August, but right now I just want it acceptably working.

PS:
- I'll be following this question closely, so if I can provide any more information, just ask!
- Any commands/code you give will have to be fully argumented etc., because although I can work my way around I'm still a major newbie.

Zero AI Policy

We believe in human intelligence. Our moderation policy strictly prohibits the use of LLM content in our Q&A threads.


ASKER CERTIFIED SOLUTION
Avatar of Kerem ERSOYKerem ERSOY

Link to home
membership
Log in or create a free account to see answer.
Signing up is free and takes 30 seconds. No credit card required.
Create Account

Avatar of Kerem ERSOYKerem ERSOY

Hi,

I forgot to add. There's a Firewall called IPCop and this firewall has an addon called Advanced Proxy add-on.  IPCop is a firewall distribution. It comes with its distribution of linux and takes a snap to install. After configuring your ethernet carsds you are ready to go. Then you install this add on. After enabling the add-on you will be able to use authentication. Also it has its own screen to create user accounts in 3 different groups (Standard, Extended, Blocked). Also there's another add-on called URL-filter. Using this you can block websites based on the category.

just check the IPCop at: http://www.ipcop.org
For the Advanced Proxy add-on: http://www.advproxy.net/
For the URL-Filter add-on visit: http://www.urlfilter.net/

The Advanced Proxy Add-on is also capable of authenticating users against Windows or LDAP too !!!
Here's a screenshot form the config screen:
http://www.advproxy.net/images/ipcop-main.png

Cheers,
K.

Avatar of molernermolerner🇺🇸

ASKER

Thanks, I'll try the first fix first. By the way, does IPCop's Advanced Proxy add-on do white-listing against all URLs?

Hi,

It does all you do with your present system:
- Blocks internet usage on time base.
- Authenticates people against internal database, LDAP or Windows domain.
- Allows URL's to be acceseesd without authentication.

Wht you mean by Whitelisting URL's is Allow the access of URL's where similar URL's are blaklisted. This means you are also using some sort uf URL blocker or something. There's another add-on from the same provider. Which is called URL -F,lter waht it does is Block web pages on contetnt (such as Porn, Gambling, Chat etc.)  It also allows to create whitelists (For ex. allow a gambling site while Gambling is prohibited as a category) or Blacklisting etc.

Here's the picture of its control panel:
http://www.urlfilter.net/images/ipcop-urlfilter.pngand here's a screenshot of extended blocking info.

Cheers,
K.

Reward 1Reward 2Reward 3Reward 4Reward 5Reward 6

EARN REWARDS FOR ASKING, ANSWERING, AND MORE.

Earn free swag for participating on the platform.


hi,

Here's the extended blocking info:
http://www.urlfilter.net/images/ipcop-biglist.png
and the blocking screen:
http://www.urlfilter.net/images/ipcop-urlfilter.png

Cheers,
K.

Avatar of molernermolerner🇺🇸

ASKER

KeremE:

I used your command check- regular went through, proxy didn't. (Also, client computers didn't go through, but lynx did on the server)
I wiped out all of the cache files. No change, same error- and again, it happens to varying sites at varying times.

By the way, I can't just type in "squid" at the prompt- I have to go and go to the directory each time. Typing "squid" at the prompt, even when cd'd at /, gives an error- though "man squid" doesn't. Very odd...

By the way, IPCOP is a Linux distro!... which I would have known if I read your post a little more carefully.  I could also use SmoothWall, which is compatible with both plugins. And I also didn't know that IPCOP goes through squid as well- it's starting to look better by the minute.

Hi,

First of all if squid does not execute when you type means squid path is not included in your path which is usually set in your .bash_profile. You just edit it and add your squid path into your PATH statement.

The command just don't execute even after you've cd'ed means that yourt PATH statemnt does mnot include "." meaning "this directory" just add it to your path and it would execute when you type just the command. Remember since . is not genrally included in PATH listings pepole generally run programs with a commands like that ./squid after they've cd'ed th correct directory.

If you thpe man squid and get the output it means that manual pages are just installed to some directory which is included in MANPATH system variable. So it is not so strange. These are all configuration issues. It seesm that since the guys administering the server in the past we're programmer like and always run the program using the full path so that they did not try to make it easier for users and did not set these variables.

My example was assuming that Squid is running with default port settings (3128) you should first veriffy that if ths is the correct port setting for squid.

Re: IPCOP. Yeah IPCOP comes with its distro but it is based on RedHat Linux. So I don't hink it will be a problem for you. IPCop  has a bigger community and has better support hough initially it was based on smoothwall.

Cheers,
K.


Free T-shirt

Get a FREE t-shirt when you ask your first question.

We believe in human intelligence. Our moderation policy strictly prohibits the use of LLM content in our Q&A threads.


Avatar of molernermolerner🇺🇸

ASKER

I have no idea how to PATH things. All I know is that the squid.conf is in /etc/squid and the program is in /usr/bin/squid. What would be the process here?

3128 is indeed the right port

PATH is an invironment variable. This is set when you login using the file called ~/.bash_profile (~means your home directory.

when you edit the file you see the contents has asection for PATH such as this:

# User specific environment and startup programs
PATH=$PATH:$HOME/bin
export PATH

When you edit the path if you add your default path for squid and . the result will be something similar to this:
# User specific environment and startup programs
PATH=/usr/bin/squid:.:$PATH:$HOME/bin
export PATH
notice that items are seperated using a colon ':' and notice the "." after the path for squid. then save and logoff/logon.  OK this was a bit off topic but I hope it'll help.

Ok lets get back to our topic. It seems that you squid.config sets some dns servers via the config parameter:

dns_servers

Just locate it in your squid.conf. Get back to the command line and try the servers listed here with a command such as this:

dig @dns_servers_server_one_at_a_time unresolved.domain

here dns_servers_server_one_at_a_time represents the serever list obtained from your dns_server squid parameter entered one by one if it contains multiple servers.

unresolved.domain represents a domain ssuch as www.ge.com which is causing erros in squid.

You can also change parameter to dns_servers with the server in your /etc/resolv.conf "nameserver" idenditier contents.

cheers,
K.





Avatar of molernermolerner🇺🇸

ASKER

Squid's using the nameservers in the /etc/resolv.conf  . There's two nameservers listed (neither of them local)- now, if I add say the two for OpenDNS on the bottom; if the top one fails, will Squid try the next, etc?

Also, I noticed an entry on the tippytop that says "search blah.ourserver.com" (obviously the blah.ourserver.com being a replacement for our local server).  I have little grasp of the name search concept, and I don't know if it's a proper entry

Reward 1Reward 2Reward 3Reward 4Reward 5Reward 6

EARN REWARDS FOR ASKING, ANSWERING, AND MORE.

Earn free swag for participating on the platform.


Avatar of molernermolerner🇺🇸

ASKER

Update: all the nameservers work fine, I used the dig method you showed me

I finally got squid to run with all changes made- still no luck.

I didn't upgrade, though- because in order to install the latest version of squid onto RedHat9 you have to upgrade the GLIBC- and I have no intention of doing that!

The IPCop idea is looking better by the minute...

Hi,

"search" keyword is just to tell the DNS server that when you query a name just like "www" how will the dns complete it to have a FQDN (Fully Qualieid Domain Name) such as www.ourserver.blah.com. It has nothing to do when you query a FQDN (or rather though it adds this after the domain in the end it would not find anything).

So did you find anything in squid.conf that looks like
dns_servers ns.blah.com
etc. ?
Does your squid conf something similr to that? Does it contain the same info as your resolv.conf? Otherwise may be it does not contain anything like that at all ? So that it will use your dns server settings (in resolv.conf)

I agree with you on not trying to upgrade RH9 which is a very old and unsopported obsolete OS.  I reely think that IPCOP seems like a viable alternative in your case.

I will also strongly suggest you to clone your existing hard disk with a software such as g4l (oepn source Norton Ghost replacement) since if there's something wrong wtih the O/S it will not be very easy to get it worked again.

Cheers,
K.

BTW all the settings we did into .bash_profile was for only for your easy access to the squid command ans has nothing to do with squid internal working this is why I had told that it was a bit off-topic.

But in the end this issue seems to be a DNS issue and we need to solve this issue using tools like dig and comparing the results with squid queries etc.

Cheers,
k.

Free T-shirt

Get a FREE t-shirt when you ask your first question.

We believe in human intelligence. Our moderation policy strictly prohibits the use of LLM content in our Q&A threads.


Avatar of molernermolerner🇺🇸

ASKER

dns_servers is commented out- it's using what's in /etc/resolv.conf

The dig command comes up fine with every one of the 4 nameservers- even while I'm refreshing with a client PC that's showing an error. I don't get what could possibly be going wrong here!

Avatar of molernermolerner🇺🇸

ASKER

KeremE, you get a plauqe in the server room when this is all over. I'm not joking.

I'll be waiting for my plaque ;-) (I am not joking too)

If everything is ok with DNS's then there's one more thing. I guess may be there's something wrong with your ACL settings and may be it is mixing up with your timebase settiings and getting blocked while it should not.

URL filter has some similar functionality. You can either block the pages with a HTML screen or just with a DNS error. My guess is beause of several rules are shading some other and causing some sites to get blocked at times.

You should trace all ACL's as they are exist in your config file. Look for some rules that are added becuase of a mistake etc. causing a deny to some sites on timebase.

If you don't mind getting it public you can post it here or may be sending it with sensitive areas are concealed etc.

Cheers,
K.



Reward 1Reward 2Reward 3Reward 4Reward 5Reward 6

EARN REWARDS FOR ASKING, ANSWERING, AND MORE.

Earn free swag for participating on the platform.


Avatar of molernermolerner🇺🇸

ASKER

Here's all the stuff from the squid.conf - I stripped out everything that was #'d out


cache_dir ufs /var/spool/squid 100 16 256
cache_access_log /var/log/squid/access.log
cache_log /var/log/squid/cache.log
cache_store_log /var/log/squid/store.log

mime_table /etc/squid/mime.conf
pid_filename /var/run/squid.pid
dns_children 32
unlinkd_program /usr/local/squid/bin/unlinkd

load_module /usr/local/squid/bin/redirect.so /etc/squid/filter.exp

load_module /usr/local/squid/bin/allowtype.so text/html application/x-javascript text/directory text/css text/plain

refresh_pattern            ^ftp:            1440      20%      10080
refresh_pattern            ^gopher:      1440      0%      1440
refresh_pattern       .            0      20%      4320

acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl SSL_ports port 443 563
acl Safe_ports port 80 21 443 563 70 210 1025-65535
acl Safe_ports port 280            # http-mgmt
acl Safe_ports port 488            # gss-http
acl Safe_ports port 591            # filemaker
acl Safe_ports port 777            # multiling http
acl CONNECT method CONNECT


http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports

acl localhost src 127.0.0.1/32
acl net_edu src 192.168.200.0/8
acl net_ppp src 192.168.5.128

http_access deny all !net_edu !localhost !net_ppp

acl home dstdom_regex intserv\.edu\.yna\.edu$
http_access allow home

acl home_old dstdom_regex intserv\.edu\.netiv\.org$
http_access allow home_old
acl home1 dst 192.168.200.253
http_access allow home1
acl home2 dstdom_regex www\.netiv\.org$
http_access allow home2
acl home3 dstdom_regex www\.netivaryeh\.org$
http_access allow home3
acl home4 dstdom_regex www\.yna\.edu$
http_access allow home4
acl anti_virus dstdom_regex liveupdate\.symantecliveupdate\.com$
http_access allow anti_virus
acl redir1 urlpath_regex -i ^http:\/\/shurl\.org\/mail2web$
http_access allow redir1
acl redir2 urlpath_regex -i ^http:\/\/www\.shurl\.org\/mail2web$


acl jpg1 urlpath_regex -i ^http:\/\/intserv\.edu\.yna\.edu/(.*)\.jpg$
acl exe1 urlpath_regex -i ^http:\/\/intserv\.edu\.yna\.edu/(.*)\.exe$
acl jpg2 urlpath_regex -i ^http:\/\/127\.0\.0\.1\/(.*)\.jpg$
acl jpg3 urlpath_regex -i ^http:\/\/www.netivaryeh\.org/(.*)\.jpg$
acl jpg4 urlpath_regex -i ^http:\/\/www\.144\.bezeq\.com\/(.*)\.jpg$
acl jpg5 urlpath_regex -i ^http:\/\/www\.hasc\.net\/images\/(.*)\.jpg$
acl jpg6 urlpath_regex -i ^http:\/\/www\.yna\.edu/(.*)\.jpg$
acl jpg7 urlpath_regex -i ^http:\/\/www\.netiv\.org/(.*)\.jpg$
http_access allow exe1


acl nojpgs urlpath_regex -i \.jpg$
http_access allow jpg1 jpg2 jpg3 jpg4 jpg5 jpg6 jpg8
http_access deny nojpgs !jpg1 !jpg2 !jpg3 !jpg4 !jpg5 !jpg6 !jpg8
acl noswf urlpath_regex -i \.swf$
http_access deny noswf
acl nomp3 urlpath_regex -i .\mp3$
http_access deny nomp3
acl nowav urlpath_regex -i .\wav$
http_access deny nowav
acl nobmp urlpath_regex -i .\bmp$
http_access deny nobmp
acl noau urlpath_regex -i .\au$
http_access deny noau
acl nora urlpath_regex -i .\ra$
http_access deny nora
acl noram urlpath_regex -i .\ram$
http_access deny noram
acl nodoc urlpath_regex -i .\doc$
http_access deny nodoc
acl nortf urlpath_regex -i .\rtf$
http_access deny nortf
acl nopdf urlpath_regex -i .\pdf$
http_access deny nopdf
acl noxls urlpath_regex -i .\xls$
http_access deny noxls
acl nozip urlpath_regex -i .\zip$
http_access deny nozip
acl noexe urlpath_regex -i .\exe$
http_access deny noexe
acl nomid urlpath_regex -i .\mid$
http_access deny nomid
acl nomidi urlpath_regex -i .\midi$
http_access deny nomidi
acl norar urlpath_regex -i .\rar$
http_access deny norar
acl noavi urlpath_regex -i \.avi$
http_access deny noavi

acl BAD dst 0.0.0.0/0

acl hk1 dstdom_regex (.*)\.baltimoresun\.com$
http_access allow hk1
acl hk2 dstdom_regex (.*)\.observer\.com$
http_access allow hk2
acl hk3 dstdom_regex (.*)\.newsday\.com$
http_access allow hk3
acl hk4 dstdom_regex (.*)\.newsweek\.com$
http_access allow hk4
acl hk5 dstdom_regex (.*)\.billboard\.com$
http_access allow hk5
acl hk6 dstdom_regex (.*)\.dailynews\.com$
http_access allow hk6
acl hk7 dstdom_regex (.*)\.rnc\.org$
http_access allow hk7
acl hk8 dstdom_regex (.*)\.sky\.com$
http_access allow hk8
acl hk9 dstdom_regex (.*)\.honestreporting\.com$
http_access allow hk9
acl hk10 dstdom_regex (.*)\.woot\.com$
http_access allow hk10
acl hk11 dstdom_regex (.*)slashdot\.org$
http_access allow hk11
acl hk12 dstdom_regex (.*)usatoday\.com$
http_access allow hk12
acl hk13 dstdom_regex (.*)www\.drudgereport\.com$
http_access allow hk13
acl hk14 dstdom_regex (.*)\.guardian\.co\.uk$
http_access allow hk14
acl hk15 dstdom_regex (.*)\.idf\.il$
http_access allow hk15
acl hk16 dstdom_regex (.*)\.cnn\.com$
http_access allow hk16
acl hk17 dstdom_regex (.*)\.cnn\.net$
http_access allow hk17
acl hk18 dstdom_regex (.*)www\.reuters\.com$
http_access allow hk18
acl hk19 dstdom_regex (.*)\.yahoo\.com$
http_access allow hk19
acl hk20 dstdom_regex (.*)pqasb\.pqarchiver\.com$
http_access allow hk20
acl hk21 dstdom_regex (.*)\.nytimes\.com$
http_access allow hk21
acl hk22 dstdom_regex (.*)www\.nydailynews\.com$
http_access allow hk22
acl hk23 dstdom_regex (.*)www\.boston\.com$
http_access allow hk23
acl hk24 dstdom_regex (.*)\.a7\.org$
http_access allow hk24
acl hk27 dstdom_regex (.*)\.timesonline\.co\.uk$
http_access allow hk25
acl hk28 dstdom_regex (.*)\.sunday-times\.co\.uk$

#[and so on for all the whitelisted pages- Mo]

http_access deny BAD
http_access deny all
icp_access allow all
miss_access allow all
cache_effective_user squid
cache_effective_group squid

acl hakotelhome dstdomain home.hakotel.edu
always_direct allow hakotelhome

Is it that it hasn't opened the port used for TCP/UDP DNS lookups? Does that question even make sense?

Thanks again,
--- Mo

Avatar of molernermolerner🇺🇸

ASKER

__________________________________________________________________________
|****************************************************************************************************************|
|********************************************* IT'S OVER!!! ****************************************************|
|****************************************************************************************************************|
 ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

It was a stupid old bug with a stupid old version of squid.

I embarked on an adventure on installing and configuring YUM with the ancient Red Hat 9, and it updated Squid to a decently new version (2.5x)

Thank you so much KeremE! I'll get you that plaque and send you a picture.

Thanks again.
--- Mo

P.S: The learning about Linux and Linux networking made it all worth it. And the plaque will make it worth it for you.

Hi,

I'm happy that your problem is solved now.  I hope this was helpful. I will be happy to see this plaque.

Cheers,
K.

Free T-shirt

Get a FREE t-shirt when you ask your first question.

We believe in human intelligence. Our moderation policy strictly prohibits the use of LLM content in our Q&A threads.

Linux

Linux

--

Questions

--

Followers

Top Experts

Linux is a UNIX-like open source operating system with hundreds of distinct distributions, including: Fedora, openSUSE, Ubuntu, Debian, Slackware, Gentoo, CentOS, and Arch Linux. Linux is generally associated with web and database servers, but has become popular in many niche industries and applications.