We have a proxy here, squid 2.6 on a Mandriva box. It performs extremely well for browsing, but it turns out to be a pain in the b.tt when downloading large files. It limits somehow the bandwidth to somewhere around 40 kbs while by just bypassing squid it goes up to everything available here, means 300 kbs.
I took the standard distribution that came with mandiva and it should not have delay pools, anyway there are none configured.
So the only thing I really need to know is, how can I get squid to download large files faster. I assume I have some bug in my conf.
This Question has been solved and asker verified All Experts Exchange premium technology solutions are available to subscription members.
Experts Exchange has been collecting answers to technology questions since 1996…3 million and counting! If you have a question, chances are we already have your answer.
If you can't find the exact answer you're looking for, ask our exclusive community of 50,000 experts. You’ll get a personalized answer from a trusted professional.
Thousands of free tech tips, tricks, how-to’s and tutorials are available in our peer reviewed articles section. See for yourself how smart our experts are, no login required.
Save solutions to your questions, answers you’ve discovered through searching plus helpful articles in your personal knowledgebase for easy future access.
Access the answers to your technology questions today.
30-day free trial. Register in 60 seconds.
Members of the expert community talk about why the experience at Experts Exchange is different than what you will find anywhere else.
Try it out and discover for yourself.
30-day free trial. Register in 60 seconds.
Join the community of experts here and help other tech pros by answering question in your area of expertise. You can earn FREE access to all Experts Exchange's premium features and resources.
Sorry it took me so long. I just upgraded to stable 3 in the hope it would help, also I compiled in the delay pools (which I still don't use).
First, it did not change anything with the performance. Now my results:
I tweaked lots of parameters and tested. If I "optimize" cache_mem (16MB) and max_object_size (4 MB) I can get it to go up to 90 kbps, but that is it. All other parameters that I touched did not have an effect.
If I use a direct connection I get 300 kbps.
As for the memory, definetively an option, I will add this parameter and see what the log says, but the machine has 3GB and it tells me it uses just 500 MB. All other hardware parameters of the machine should be acceptable, 1.8GHz processor, two non raid scsi drives for the cache_dirs.
Please post output of
$ rpm -q squid
and output of
squid -v
How do you find "it" uses 500MB ??? Do you get logged errors in cache log or dmesg?
Is your machine 32bit or 64bit ???
Does disabling store log and debug options help???
Using 'none' cache type instead of ufs ?
Post output of df -k ; mount -v
Post uname -a
netstat -ina
for me squid fetches at link speed no matter configuration.
and get rid of mindlessly large fqdn cache - it is not btree it is chained array
rmp -q squid:
package squid is not installed
squid -v
Squid Cache: Version 3.0.STABLE6
configure options: '--prefix=/opt/squid' '--exec-prefix=/opt/squid'
500 MB: using webmin 3GB total 2.6GB free, squid runs right now on 44MB
32 bit
log disabling does not help, it just saves a lot of space
fqdn reduced
how do I change to "none" cache type?
disabling log, nothing changes
uname -a:
Linux servername 2.6.17-13mdv #1 SMP Date i686 Intel(R) Xean(TM) CPU 1.80 GHz GNU/Linux
netstat -ina
Kernel Interface table
Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0 1500 0 11412898 0 0 0 22019055 0 0 0 BMRU
lo 16436 0 3105 0 0 0 3105 0 0 0 LRU
sit0 1480 0 0 0 0 0 0 0 0 0 O
df -k
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdb1 20153140 361012 18768388 2% /data/squid/cache0
/dev/sdc1 20153140 361120 18768280 2% /data/squid/cache1
/dev/sdb5 20153140 199536 18929864 2% /data/squid/log
mount -v
/dev/sdb1 on /data/squid/cache0 type ext3 (rw)
/dev/sdc1 on /data/squid/cache1 type ext3 (rw)
/dev/sdb5 on /data/squid/log type ext3 (rw)
Install mandriva's squid - pay a brief visit to http://easyurpmi.mandriva.
Then run "urpmi squid"
Then transfer your access lists into /etc/squid/squid.conf
Then run /etc/init.d/squid start
Kind of they seem to know building trade better than you and I (2008.1 is mine, 2.6 on 2007.0 is fine too) (see proof in code attachment)
I see following deficiencies:
You do not enable any removal policies at build and when memory cache is full documents are transfered into disk using fifo method one object at a time - that makes squid slow, especially when not using async-io or aufs features.
It is somewhere in Squid docs - disk cache is backed by 10% of its size of RAM. Examine memory used by command line 'top' utility, webmin is heap of cow droppings for performance monitoring.
You use 32bit system and process is limited to 2GB unless you overcustomize kernel.
So simple 2048MB config:
1000connectionsx64kB socket buffers=64MB
Process as such - 64MB
So we do have 1920MB at our disposal
cache_mem should be half of that - 960 MB
disk caches can total to 9600 MB using remaining memory
so - 4800 on each disk, not more
all the involved filesystems must use ext2+noatime filesystem since others do write twice, do not use drakdisk - it reboots. umount newfs and manual fstab editing and mount performs quicker
maximum object sizes - not more than half of cache_mem
You may alter between ufs/aufs storage schemas - i cannot guess which is better in your exact setup, example config file is full of explaining comments.
http://easyurpmi.zarb.org/
And install factory kernel, your custom build is supported by you alone and you see the result in initial question.
My uname looks like this:
Linux hostname-discarded 2.6.24.5-laptop-1mnb #1 SMP Tue May 27 12:52:18 EDT 2008 i686 Intel(R) Pentium(R) 4 CPU 2.40GHz GNU/Linux
build date is important to identify running kernel. bear in mind that next time.
ok, I went back to the mandriva rpm squid 2.6, did the file system change and I am back to max 70kbps large file download speed. Now with your recommended memory settings it is flying fast for websites but these large files are just slow motion. I guess if I have to download something large (anyway nobody else can ;-) I just switch my browser from proxy to direct.
I give up!
Business Accounts
Answer for Membership
by: uetian1707Posted on 2008-06-03 at 11:40:24ID: 21703180
Hi,
Try out squeezer for optimizing Squid:
http://maciej_kozinski.web