Best caching proxy to reduce webserver bandwidth

ecocozza
ecocozza used Ask the Experts™
on
I closed the previous related question:
http://www.experts-exchange.com/Virus_and_Spyware/Anti-Virus/Proxy-Firewall_Anti-Virus/Q_25091573.html

Because I received a great response that answered my question as stated.  However the two primary recommendations by the expert were based on erlang, which I am not comfortable jumping into at the moment.

His 3rd suggestion was Nginx which I'll give a +2, as he recommended it, and I have previous experience.

 I would very much like further suggestions from anyone with experience in this area.

Original Question:
I have my 'main' server located in a great facility.  However bandwidth is a bit on the pricey side there.

I have several servers at budget hosts, which I use for static content to reduce my bandwidth bill at the better facility.

Currently I'm using lighttpd and rsync on these hosts.  This is not optimal for a few reasons:
1. I'm 'caching' everything which I don't need to do, as only a small fraction of content is active at a given time.
2. rsync takes a very long time, as I have hundreds of thousands of small files that it needs to make sure are up to date.

I've come across the flowing caching proxies... some are specifically caching proxies, while others offer configuration options which offer the same end result.

Here are the contenders:
Nginx + 2
Varnish
Squid
Apache
Lighttpd
HAProxy
Unicorn

I would love feedback on even just one of these ideas, or any other caching proxy to help me narrow down the list.

Thanks,

 -Eric

Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Commented:
Highly recommend running varnish on your budget hosts. It will download content from the primary server automatically and cache the data so you don't have to worry about rsyncing all the time.

Author

Commented:
Thanks for your comment.

Do you have experience with Varnish and/or other proxies?  Why do you think varnish is the best of the options?

Thanks,

 -Eric

Commented:
I haven't actually used varnish yet, but I've been doing some research on it as I may be using it soon. Varnish is very fast (significantly faster than squid) and seems quite flexible in it's configuration.

The other options aren't proxies (although I don't know what unicorn is). So your proxy options are squid and varnish. I believe squid was written to be a forward proxy, whereas varnish was written to be a reverse proxy and this is what you're looking at setting up here.
Success in ‘20 With a Profitable Pricing Strategy

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden using our free interactive tool and use it to determine the right price for your IT services. Start calculating Now!

Author

Commented:
Thank  you for your help.  I would really like to hear from someone with experience deploying one of these solutions.

Best,

 -Eric

Author

Commented:
Thanks for the advice.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial