I closed the previous related question:
Because I received a great response that answered my question as stated. However the two primary recommendations by the expert were based on erlang, which I am not comfortable jumping into at the moment.
His 3rd suggestion was Nginx which I'll give a +2, as he recommended it, and I have previous experience.
I would very much like further suggestions from anyone with experience in this area.
I have my 'main' server located in a great facility. However bandwidth is a bit on the pricey side there.
I have several servers at budget hosts, which I use for static content to reduce my bandwidth bill at the better facility.
Currently I'm using lighttpd and rsync on these hosts. This is not optimal for a few reasons:
1. I'm 'caching' everything which I don't need to do, as only a small fraction of content is active at a given time.
2. rsync takes a very long time, as I have hundreds of thousands of small files that it needs to make sure are up to date.
I've come across the flowing caching proxies... some are specifically caching proxies, while others offer configuration options which offer the same end result.
Here are the contenders:
Nginx + 2
I would love feedback on even just one of these ideas, or any other caching proxy to help me narrow down the list.