Proxy to multiple servers (getting /images to work)

hmm, did you miss the trailing / at the path?

ProxyPass /server1/ http://server_1.internal.mydomain.com/
ProxyPassReverse /server1/ http://server_1.internal.mydomain.com/

No, I am almost certain that I've tried that as well. (Unable to try it at the moment as I am at a different location)

Let me clarify:
On the internal servers there is an index.html file.
if the user goes to: www.mydomain.com/server1/ the proxy gets the index.html of server1.internal.mydomain.com.
However if there are links to images in index.html and these links start with i.e. /images/ it will not be displayed.
I have found that if I add:
ProxyPass /images http://server1.internal.mydoain.com 
it works fine.
But how do I solve it for server2 in that case?

did you set in httpd.conf:
  options FollowSymLinks

No difference

M,

This might be a typo.  But notice the 2nd ProxyPassReverse.  Is it suppose to be /server2 ?

ProxyPassReverse /server1 http://server_1.internal.mydomain.com/

   ProxyPass /server2 http://server_2.internal.mydomain.com/
   ProxyPassReverse /server1 http://server_2.internal.mydomain.com/

Have you consider using mod_rewrite along with mod_proxy?  

How about using different VirtualHost and have ProxyPass/ProxyPassReserve in each to proxy the two different internal server.  This approach would work but instead of using www.domain.com/site1/ and www.domain.com/site2/, you might end up with site1.domain.com, and site2.domain.com with each would have;

<VirtualHost *>
ServerName site1.domain.com
ProxyPass / http://internal1.domain.com/
ProxyPassReverse / http://internal1.domain.com/
</VirtualHost>

<VirtualHost *>
ServerName site2.domain.com
ProxyPass / http://internal2.domain.com/
ProxyPassReverse / http://internal2.domain.com/
</VirtualHost>

I woulld believe rewriting would work, but would be a bit tricky.  Perhaps you might want to look at Rewriting Guide on Apache docs.  Try to jump to section that talks about "Dynamic Mirror",  and/or "Reverse Dynamic Mirror", and/or "Reverse Proxy".  It may/not be applicable, but worth to look at.

http://httpd.apache.org/docs/misc/rewriteguide.html

cheers;

Yes it was just a typo.
I have been looking at the rewrite module and are currently playing around with that.
Haven't got it to work though. The Dynamic Mirror sections seemed to be what I am looking for...well actually, I hope is what I'm looking for ;-)

M,

good luck.  

mod_rewrite is claimed to be quite powerful+flexible, if and only if one could get the rewrite condition+rule, otherwise it you would be spending long hours...

I am a bit surprise to why your initial config does not work.  It looks fine (except to the trailing slash as pointed by ahoffman).

Perhaps you might want to spend a few minutes on the Vhost approach.  I tried to do the same with EE, and microsoft (to so called accelerate it), and it works fine.  The catch is, it wont't work for hardcoded URL.

cheers.

Well I am a bit surprised it didn't work as well. I am a total newbie at this apache configuration thing, so I do not quite understand why it works when I do:
ProxyPass / http://server1.internal.mydomain.com/
and doesn't work when I do:
ProxyPass /server1/ http://server1.internal.mydomain.com/

Guess I'll have to work a bit more with the rewrite module and get that to work.

I am going to try the Vhosts.

Raising it to 300 points as well.

M,

The only possibility that I can think of on why

ProxyPass /server1 http://server_1.internal.mydomain.com/
ProxyPassReverse /server1 http://server_1.internal.mydomain.com/

does not work is that the link in HTML documents in the internal server (http://server_1) contains hardlink - ie <a href=http://server_1.internal.mydomain.com/somepage.html>A link</a>, compared to <a href=/somepage.html>A Link</a>.

Some would argue that mod_rewrite will be a bit slow (since some additional overhead is needed on the server side).  But it should work.

If you feels like looking at Rewriting, take a peek at;
http://httpd.apache.org/docs/misc/rewriteguide.html

Or jump to "Reverse Proxy" section.  There is a complete sample config that you could take a look.  Just make sure you had mod_rewrite.so loaded (in AddModules directive).

cheers.

Samri,

the main problem is when pages on server1 and server2 uses
/images/<imagename>.gif to display images, then all the users sees is broken links.
I'm having trouble understanding why at the moment
I guess it looks for images on www.mydomain.com under /images since if I add an proxypass for /images it works fine. But then, as I said, how do I know from which server (1 or 2) to fetch the image.

I'm reading the rewriting guide now...hope that helps me

/M

M,

Hmm.... that is a very interesting question.  And yes (to my knowledge) it would come from www.domain.com, but if the link is image/someting.gif (in HTML code), then it would be translated fine by ProxyPass and ProxyPassReverse.

Try the mod_rewrite, and see if that fixed the images portion.  Other than that. I would presume that using a Vhost for each accelerated server would solve the problem since there would be an assurance that /image for internal_server1 would always be coming from internal_server1.

cheers.

samri,

the vhosts doesn't quite work either, mainly because I want the users only to go to www.mydomain.com and then click on links.

1. User goes to www.mydomain.com
2. user clicks on the link server1
3. apache fetches server1.internal.mydomain.com
4. content of the server1 is displayed to the user
5. to the user it looks like he is at: www.domain.com/server1/

still working on rewrite though...trying to get that to work. The examples didn't work (or rather, I am new to it so I've probably missed something)

/M

hmm, been playing with the rewrite module while reading the URL Rewriting Guide. Now, the dynamic mirror part seems to be exactly what I want to do, but I am obviously missing something here.
I've changed my virtual host to this:
<VirtualHost *>
     UseCanonicalName On

     RewriteEngine  on
     <Directory / >
          RewriteBase     /news/
          RewriteRule     ^/(.*)$                    http://www.cnn.com/$1     [P]
          RewriteCond     /news/$1               -U
          RewriteRule      ^http://www\.cnn\.com/(.*)$     /news/$1
     </Directory>

</VirtualHost>

and if I understand it correctly, now when I go to the proxy and enter news/ I should see the content of cnn.com.
However, all I get is :
Not Found
The requested URL /news/ was not found on this server.

Any ideas?

Thanks in advance

/Markus

Markus,

I am not very good in rewriting myself.

Tried this example, and it worked.  well mine is running on win2k.


<Directory "C:/Program Files/Apache Group/Apache/htdocs">
    Options Indexes FollowSymLinks MultiViews
    AllowOverride None
    Order allow,deny
    Allow from all
    RewriteEngine  on
    RewriteBase    /
    RewriteRule    ^stokely/(.*)$  http://www.stokely.com/$1  [P]
</Directory>

And the requested URL is: http://localhost:8000/stokely/index.html

try to stay away from Vhost at the moment (or maybe you might give it a shot anyway).

The reason why I picked Stokely is that I am pretty much comfortable with the fact that the site doesn't have much external link, which might render undisplayed link (or images).

Hope this gets you going.

cheers.

Samri,

I got it to almost work with the following:

RewriteCond %{HTTP_REFERER} .*/news/.*
RewriteRule ^/(.*) /news/$1
RewriteRule ^/news/(.*) http://www.cnn.com/$1
ProxyPassReverse /news/ http://www.cnn.com/

I am not at the place where I tried this so this is as I recall it.
It works pretty good, but I need to fine-tune it 'cause the HTTP_REFERER value changes at a point, that is the rewrite-rules don't work in some places. Have to debug it further tomorrow.

Will try what you did as well, to see how that works.

thanks

/M

Markus,

That is the beauty (or the danger) of apache rewriting engine.

Your example might work.

some people would discourage the use of HTTP_REFERER variable since it might not be availabe in some sessions.  

give it a shot.  I would be interested to know too..

regards.

Samri,

This is what I've come up with now, it works ok if, and it's a big if, the remote server html-pages are coded in a good way.

I noticed that the referer header got re-written if the link on the page did -not- start with an /
ex:
 <a href="index.html"
compared to:
 <a href="/index.html"
the latter works fine, the former doesn't (well the page will be displayed but links to /images/*.gif will not)

anyway, here is part of my httpd.conf:

<VirtualHost *>
    UseCanonicalName On

    RewriteEngine On
    RewriteLog logs/rewrite.log
    RewriteLogLevel 0
   
   
#Disable all caching
    NoCache *

    ProxyVia Block

# Rewrite/ProxyPassReverse for server www.cnn.com "located" in /_news/
    ProxyPass /_news/ http://www.cnn.com/
    RewriteCond  %{HTTP_REFERER} ^.*/_news/.* [NC]
    RewriteRule  ^/(.*)  /_news/$1 [C]
    RewriteRule  ^/_news/(.*) http://www.cnn.com/$1 [P]
    ProxyPassReverse /_news/ http://www.cnn.com/
</VirtualHost>


You are right regarding the HTTP_REFERER (as I mentioned above). What I would like is to somehow get the remote server address, or a way to save it in an enviroment variable. I looked at the SetEnv but couldn't quite figure out how to make it work. (I mean in conjunction with RewriteRule, SetEnv was simple enough ;-)

well...it feels like I've gotten about as far as I can on this. Unless you, or anyone else, has some brilliant way to make it fool proof :)

regards
/Markus

I noticed that I get: /_news/_news/ sometimes, so I added:
RewriteCond  %{REQUEST_URI} !^/_news/.* [NC]
after:
RewriteCond  %{HTTP_REFERER} ^.*/_news/.* [NC]


------------------------------------
ProxyPass /_news/ http://www.cnn.com/
RewriteCond  %{HTTP_REFERER} ^.*/_news/.* [NC]
RewriteCond  %{REQUEST_URI} !^/_news/.* [NC]
RewriteRule  ^/(.*)  /_news/$1 [C]
RewriteRule  ^/_news/(.*) http://www.cnn.com/$1 [P]
ProxyPassReverse /_news/ http://www.cnn.com/


/M
 

Markus,

There goes the *newbie* in mod_rewrite.  To be honest, you far exceed my expectation for somebody new to apache. (gee... looks like I am learing more than what I initially tend to...  umm... help :)

anyway,I will be scanning for anything that I could jump in and give a hand.  At the moment, I'm flipping the mod_rewrite manual (and kinda learnign from you.).

There is another Q that is closely similar to yours which I happens to be *learning* too.
https://www.experts-exchange.com/questions/20319552/Yet-another-proxying-question.html

cheers.

Well
I have solved my problem (right now actually :)
.....funny, when you're stuck in a line of thought...it's hard to break it ;-)

I happend to see an example on virtual hosts and that helped me. (I had made some error when I tried it previously, which is the reason why I didn't think it would work)

anyway, this is what I did:

NameVirtualHost IP-number
<VirtualHost IP-number>
    ServerName proxy.mydomain.com
</VirtualHost>

<VirtualHost IP-number>
    ServerName bogus-name1.proxy.mydomain.com
    ProxyRequests on    
    ProxyVia On
   
#Disable all caching
    NoCache *

   ProxyPass / http://back-endserver1.mydomain.com/
   ProxyPassReverse / http://back-endserver1.mydomain.com/
</VirtualHost>

<VirtualHost IP-number>
    ServerName bogus-name2.proxy.mydomain.com
    ProxyRequests on    
    ProxyVia On
   
#Disable all caching
    NoCache *

   ProxyPass / http://back-endserver2.mydomain.com/
   ProxyPassReverse / http://back-endserver2.mydomain.com/
</VirtualHost>

I also added these bogus names in the hosts file
and then just edited the .html-file to
<a href="http://bogus-name2.proxy.mydomain.com/">News</a><br>

works!

I learned alot from this though, just hope I can remember all this Rewriting stuff...pretty cool :)

Anyway, thanks for the input
and hope this helps the other guy to...

markus,

It's great to hear that it finally gets somewhere..

sure hope that Lars catches this one too.

Since you technically solved you own problem; at this stage; you are left with two options (do I sound like a State Trooper :), you can post a quesstion in Community Support (https://www.experts-exchange.com/commspt/) to request for a deletion to the Q (which I would object), or get your pts refunded, by getting the Moderator to reduce the pts to 0, and that way the invaluable information is kept as PAQ, or (here comes the best part), you could award the member who you feels like awarding.  You can split pts too, ask the Moderators.

gee.. checked you profile -- an EE veteran! just ignore my comment :)

cheers.

hahaha, a veteran? yes maybe, but only date wise ;-)
anyway, thanks again, I'll contact a Moderator and PAQ it.

cheers
/Markus

samri
I'm going to award you some points since your input was very valuable. See topic area.

Markus,

Yep... I got it (https://www.experts-exchange.com/questions/20322005/POINTS-FOR-samri.html)

I just wished that I could be of more help.

Anyway, many thanks for the pts.


cheers.