VPN Working - Web Browsing not

I have a dedicated web server behind a firewall at a major hosting provider.
The firewall offers a VPN connection, which has been setup to talk to my linksys through the VPN.
The VPN portion is working fine.
From my network, I can browse anywhere on the web I want to except to the public IP's of my web server.  Those time out.  I can browse to the VPN IP's without any issues.
The outside world can browse my web server on the Public IPs with no issues.

Question:  Is this how it should work?  I would like to be able to browse on the public IPs liike everyone else.