Wireless connection but page cannot be displayed

I recently got verizon dsl.  I ordered the wireless router and installed it.  I have 5 computers on my network, one is hardwired to the router, and the other 4 are wireless.  Anyway, it was a westell router and the internet connection setup fine, but it would intermittenly lose connection.  After researching this online I found that this was a known problem with the westell routers, so I had Verizon send me a regular dsl modem and I went out and bought a different wireless router.  I set this up last night.  I now have internet connection on the wired computer and also on the wireless laptop (that came with a built-in card) however the 3 desktops will not connect to the internet.  I have a strong wireless signal and I know the network is working because I can access the router from one of the wireless desktops, I just can't get on the internet I get a page cannot be displayed error.  Two of the machines are Windows xp and one is Windows 2000.  I was on the phone with linksys last night and they told me to call the wireless card manf.  Well this doesn't make sense to me because I had internet (although it bounced) connection using the other westell router.  Everything appears to be functioning properly, can anybody shed some light on this???