IIS - randomly doesn't render graphics in a virtual directory

Background:
Our business model has multiple clients, all using a core set of code but customized configuration / look and feel.  The code was developed on the ASP.NET 2.0 framework.  To achieve this, we coded the core routines in a web project, called Core.  Each client also has a unique folder to house their graphics, CSS, and unique JavaScript.  Within IIS we create the clients website name as a virtual directory pointing to the Core project, then under that create a virtual directory called Resources that points to the unique client folder.  This allows a consistent way for the core routines to link to the resources, without knowing the specific client details.  This environment is then replicated across a web farm of many servers.  There are no ISAPI filters running, and all machines are configured the same.

Example of Physical Directory
\wwwroot
\wwwroot\Core
\wwwroot\Core\App_Code
\wwwroot\Core\SomePage.aspx
     &etc&
\wwwroot\Client1_Resources
\wwwroot\Client1_Resources\SomeGraphic.jpg
     &etc&
\wwwroot\Client2_Resources
\wwwroot\Client2_Resources\SomeOtherGraphic.jpg
     &etc&

Example of Virtual Directory
http://ourcompany.com/Client1   --> \wwwroot\Core
http://ourcompany.com/Client1/Resources   --> \wwwroot\Client1_Resources
http://ourcompany.com/Client2   --> \wwwroot\Core
http://ourcompany.com/Client2/Resources  --> \wwwroot\Client2_Resources


Problem:
We are getting inconsistent resource rendering results over time, across random servers.

Details:
At any given time, all web servers will render correctly.  For no apparent reason, one will suddenly NOT render a page consistently.  The CSS will not be loaded, or one or more of the graphics will not show up.   Hitting refresh on the page will continue to provide sporadic results, sometimes rendering the page entirely, and other times having one or more errors with the graphics, CSS or both.  After reviewing this in HttpFox, we are seeing the error NS_ERROR_NET_RESET, which implies the connection was established, but no data was delivered.

Thinking this was a .NET problem, we removed that by taking a View Source, and saving it as a static HTML file.  We continued to get inconsistent resource rendering even with the static page.  In our mind, this removed .NET as the culprit.

Continuing then with just the static file, we created test scenarios.  The current file used relative path names for all resources, and relative names within the CSS file since there was no guarantee we would be on any specific server within the web farm.  

1.      The first scenario was to make the URLs to the resources fully resolved (ex: https://server1.ourcompany.com/client_website/resources/image.jpg) both within the CSS and anywhere within the HTML page.  This came up with the same inconsistent results.
2.      The second scenario was to use relative URLs to the resources within the CSS file, and fully resolved within the HTML page.  Same inconsistent results.
3.      The third scenario was to take the virtual directory resources out of the picture, and use fully resolved paths to the client specific folder.  This worked correctly 100% of the time.
4.      The fourth scenario was to repeat the third, but with relative paths.  This again worked correctly 100% of the time.

We concluded that this must be a problem with either ISS or how we structured our Virtual Directories.  

The Kicker:
If we reboot a server that is suffering from this problem, it resolves itself.  Over time, it can happen to any server within our farm, with the same inconsistent rendering issue, that can be resolved with a  reboot.  

Obviously rebooting as a corporate methodology cannot be an acceptable answer.  Any help would be appreciated.