Data Caching, SyncLock Applications, Expiry and HELP!
Posted on 2006-05-27
In way over my head.... I have a asp.net 2 web application that has some problems - most using cached datasets/tables. I have implemented data caching that loads some very frequently-used tables into cache. Essentially, the web app serves a number of domains and needs different settings/content for each - and much of that content is contained in the cached datatables/datasets but the datasets are not big but can be accessed 20-30 times in the life of a page request for a complex page.
The data caching was simply set up to hold the datasets/tables in cache and then expire them after 30 minutes - so I could reload any new settings/content. When in production under load the app had serious problems - I think related to race conditions when items in the cache expired - so I added created a LoadCache class and added a SyncLock on an object around the cache insert procedures to stop instances/threads trying to add stuff at the same time. But the problems remain it seems because multiple threads are trying to READ the cache object, then find the data doesn't exist (because it expired), then requests a load of the cache (and finds the lock on it) while another thread has already issued the request to load it. I am not sure this is what is actually happening as it really hard to trace/debug. In any event I suddenly get errors galore as threads somehow can't see the cached data.
Here is summary of what I've done:
private shared cachelock as new object
public sub loadcache(tblname as string)
'...get the table from db
httpcontext.current.cache.insert(tbl) 'the table is actually in its own dataset
then elsewhere I have:
dim ds as new dataset
if isnothing(ds) then
dim c as new cacheloader
I suspect that there is an issue with the scope of the cacheloader cachelock object (surely this is private to each instance of cahceloader?) and thus not a functioning lock? Should cacheloader be declared globally somewhere?
All that aside, I have come to the conclusion that the really-frequently accessed data needs to not expire in cache cuz I just can't seem to get it to work properly under load and keep everything synched. There is a good possibility that I haven't created and used the LoadCache class properly
Now to the question: if I load the datatable/sets into Application (i.e. as an Application("mydataset")) then when does this get refreshed? I have multiple worker processes configured - do each of these use the same Application object? If I load the dataset in Application during Application_Start, will each worker process run the Application_Start and have it's own copy of Application, or is it global/shared to all the worker processes. I ask this as I have the worker processes set in IIS to restart every 1/2 hour - and that would be an OK interval to reload this data.
Is there any worker process/instance specific way of loading this data so as each worker process stops/restarts, it gets its own (current) copy of data and loads it?
Sorry for the rambling... What I thought was a good idea (using expiring cache for extremely frequently accessed data - sometimes 10+ times per page request) is not as easy as it would appear from MS docs. In fact, in no example of data caching I have seen from MS does they even mention the need to SyncLock to avoid multiple threads modifying the cache.
And... before anyone suggest sql data dependencies: I don't want the complexity of this. If I can't get simple expiring caches to work, I can't imagine having to try to debug sql data dependencies....
Thanks for any help.