Request timed out - help!

Hmm it could be a slow database connection. May be you have a query which might often take long time to return

Hi

I've just had the same problem, was down to the asp_net worker process deadlocking, .NET 1.1 service pack 1 sorted it out

So that may be where to start

Cheers

Len

Actually, I say that; the users are getting "Page cannot be displayed most of the time".

I'll run a Sql Trace tomorrow morning.
len, we've already got SP1; good thought tho'.

Any other thoughts are welcome. Could it be something to do with a new caching class I added? This was a couple of weeks ago though.

The server as a whole was running slowly today, although it's only really got this one small app & Sql Server on it. Does this point to a db problem?

ryerras, I'm aware of the timeout setting, thanks -  there's no way it should be taking 90 secs!

I thought I'd been pretty rigorous closing readers and connections, and i mostly use MS's SqlHelper application block. I will have a look at the number of connections in Enterprise Manager tomorrow... but this isn't convincing me somehow. I don't know how it can have been fine for literally weeks, then break, then break again after an hour...

Well if you're using SQLHelper then only thing you should be looking from my view is "Closing DataReader".

-tushar

See why I'm not convinced it's the database connetions?... I'm really stuck.

Hi sumo_the_cat,

2 things:
1. are there other applications that are using sql server? they may cause a lot of work and that can be the reason your small application to be requested time out
2. is that problem accurs on every page or just on the pages that you are using this caching class you were talking about ?
3. is your database contains a lot of records? If you made a changes to your database may you have to update the statistics which will increase the performance of the sql server for this database ( exec sp_updatestats under your database ).

Regards!
B..M
mmarinov

thanks for the ideas bm.
1. No other app is using the db.
2. every page uses the caching class.
3. not many at this point. will do anyway. but the db can be accessed no problem (e.g. a large table opens fully) in ent manager with no delay.
pete.

sumo_the_cat,

it may be caused by this caching class
is the problem is going after you implemented in your application ?

Regards!
B..M
mmarinov

It worked fine for at least 2 weeks, and no problems on the development laptop. The strange thing is that, when we updated the application to force a restart, it worked, but only for an hour or so.

Here's a summary of the caching class. It's probably a complete misunderstanding of how to use the cache, but hey!

Imports System.Web.HttpContext
Imports Microsoft.ApplicationBlocks.Data

Public Class AppCache

#Region "Settings"

    'Thoughout the app, custom settings are accessed via the
    ' AppCache.Setting(key As AppCache.SettingKey.[settingKey]) method,
    Public Enum SettingKey

        '<!--  Database connection strings (main "Rehab" database and "Addresses" database) -->
        sqlConnectionString
        sqlConnectionStringAddresses

        '<!--  Application name & default settings  -->
        applicationName
        defaultPostcode
        defaultSearchRowCount

'etc
    End Enum

    Public Shared Function Setting(ByVal key As AppCache.SettingKey) As String
        'Normal method for retrieving a Setting.

        If Not Current.Cache.Item(key.ToString()) Is Nothing Then
            Return Current.Cache.Item(key.ToString()).ToString()
        Else
            '(Re)Cache the Settings if necessary. Settings should never be Nothing.
            CacheSettings()
            If Not Current.Cache.Item(key.ToString()) Is Nothing Then
                Return Current.Cache.Item(key.ToString()).ToString()
            Else
                Throw New ApplicationException("Setting '" & key.ToString() & "' is missing from the cache.")
            End If
        End If
    End Function
 
    Public Shared Function CacheSettings() As DataSet
        '(Re)Cache the Settings.

        'Read from the Settings.xml file.
        Dim ds As New DataSet
        ds.ReadXml(ConfigurationSettings.AppSettings("settingsFilePath"))

        '(Re)Cache every Setting.
        For Each setting As DataRow In ds.Tables(0).Rows
            Current.Cache.Item(setting("key").ToString()) = setting("value")
        Next setting

    End Function

#End Region

#Region "Users"
    Public Shared Function Users() As DataTable
        If Not Current.Cache.Item("users") Is Nothing Then
            '...Return the object from the cache.
            Return CType(Current.Cache.Item("users"), DataTable)
        Else
            '...Re-cache the Users, and return the object. Users should never be Nothing.
            CacheUsers()
            If Not Current.Cache.Item("users") Is Nothing Then
                Return CType(Current.Cache.Item("users"), DataTable)
            Else
                Throw New ApplicationException("'Users' DataTable is missing from the cache.")
            End If

        End If
    End Function

    Public Shared Function CacheUsers() As DataSet
        '(Re)Cache the Settings.

        'Read from the Worker table.
        Dim ds As DataSet = SqlHelper.ExecuteDataset( _
            AppCache.Setting("sqlConnectionString"), _
            CommandType.StoredProcedure, _
            "dbo.pWorker_GetRoles")

        '(Re)Cache Users DataTABLE.
        Current.Cache.Item("users") = ds.Tables(0)

        'As it may well be useful for diagnostics...
        Return ds

    End Function

#End Region

End Class
--------------------------------
Could it be the fact that I'm called a shared method of the class from within itself? I'm still new to OOP.
       'Read from the Worker table.
        Dim ds As DataSet = SqlHelper.ExecuteDataset( _
            AppCache.Setting("sqlConnectionString"), _
...

Pete.

sumo_the_cat,

how do you handle the exceptions? can you left a blank catch block where you call the Cache... methods?
also , have you checked what server does when a client request it? you can check if it connected with the sql by the Profiler tool of the SqlServer


Regards!
B..M
mmarinov

There shouldn't be any exceptions in the Cache... functions, apart from access the db, and the SqlHelper class has its own exception handling. Is this code ok though? Could it be causing the problem...?
We're just waiting to get a Sql trace running on the server.

Hi

Take a look here, this may be your problem

http://support.microsoft.com/default.aspx?scid=kb;en-us;313888

cheers

Len

len, didn't you say NET 1.1 service pack 1 sorted it out?

i know that sqlhelper has its own error handling but you throw exceptions that must be catched in the higher level ( Users table missing and
 Throw New ApplicationException("Setting '" & key.ToString() & "' is missing from the cache.") )

B..M
mmarinov

Just to add to the confusion, users are reporting they are now getting an authentication dialogue (win authentication), which then throws them out...! What's going on??!

bm, I'm sorry, I don't understand. Could you provide an example or something. Could this be causing the problems...?

just tell you to verify that everywhere you have a call to
Setting or Users method you catch exceptions and don't leave empty catch block

B..M
mmarinov

Ok, thanks BM.
I have identified another possible cause. I had forgotten that I was testing an idea to have the application broadcast a web service. I have deleted these from the web app, and copied the app to the server again, rebooted, and it is working again. I don't know how long for - this has happened before. But could this web service (one simple WebMethod, which no one can possibly have beeb calling) have been causing the time out??

yes it can
because there is a timeout for the connection and it can crashes


B..M
mmarinov

how exactly? pete.

are you creating a connection to this web service ? can you post a code ?

b..m
mmarinov

No, I had simply added a web service to the asp.net application while playing around, and forgotten about it. It wasn't calling anything. It was not much more complicated than the Hello World sample web method. it queried the db and returned a dataset. However, the server is behind a firewall - no one could have been calling it.

   <WebMethod()> _
    Public Function HelloWorld() As String
        Return "Hello World"
    End Function

    <WebMethod()> _
Public Function AllCurrent() As DataSet
        'Execute dataset.
        Return SqlHelper.ExecuteDataset(AppCache.Setting("sqlConnectionString"), _
            CommandType.StoredProcedure, "dbo.pService_Select")
    End Function

Unfortuantely i could come only with remove the webservice, rebuild, upload the project again and see what will happened. Sorry!

B..M
mmarinov

that's what i did, and it seems to be working ok now.. but for how long?...!

unfrotunatelly i can not say why this was a problem
i suggest you to not close the question and wayt 1-2 days and came if there is a problem
if it works - then close the question

B..M
mmarinov

Plan! Pete.

We had this problem to when our server was taking too long to do things, ie the default setting ASP.Net is to Timeout any single code execution that takes longer than 90 seconds.  So we had a query that used to take 2 minutes to run and the request would time out.  If you want to override this setting that is in your machine.config, just for your application you can add the following line to the <system.web> section of your web.config file in the root of your ASP.Net Application :

     <httpRuntime executionTimeout="180" maxRequestLength="4096" useFullyQualifiedRedirectUrl="false"
minFreeThreads="8" minLocalRequestFreeThreads="4" appRequestQueueLimit="100" />

and change the executionTimeout to whatever value that you want in this case I use 180.

H.T.H

i know it's along thread, but see near the top. pete.

Whoops, sorry.  Yep it is a bit of a long thread, don't know how I missed that.

Hi anyone that's still listening,
Guess what?! Page cannot be displayed again this morning... Still open to suggestions...

grrrrrrrrrrrrrrrrrr, this is unnormal
Just a shot, but try to remove the content for your application from the asp.net temp directory
%system root%:\%windows directory%\Microsoft .NET\Framework\v.1.1.xxxx\Temporary ASP.NET Files\your web application directory name
if you have no other web applications you can remove all the content of
%system root%:\%windows directory%\Microsoft .NET\Framework\v.1.1.xxxx\Temporary ASP.NET Files\

HTH
B..M
mmarinov

P.S.: are there any errors in the event log from the web server ?

Do you ever sleep BM? I'm waiting until the network guy gets in this morning to send me the Sql Trace. Will ask him to remove the temp stuff.

And you don't think it can be to do with the caching class above? I'm loathed to remove it as it means going through the entire app removing referrences to it.

:) i sleep sumo. actually i'm from bulgaria an it is 12:00 AM here
as for question -> unfortunatelly i can not say it -> obviously it cause something and it can be understand what if you remove it from the application and check if it is working

you have to look if there are statements in your code that fill a lot of cache, because it can be causing a lot of HDD space to be used
what kind of data do you cache?

B..M
mmarinov

a few strings, a tiny datatable or two.. memory is not the problem. we have a brand new, dedicated win 2k3 server, running ONLY this small web app and sql server.

but are you caching it for every user or for the complete application ?

B..M
mmarinov

application. (see code i posted.)

sorry, i forgot it :(
B..M
mmarinov

Have just heard from network guy. He says he can actually open the app but it has become very very slow. We have a test application called TestApp on the server (just a very simple asp.net app which queries the Northwind db) which opens immediately. There is no problem with the SQL database itself, which can be opened instantly in enterprise manager. It is only the main app which is slow. I will try to remove the caching class and see if that helps....

Looking at the Sql trace, the last command against the db was 5pm last last. A user reported at 8am this morning that the page could not be displayed. The next command against the database was achieved at 9am. This means that, over night, the application itself has slowed down to the point where it times out. Nothing to do with the database.

have you execute
exec sp_updatestats for you database in the query analyzer ?

B..M
mmarinvo

thanks for reminding me!

Ok BM, if you're still with me. I just got the guy to delete the entire application folder, copy and old version from last month in there, reboot the web server. It still does not work. Sits there for ages, then page cannot be displayed. The horrible thing is that the TestApp application is still fine. What on earth could be causing this? I'm ready to wipe the server.

BTW, the version in dropped in place just before reboot was prior to the caching class, so we can ignore that now.

have you remove the temp directory's content ?

B..M
mmarinov

yes!

how do you connect to the database server ?

B..M
mmarinov

Using SqlHelper, mostly. Connection string is:
            <add key="sqlConnectionString" value="Data Source=(local);Initial Catalog=DbName;Integrated Security=SSPI" />

I have yet to see whether this database can be accessed fine from another web app, which I will do tomorrow.

BM, I'm tryig to think of unusual things my app does. Here's one: It uses custom authorisation with Windows authentication. The users' windows usernames are stored in the SQL database along with their custom roles (e.g. "readUser,writeUser,adminUser").  Here's the code in Global.asax :


    Sub Application_AuthenticateRequest(ByVal sender As Object, ByVal e As EventArgs)
        ' Fires upon attempting to authenticate the user

        'Assign custom roles to the user.
        AssignUserRoles()

    End Sub

#Region "Assign user roles"
    'Assign the custom roles to the user. This is done on every request.
    Private Sub AssignUserRoles()

        'Only do this if user is authenticated.
        If Not Request.IsAuthenticated Then
            Throw New ApplicationException("Access denied. But how did you get this far?!")
            Return
        End If

        'Get the essential Windows domain + group. (e.g. "WINDOMAIN\AppUsers")
        Dim windowsDomainAndGroup As String = AppCache.Setting(AppCache.SettingKey.windowsDomain) _
            & "\" & AppCache.Setting(AppCache.SettingKey.windowsGroup)

        'If the user is not in the essential Windows group initially, NO CHANGES will be made to her Roles.
        If User.IsInRole(windowsDomainAndGroup) Then

            'Get the user's custom roles from the cached "users" datatable, and
            ' try to assign the user the roles from it.
            Dim users As DataTable = AppCache.Users
            '
            Try
                'Get the value in the "Roles" column of the first ((0) - and only) DataRow
                ' in an array of DataRows produced from Selecting from the "users" DataTable
                ' where "Username" = the user's name.
                ' If the username doesn't exist, an IndexOutOfRangeException will occur as there
                ' will be no DataRows in the array. (Thrown below.)
                Dim customRoles As String = users.Select("Username = '" _
                    & CustomSecurity.Username(User.Identity.Name) _
                    & "'")(0)("Roles").ToString()

                'Assign the roles to the user, including the essential Windows group
                ' (as this is not retained when the User is replaced by a New GenericPrincipal).
                HttpContext.Current.User = New System.Security.Principal.GenericPrincipal( _
                    Context.User.Identity, (windowsDomainAndGroup & "," & customRoles).Split(Convert.ToChar(",")))

            Catch ex As System.IndexOutOfRangeException 'username doesn't exist in cached datatable.

                'Update "users" datatable (used to store their custom roles)
                ' in case it is an old cached version which is causing the problem.
                ' They can then refresh to try again.
                AppCache.CacheUsers()

                'Throw custom exception, with advice about what to do.
                Throw New ApplicationException("Access denied. Please try the following: " & Environment.NewLine _
                    & "1. Press the Refresh button on your browser. " & Environment.NewLine _
                    & "2. Log out of Windows and try again. " & Environment.NewLine _
                    & "3. Have an ""adminUser"" configure your account correctly, using your exact Windows username.")
            End Try

        End If 'User.IsInRole(windowsDomainAndGroup)

    End Sub
#End Region

-------
Any problems with this? Should I be using WindowsAuthentication_OnAuthenticate instead of Application_AuthenticateRequest ? Thanks, Pete.

i don't see any problems with this, but today i won't be able to test it within the onauthenticate event
you can check it and you can also check the code through the Session_Start

B..M
mmarinov

it's been working fine for a couple of months... I'm just trying to spot significant differences between the main application (which is failing) and the little test application (which is working fine). pete.

are there any changes on the web server ( updates, security changes, anything ) from the time it works to now ?

B..M
mmarinov

No! Not as far as I know. Oh, they did open port 80 outbound a week or two ago, and enable SMPT. This was so I could access web services from the app, but I haven't done so yet. One thing (I'm clutching straws here) is that the SMPT was enabled so my custom exception notifier class would email me any exceptions. For some reason, none have been sent. These couldn't be "clogging up" the server could they?

I'm travelling to see the server tomorrow. We did .NET SP1 about a month ago; no problems.
One thing I'm going to do is increase the timeout setting and turn on application tracing. It's absolutley beyond me at the moment.

i suggest you to try to close for a while 80 port and see if there is some change, also the problem can be caused by your emailing

B..M
mmarinov

but how? and why would it only affect this one app?

i don't know how was configure and what you doing to send emails and unfortunately i can not say

B..M
mmarinov

I just used the .NET Smtp class to send the emails. It worked fine on my laptop, but they never arrived from the production app on the server, which made me assume that there was something more that needed to be configured in IIS on the server. Is there any way an old "queue" of emails generated by this application could build up and cause it (but no other application) to slow down (even though it's not trying to send emails any more because I disabled the email functionality a few weeks ago)? It's a ridiculously long shot, but I'm trying to think of anything here.

have you enable to relay the message outside of the smtp server ?

B..M

i don't think so - i don't know what that is...?

this is configuration of the smtp on the server
check this article: http://support.microsoft.com/kb/324281
when you go to the Relay mark All Except The list Bellow

B..M
mmarinov

i don't really care about the emails not working per se, only if it could possibly be causing the slowdown. could it really? but thanks.

Don't fully inderstand this, but the network guy just told me he discovered that an IP address was entered wrong on the server. Everything's working fine now he's fixed it! So it looks like a DNS error was causing the timeout.......! I'd like to be more specific - this looks a really tricky problem for developers like me because it doesn't throw up any ASP.NET error - just times out for no reason. So I'll try and get some more specifics from him. He also seems to think that it explains why one application on the server worked but another one didn't - again, I don't really understand how DNS works exactly so I can't provide you any more info right now.

Thanks so much everyone, expecially BM - this has been a long, frustrating week and and an equally long thread. I'm going to award points of course for all the input; the answer is here in this comment for future searchers.

Pete.

I was facing connection timed out on deployment server, but not on my local system.

I did following to fix this problem. I added "Incr Pool Size", and removed "Max Pool Size" and it worked.

E.g.

 <add key="ConnectionString" value="Incr Pool Size=2;User ID=hendcama;Password=hendcama;Data Source=hendcama" />