Link to home
Start Free TrialLog in
Avatar of ccbailey
ccbailey

asked on

System.OutOfMemoryException Error when Using Datagrid to Display Data

I have a page with several list/menu drop down boxes in which the user selects criteria.  Then, upon clicking "Submit", the criteria is passed to a aspx page with a datagrid which displays the data using the user's criteria as parameters for the dataset.  When the user selects to see all the data (rather than selecting particular criteria), they get either no results or only a few results, and at the top of the page they get the following error message:

System.OutOfMemoryException: Exception of type System.OutOfMemoryException was thrown.

The query is not particularly large (less than 800 items), and I have successfully displayed all values of queries that were much larger (I have one that is over 2800 records).  All I can figure is that the query is too complex, since it evaluates six different parameters and involves several tables in the database.  However I do not know how to get around it.  

Here is my query, if it is any help:

SELECT [Table-Sections].FacilityID, [Table-Facility].FacilityName, [Table-Facility].SchoolLevel, [Table-Surveys].FeederZone, [Table-Sections].SectionID, [List-Sections].SectionName, [List-Sections].SectionSort, [Table-Sections].SystemTypeID, [List-SystemType].SystemType, [Table-Sections].YearInstalled, [Table-Sections].ExpectancyID, [List-Expectancy].Expectancy, [Table-Sections].SquareFootage, [Table-Sections].CostperSqFt, [Table-Sections].SlopeID, [List-Slope].Slope, [Table-Sections].History, [Table-Sections].[Photo-Footprint], [Table-Sections].[Photo-Overall], [Table-Sections].Cost, [Table-Sections].SummaryCost, [Table-Sections].WarrantyPeriod, [Table-Sections].ManufacturerName, [Table-Sections].RoofingContractor, [Table-Sections].EstimatedRepairYear
FROM ((([Table-Surveys] RIGHT JOIN ([List-Sections] RIGHT JOIN ([Table-Facility] RIGHT JOIN [Table-Sections] ON [Table-Facility].FacilityID = [Table-Sections].FacilityID) ON [List-Sections].SectionID = [Table-Sections].SectionID) ON [Table-Surveys].FacilityID = [Table-Sections].FacilityID) INNER JOIN [List-SystemType] ON [Table-Sections].SystemTypeID = [List-SystemType].SystemTypeID) INNER JOIN [List-Expectancy] ON [Table-Sections].ExpectancyID = [List-Expectancy].ExpectancyID) INNER JOIN [List-Slope] ON [Table-Sections].SlopeID = [List-Slope].SlopeID
WHERE ((([Table-Facility].SchoolLevel) Like '%'+?+'%') AND (([List-Sections].SectionName) Like '%'+?) AND (([List-SystemType].SystemType) Like '%'+?+'%') AND (([Table-Sections].YearInstalled) Like '%'+?+'%') AND (([List-Expectancy].Expectancy) Like '%'+?+'%') AND (([List-Slope].Slope) Like '%'+?+'%'))
ORDER BY [Table-Facility].FacilityName, [List-Sections].SectionSort

The query works successfully if you drill down enough (to where only one or two records are displayed), but not if all or many records are queried.  Any help would be greatly appreciated.

Thanks,
Christy
Avatar of AerosSaga
AerosSaga

please post your entire routine
You don't have over 3gigs of ram on the server do you?
Another issue could be a memory leak, see here:
http://www.groupsrv.com/dotnet/viewtopic.php?p=207245
More Information

Theoretically, the maximum file upload size is fairly large. However, because of ASP.NET health monitoring, you cannot upload very large files in ASP.NET. The ASP.NET worker process has a virtual address space of 2 gigabytes (GB). However, the ASP.NET worker process only uses a little more than 1 GB because of health monitoring and memory fragmentation.

During the upload process, ASP.NET loads the whole file in memory before the user can save the file to the disk. Therefore, the process may recycle because of the memoryLimit attribute of the processModel tag in the Machine.config file. The memoryLimit attribute specifies the percentage of physical memory that the ASP.NET worker process can exhaust before the process is automatically recycled. Recycling prevents memory leaks from causing ASP.NET to crash or to stop responding.

Additionally, other factors play a role in the maximum file size that can be uploaded. These factors include available memory, available hard disk space, processor speed, and current network traffic. With regular traffic of files being uploaded, Microsoft recommends that you use a maximum file size in the range of 10 to 20 megabytes (MB). If you rarely upload files, the maximum file size may be 100 MB.

    Note: You can upload files that are larger than 100 MB in ASP.NET. However, Microsoft recommends that you follow the maximum file upload sizes that are mentioned in this article. To determine more precise file sizes, perform stress testing on computers that are similar to the ones that will be used in production.

You may notice the following error messages if you encounter file size limits during the file upload process:

    * The page cannot be displayed.
    * Server Application is Unavailable

In the event log, the error message will be similar to the following:

aspnet_wp.exe (PID:PIDNumber) was recycled because memory
consumption exceeded the SizeLimit MB (Percentage percent of
available RAM).

# Exception of type System.OutOfMemoryException was thrown.

You may also find that uploads occur very slowly. If you watch the Aspnet_wp.exe process in Windows Task Manager, you will notice that the memory delta changes by 64 KB every 1 to 2 seconds. Depending on the size of the file, this delay may cause the ASP.NET worker process to recycle because of a responseDeadlock error.

An OutOfMemoryException means that you ran out of space in the heap, not
that you have no memory available on the machine. When you create a new
object, the CLR makes a request for a contiguous block of memory in the
managed heap. It does this by checking the pointer for the next object on
the heap and then determining if there is enough memory on the heap for the
new object you are creating. If there is no memory available on the heap
(we're talking contiguous memory) and there are no objects that can be
collected to free contiguous space, you will receive an
OutOfMemoryException.

The garbage collector will compact memory as necessary when it collects
objects in order to keep as much free contiguous memory available in the
heap, but if you have large objects in the heap (objects that are over 85K)
and you're running ASP.NET 1.0, the heap won't be compacted. If you are
running ASP.NET 1.1, we have a new generation (generation 3), and gen3 is
only for large objects. Gen3 does get compacted, but very rarely.

Bear in mind that an object is only going to be collected and the
associated heap compacted if the object is not rooted.

You could also be running into a native leak somewhere. If that's the
case, when the CLR attempts to get another 64K block of memory for the
heap, it cannot.

The only way to determine for sure what is causing this is by getting a
memory dump and examing it in a debugger to see what's going on.
You can download the Debugging Tools for Windows at this location: http://www.microsoft.com/whdc/ddk/debugging/installx86.mspx

Included in this toolkit is adplus, a VBScript file that will allow you to
attach the debugger (cdb) to the aspnet_wp.exe process. By default, adplus
will only allow you to get a dump when a crash occurs or manually via hang
mode. In your case, you'd need one at the moment that the
OutOfMemoryException occurs . You can do that by setting a breakpoint via
a configuration file, but first you have to know the token number of the
exception in the version of mscorwks.dll or mscorsvr.dll.

After you get the dump, you open it in WinDbg, another tool that is part of
the debugging tools. WinDbg is designed to be a native debugger, but you
can (assuming you're using the 1.1 version of the Framework) use SOS, and
extension to WinDbg that ships with the .NET Framework, to debug managed
code. After you've loaded SOS, you can use !help for a list of the
commands that you can run.

Keep in mind that analyzing a dump like this requires experience in doing
so and can sometimes take several hours. It's not something that is
intuitive. You have to know how to do it. Unless you have someone there
who is experienced in post-mortem debugging, the best way to isolate these
kinds of issues is to contact us. We can assist you in getting a dump
created and will then analyze it for you.
Avatar of ccbailey

ASKER

Here is the results page (the page with the datagrid).  Let me know if you need anything else.

Thanks for any help,
Christy

<%@ Page Language="C#" ContentType="text/html" ResponseEncoding="iso-8859-1" runat="server" Debug="True"%>
<%@ Register TagPrefix="MM" Namespace="DreamweaverCtrls" Assembly="DreamweaverCtrls,version=1.0.0.0,publicKeyToken=836f606ede05d46a,culture=neutral" %>
<MM:DataSet
id="dsSections"
runat="Server"
IsStoredProcedure="false"
ConnectionString='<%# System.Configuration.ConfigurationSettings.AppSettings["MM_CONNECTION_STRING_dsKatyRoofingSurvey"] %>'
DatabaseType='<%# System.Configuration.ConfigurationSettings.AppSettings["MM_CONNECTION_DATABASETYPE_dsKatyRoofingSurvey"] %>'
CommandText='<%# "SELECT [Table-Sections].FacilityID, [Table-Facility].FacilityName, [Table-Facility].SchoolLevel, [Table-Surveys].FeederZone, [Table-Sections].SectionID, [List-Sections].SectionName, [List-Sections].SectionSort, [Table-Sections].SystemTypeID, [List-SystemType].SystemType, [Table-Sections].YearInstalled, [Table-Sections].ExpectancyID, [List-Expectancy].Expectancy, [Table-Sections].SquareFootage, [Table-Sections].CostperSqFt, [Table-Sections].SlopeID, [List-Slope].Slope, [Table-Sections].History, [Table-Sections].[Photo-Footprint], [Table-Sections].[Photo-Overall], [Table-Sections].Cost, [Table-Sections].SummaryCost, [Table-Sections].WarrantyPeriod, [Table-Sections].ManufacturerName, [Table-Sections].RoofingContractor, [Table-Sections].EstimatedRepairYear  FROM ((([Table-Surveys] RIGHT JOIN ([List-Sections] RIGHT JOIN ([Table-Facility] RIGHT JOIN [Table-Sections] ON [Table-Facility].FacilityID = [Table-Sections].FacilityID) ON [List-Sections].SectionID = [Table-Sections].SectionID) ON [Table-Surveys].FacilityID = [Table-Sections].FacilityID) INNER JOIN [List-SystemType] ON [Table-Sections].SystemTypeID = [List-SystemType].SystemTypeID) INNER JOIN [List-Expectancy] ON [Table-Sections].ExpectancyID = [List-Expectancy].ExpectancyID) INNER JOIN [List-Slope] ON [Table-Sections].SlopeID = [List-Slope].SlopeID  WHERE ((([Table-Facility].SchoolLevel) Like \u0027%\u0027+?+\u0027%\u0027) AND (([List-Sections].SectionName) Like \u0027%\u0027+?) AND (([List-SystemType].SystemType) Like \u0027%\u0027+?+\u0027%\u0027) AND (([Table-Sections].YearInstalled) Like \u0027%\u0027+?+\u0027%\u0027) AND (([List-Expectancy].Expectancy) Like \u0027%\u0027+?+\u0027%\u0027) AND (([List-Slope].Slope) Like \u0027%\u0027+?+\u0027%\u0027))  ORDER BY [Table-Facility].FacilityName, [List-Sections].SectionSort" %>'
Debug="true"
>
  <Parameters>
    <Parameter  Name="lstLevel"  Value='<%# ((Request.QueryString["lstLevel"] != null) && (Request.QueryString["lstLevel"].Length > 0)) ? Request.QueryString["lstLevel"] : ""    %>'  Type="WChar"   />  
    <Parameter  Name="lstSection"  Value='<%# ((Request.QueryString["lstSection"] != null) && (Request.QueryString["lstSection"].Length > 0)) ? Request.QueryString["lstSection"] : ""                  %>'  Type="WChar"   />  
    <Parameter  Name="lstSystemType"  Value='<%# ((Request.QueryString["lstSystemType"] != null) && (Request.QueryString["lstSystemType"].Length > 0)) ? Request.QueryString["lstSystemType"] : ""                  %>'  Type="WChar"   />  
    <Parameter  Name="lstYearInstalled"  Value='<%# ((Request.QueryString["lstYearInstalled"] != null) && (Request.QueryString["lstYearInstalled"].Length > 0)) ? Request.QueryString["lstYearInstalled"] : ""                  %>'  Type="WChar"   />  
    <Parameter  Name="lstLifeExpectancy"  Value='<%# ((Request.QueryString["lstLifeExpectancy"] != null) && (Request.QueryString["lstLifeExpectancy"].Length > 0)) ? Request.QueryString["lstLifeExpectancy"] : ""                  %>'  Type="WChar"   />  
    <Parameter  Name="lstSlope"  Value='<%# ((Request.QueryString["lstSlope"] != null) && (Request.QueryString["lstSlope"].Length > 0)) ? Request.QueryString["lstSlope"] : ""                  %>'  Type="WChar"   />  
  </Parameters>
</MM:DataSet>
<MM:PageBind runat="server" PostBackBind="true" />
<html>
<head>
<title>Katy ISD Roof Assessment Survey</title>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">

</head>
<body bgcolor="#FFFFFF" text="#000000" link="#891819" vlink="#891819" alink="#891819">
<table width="640" border="0" cellspacing="0" cellpadding="0">
  <tr>
    <td><a href="../KatyRoofing.asp"><img src="../Images/Katy-logo.jpg" border="0"></a>&nbsp;</td>
    <td width="500">
      <div align="center">
        <h1><font face="Arial, Helvetica, sans-serif"><strong><font face="Times New Roman, Times, serif">Roof
                Assessment </font></strong></font><font face="Arial, Helvetica, sans-serif"><strong><font face="Times New Roman, Times, serif">Survey</font></strong></font></h1>
        <p><strong><font face="Times New Roman, Times, serif">View Sections Report
              for Multiple Schools</font></strong></p>
      </div>
    </td>
  </tr>
  <tr>
    <td align="center" valign="top">&nbsp;
        <p><img src="../Images/Student1.jpg" width="200" height="200"></p>
        <p><img src="../Images/Student2.jpg" width="200" height="200" border="0"></p>
        <p><img src="../Images/Student3.jpg" width="200" height="200"></p>
        <p>&nbsp;</p>
    </td>
    <td width="500" align="center" valign="top">
      <p></p>
      <form action="" method="post" name="form1" runat="server">
        <p align="left"><strong><font face="Times New Roman, Times, serif">Items
              1 - <%# dsSections.RecordCount %> of <%# dsSections.RecordCount %></font></strong></p>
       
       
       
        <asp:DataGrid
  AllowPaging="false"
  AllowSorting="False"
  AutoGenerateColumns="false"
  CellPadding="3"
  CellSpacing="0"
  DataSource="<%# dsSections.DefaultView %>" id="DataGrid1"
  runat="server"
  ShowFooter="false"
  ShowHeader="true"
>
          <HeaderStyle HorizontalAlign="center" BackColor="#891810" ForeColor="#CCCCCC" Font-Name="Times New Roman, Times, serif" Font-Bold="true" Font-Size="smaller" />
          <ItemStyle BackColor="#FFFFFF" ForeColor="#891810" Font-Name="Times New Roman, Times, serif" Font-Size="smaller" />
          <AlternatingItemStyle BackColor="#CCCCCC" ForeColor="#891810" Font-Name="Times New Roman, Times, serif" Font-Size="smaller" />
          <FooterStyle HorizontalAlign="center" BackColor="#E8EBFD" ForeColor="#3D3DB6" Font-Name="Verdana, Arial, Helvetica, sans-serif" Font-Bold="true" Font-Size="smaller" />
          <PagerStyle BackColor="white" Font-Name="Verdana, Arial, Helvetica, sans-serif" Font-Size="smaller" />
          <Columns>
            <asp:BoundColumn DataField="FacilityName"
        HeaderText="School Name"
        ReadOnly="true"
        Visible="True"/>        
            <asp:BoundColumn DataField="FeederZone"
        HeaderText="School Abbreviation"
        ReadOnly="true"
        Visible="True"/>        
            <asp:BoundColumn DataField="SectionName"
        HeaderText="Section"
        ReadOnly="true"
        Visible="True"/>        
            <asp:BoundColumn DataField="SystemType"
        HeaderText="Nomenclature"
        ReadOnly="true"
        Visible="True"/>        
            <asp:BoundColumn DataField="YearInstalled"
        HeaderText="Yr. Instl."
        ReadOnly="true"
        Visible="True"/>        
            <asp:BoundColumn DataField="Expectancy"
        HeaderText="Life"
        ReadOnly="true"
        Visible="True"/>        
            <asp:BoundColumn DataField="Slope"
        HeaderText="Slope"
        ReadOnly="true"
        Visible="True"/>        
            <asp:BoundColumn DataField="SquareFootage"
        HeaderText="Sq. Ft."
        ReadOnly="true"
        Visible="True"/>        
            <asp:BoundColumn DataField="SummaryCost"
        HeaderText="Cost"
        ReadOnly="true"
        Visible="True"
            DataFormatString="{0:c}" />        
            <asp:TemplateColumn HeaderText="Photo-Footprint"
        Visible="True">
            <ItemTemplate>
&nbsp; </ItemTemplate>
          </asp:TemplateColumn>
            <asp:TemplateColumn HeaderText="Photo-Overall"
        Visible="True">
            <ItemTemplate>
&nbsp; </ItemTemplate>
          </asp:TemplateColumn>
            <asp:BoundColumn DataField="WarrantyPeriod"
        HeaderText="Warranty Per."
        ReadOnly="true"
        Visible="True"/>        
            <asp:BoundColumn DataField="ManufacturerName"
        HeaderText="Manufacturer"
        ReadOnly="true"
        Visible="True"/>        
            <asp:BoundColumn DataField="RoofingContractor"
        HeaderText="Contractor"
        ReadOnly="true"
        Visible="True"/>        
            </Columns>
        </asp:DataGrid>
      </form>
      <p><font face="Times New Roman, Times, serif"><a href="../KatyRoofing.asp">Return
            to Roof Assessment Survey Main Page</a></font></p>
    </td>
  </tr>
</table>
</body>
</html>

which line causes the error?
Wow.  This is way out of my league.  I will talk to my boss and find out what I need to do.

Thanks for the information.
Christy
yeah its not gonna be pretty.
I'm not sure how to find out which line causes the error.  Where do I look?
it should tell you when you get the exception screen
did you say wheter or not you had >= 3gigs or ram in the server?
I don't see that it tells me the line that caused the error.  It only has the error message "System.OutOfMemoryException: Exception of type System.OutOfMemoryException was thrown." at the top of the page, and then under that there is a line and part of the data.  Sometimes there are no records displayed, sometimes there are as many as 5 or 6.  If you would like to look at it, go to http://208.188.75.69/KatyRoofing/ViewMultiple/SectionsReport.asp.  Leave all the options at their default values, and click "Submit".

On the amount of RAM question, I assume we are talking the server that houses the website, not SQL server?  It has 2.0 GB of RAM.

Thanks,
Christy
You should be ok @2Gigs heres the issue I was referring to:
http://www.kbalertz.com/kb_Q820108.aspx

Looking at the page now, after I hit submit its just sitting there, been a long time now.  Will let it continue and let you know what happens.
Still nothing, I suggest you set a breakpoint on your submit button and step through as it just hangs when I do it.  It has not done anything for quite sometime now after I hit the submit.

Regards,

Aeros
**Update its now 5 eastern, the page is still just setting there no exeception, no submission, nothing.  I suspect there is something in your code causing this.  Set the breakpoint on the submit and step through see where it hangs or errors out.  I am heading home now, but I'll check back later tonight, and if not I will be in @8am eastern tomorrow to field any additional questions you might have.

Aeros

on redirect it appears your url is html encoded are you decoding on the receiving page, or did you overlook this?
Look at the status bar on the bottom of the browser(must be IE) when you hit submit to see what I'm talking about with the html encoded bit
OK, I apologize, however I don't completely follow you.  It does take a very long time to load, however it eventually does show a partial datagrid with the error as I described.  It does load a little faster (and still gives the exception) if you select a Level in the first pulldown.  

As far as encoding/decoding, I guess I don't know the answer to that either.  I do all my design in Macromedia Dreamweaver, and so it is either decoded or not based on whatever Dreamweaver does.  I thought I was pretty clear on what I was doing, but I wasn't even aware that there was an option to decode the receiving page (or not) or how to do this.  Would it have an effect on my problem?

Thanks, and I apologize again for my vague answers.

Christy
The page never loaded I never got a partial datagrid, no error message nothing, it just set on the orignal page forever.  Did you click submit and look at the bottom on the status bar and the all the %20's where there are spaces?  Thats a problem check it out.
No - I don't see any "%20's".  There are some %25s - those are my parameters.  The value of the pulldown box when the "All" option is selected is "%".  Then in the query, it uses the like operator, and since "%" is a wildcard, it gets all values.  I'm not sure where the 25 is coming from, but there are no spaces in any of my page names or in the values of the listbox.  I'm not sure where they came from.  The link I get is:

http://208.188.75.69/KatyRoofing/ViewMultiple/SectionsReportResult.aspx?lstLevel=%25&lstSection=%25&lstSystemType=%25&lstYearInstalled=%25&lstLifeExpectancy=%25&lstSlope=%25&Submit2=Submit

Does a 25 mean there is a space?

Thanks,
Christy
ASKER CERTIFIED SOLUTION
Avatar of AerosSaga
AerosSaga

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Aeros,

Thanks so much - putting the sql statement into a stored procedure did the trick.  I am very new to sql (my company just upgraded from Access last month), so I didn't even know about stored procedures.  Thanks for the help!

Christy
Stored procedures are a godsend I'm glad you eletected to take that route;)  And your very welcome Christy.

Regards,

Aeros