• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1183
  • Last Modified:

I got out of memory when processing a Linq query

Hi,

  The following code is leading the system to out of memory. The database is FireBird. Let me explain what happens:

  - srv.DadosCalculados(srv.Consulta(_fromData, _toData).ToList()) goes to the database, by Linq, and reads about 500,000 records
  - foreach gets each record and saves it to another table
  - OLAP_VENDAS olapv = new OLAP_VENDAS(); creates a new object in the Linq entity.
  - entity.AddObject(olapv.GetType().Name, olapv); insert the entity in the object
  - entity.SaveChanges(); it saves the data each 500 read reacords

  The procedure works up to 100,000 records processed. After this I get the out of memory message. My machine has 6Gb of RAM and it starts with 2.6Gb allocated RAM and raises up 6Gb very quickly.

  Is is not possible to dispose the OLAP_VENDAS object. How can I keep the RAM short?

  Thanks,
  Marco Castro
foreach (var item in srv.DadosCalculados(srv.Consulta(_fromData, _toData).ToList()))
{
 OLAP_VENDAS olapv = new OLAP_VENDAS();
 olapv.VENDAS = new Misc().GenID("OLAP_VENDAS");
 olapv.VENDAS_CABECALHO = olap.VENDAS_CABECALHO;
 olapv.ANO = item.Ano;
 olapv.CLIENTE = item.ClienteID;
 olapv.EDITORA = item.MarcaID;
 olapv.LIVRO = item.ProdutoID;
 olapv.QT_ACERTO = item.QuantidadeAcertoConsig;
 olapv.QT_CONSIG = item.EnvioConsignacao;
 olapv.QT_DEVOL = item.DevolucaoConsig;
 olapv.QT_VENDA = item.QuantidadeVenda;
 olapv.QUADRIMESTRE = item.Quadrimestre;
 olapv.MES = item.Mes;
 olapv.VALOR_SAIDA = item.ValorSaida;
 olapv.VALOR_ENTRADA = item.ValorEntrada;
 try
 {
  entity.AddObject(olapv.GetType().Name, olapv);
  if (i++ > 500)
  {
   entity.SaveChanges();
   i = 0;
  }
 }
 catch (Exception e)
 {
  throw new FaultException<FaultContract>(new FaultContract(e.ToString()));
 }
}

Open in new window

0
MarcoCastro
Asked:
MarcoCastro
  • 2
  • 2
1 Solution
 
sergiobg57Commented:
Can't you subdivide/page the srv.consulta query?
Use the limit keyword in order to maintain a shorter list and go all the way forward till the end.

select c from t where c1 = c2  limit HOW_MANY_YOU_HAVE_READ, NUMBER_OF_RESULTS_TO_SHOW
0
 
John ClaesSenior .Net Consultant & Technical AnalistCommented:
Linq knows the folowing Queryoperators
Skip and Take

Example :
Takes the Items from 51 till 100
.Skip(50).Take(50);
The Sql query is weird, but gives only the requested Items


so then you have less Items in your select-ResultSet.
I would propose to divide your list using an 'Prosed'-Field of some kind


regards

poor beggar
0
 
John ClaesSenior .Net Consultant & Technical AnalistCommented:
Some background may help :

SQL 2000 : It uses some weird selection Query
From SQL 2005 it uses the ROW_NUMBER to return the selection

0
 
MarcoCastroAuthor Commented:
The solution is: clear all the created objects with this code:

for (int j = 0; j < list.Count; j++)
   entity.Detach(list[j]);

After this: entity.SaveChanges();
0
 
MarcoCastroAuthor Commented:
I got the answer by myself.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Upgrade your Question Security!

Your question, your audience. Choose who sees your identity—and your question—with question security.

  • 2
  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now