COldfusion: Memory allocation failed during query processing: error on CF8 but not on CF7

Posted on 2008-10-31
Medium Priority
Last Modified: 2013-12-24
I am having problems performing a task on a CF8 server that works on a CF7 server. The CF7 & CF8 servers are both windows. What I am trying to do is use cfdirectory to get a list of picture files (about 15,000) then insert a record for each file into a table. When I run the cfm template on the CF7 server it eventually finishes, slow but it finishes. When I run the same template on the cf8 server it eventually comes back with this error

[Macromedia][Informix JDBC Driver][Informix]Memory allocation failed during query processing

I looked in the cf8 monitor and the jvm memory peaks to about 120M out of 512. Here are some cf8 settings which, btw, are the same as on the cf7 server

Maximum number of running JRun threads : 50
Maximum number of queued JRun Threads : 1000
Java Virtual Machine Path : D:/ColdFusion8/runtime/jre
Minimum JVM Heap Size (MB): 512        Maximum JVM Heap Size (MB) : 512
ColdFusion Class Path:  "c:\Program Files\IBM\Informix JDBC Driver 2.21.JC6\lib","D:\cfsftp\cfsftp.jar"
JVM Arguments: -server -Dsun.io.useCanonCaches=false -XX:MaxPermSize=128m -Dcoldfusion.rootDir={application.home}/../ -Dcoldfusion.libPath={application.home}/../lib

Any help or ideas is appreciated!

Here is the code

<CFQUERY datasource="foo" name="delcfd">
      delete from ws_pics

<cfif DirectoryExists("//server/A/B/C/skupics/")>
<cfdirectory directory="//server/A/B/C/skupics/" action="list" name="myD" recurse="no">
<CFLOOP query="myD"> <!--- A --->
<cfif myD.type EQ "Dir" AND  DirectoryExists("//server/A/B/C/skupics/#trim(myD.name)#/")>
<cfdirectory directory="//server/A/B/C/skupics/#trim(myD.name)#/" action="list" name="myD2" filter="a*.jpg|b*.jpg|c*.jpg|d*.jpg|e*.jpg|f*.jpg|g*.jpg|h*.jpg|i*.jpg|j*.jpg|k*.jpg|l*.jpg|m*.jpg|n*.jpg|o*.jpg|p*.jpg|q*.jpg|r*.jpg|s*.jpg|t*.jpg|u*.jpg|v*.jpg|w*.jpg|x*.jpg|y*.jpg|z*.jpg" recurse="no" type="File"> ***** anyone know of a better way to do this ? ****
<CFLOOP query = "myD2" > <!--- B --->
<CFSET variables.nlen = #len(myd2.name)#>
<CFIF nlen GTE 6>
<CFQUERY datasource="famousviews" name="instcfd">
insert into ws_pics (num, picfile) values (0, '#trim(myd2.name)#')
<cfflush> ***print to browser ***
</CFLOOP> <!--- B --->
</CFLOOP> <!--- A --->
Question by:ckercher
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
LVL 16

Expert Comment

ID: 22856074
I think this
can just be rewritten as
unless your intention is to exclude files name 1.jpg etc?  

take out your cfflush to speed things up a bit (although I'm guessing you've kept it in as a tracer while trying to debug this?)

avoid setting an unnecessary variable
<CFSET variables.nlen = #len(myd2.name)#>
<CFIF nlen GTE 6>
<CFIF len(myd2.name) GTE 6>

also note you don't need the # # around your function call inside a cfset.

in your inner cfdirectory, try setting listinfo="name", as you're only wanting the filenames, although read this article:
you could probably just use LIstLast(name, "/") to then extract just the filename part.

in your outer cfdirectory, try setting type="dir", then remove your cfif myD.type EQ "Dir" AND  DirectoryExists if clause.  

Could you redo the whole thing as one cfdirectory with recurse=true, applying your filter to just get the jpgs?  saves you having to do nested cfdirectory.

Author Comment

ID: 22866994
Hi duncan...
Thanks for the reply. There are folders and files under /a/b/c that I do not want to process, hence the long filter and multiple cfdirectory loops. Its too bad cfdirectory can not process regex for the filter :(  I'll have to play with the new cfdirectory fields in cf8 to make it more efficient. The odd thing is the cf8 monitor did showed the jvm memory did not fill up which is what I would have expected if that was the issue. Any other ideas aer appreciated.
LVL 62

Expert Comment

ID: 22867631
What is java version - i.e java -version?

-XX: parameter may not perform as expected when another JVM is used.

MaxPermSize and PermSize should be same due to bugs in old JVMs (if you use SUN's JVM)

Does it work when you get rid of parameter?
NFR key for Veeam Backup for Microsoft Office 365

Veeam is happy to provide a free NFR license (for 1 year, up to 10 users). This license allows for the non‑production use of Veeam Backup for Microsoft Office 365 in your home lab without any feature limitations.


Author Comment

ID: 22868117
Something I just found to be curious is it always bombs on the insertion of the same file. db spaces and file systems are not filling up.

The JVM is Sun's 1.6.0_04
I have not tried getting rid of the MaxPermSize but that will be my next step.

LVL 62

Expert Comment

ID: 22868279
Is there any Informix session identifiable? onstat -g ses

You should be running later JDK http://secunia.com/advisories/product/12878/?task=advisories_2008

I do not see valid reason for -XX tuning unless system does not cope with JVM's memory use well.

Setting -Xms and -Xmx to same value offloads memory management to JVM - which usually is more optimal that one provided by OS.

Accepted Solution

ckercher earned 0 total points
ID: 22880783
It turns out that the cfquery was using the native cf8 Informix DB drivers. When I switched it to use the downloaded IBM JDBC drivers it works to completion. Unfortunately I don't have time to dig into why the issue with the native drivers. Thanks for your suggestions.

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article shows how to get a list of available printers for display in a drop-down list, and then to use the selected printer to print an Access report or a Word document filled with Access data, using different syntax as needed for working with …
In this article, I’ll look at how you can use a backup to start a secondary instance for MongoDB.
Video by: Steve
Using examples as well as descriptions, step through each of the common simple join types, explaining differences in syntax, differences in expected outputs and showing how the queries run along with the actual outputs based upon a simple set of dem…
This is a high-level webinar that covers the history of enterprise open source database use. It addresses both the advantages companies see in using open source database technologies, as well as the fears and reservations they might have. In this…

777 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question