COldfusion: Memory allocation failed during query processing: error on CF8 but not on CF7

Posted on 2008-10-31
Medium Priority
Last Modified: 2013-12-24
I am having problems performing a task on a CF8 server that works on a CF7 server. The CF7 & CF8 servers are both windows. What I am trying to do is use cfdirectory to get a list of picture files (about 15,000) then insert a record for each file into a table. When I run the cfm template on the CF7 server it eventually finishes, slow but it finishes. When I run the same template on the cf8 server it eventually comes back with this error

[Macromedia][Informix JDBC Driver][Informix]Memory allocation failed during query processing

I looked in the cf8 monitor and the jvm memory peaks to about 120M out of 512. Here are some cf8 settings which, btw, are the same as on the cf7 server

Maximum number of running JRun threads : 50
Maximum number of queued JRun Threads : 1000
Java Virtual Machine Path : D:/ColdFusion8/runtime/jre
Minimum JVM Heap Size (MB): 512        Maximum JVM Heap Size (MB) : 512
ColdFusion Class Path:  "c:\Program Files\IBM\Informix JDBC Driver 2.21.JC6\lib","D:\cfsftp\cfsftp.jar"
JVM Arguments: -server -Dsun.io.useCanonCaches=false -XX:MaxPermSize=128m -Dcoldfusion.rootDir={application.home}/../ -Dcoldfusion.libPath={application.home}/../lib

Any help or ideas is appreciated!

Here is the code

<CFQUERY datasource="foo" name="delcfd">
      delete from ws_pics

<cfif DirectoryExists("//server/A/B/C/skupics/")>
<cfdirectory directory="//server/A/B/C/skupics/" action="list" name="myD" recurse="no">
<CFLOOP query="myD"> <!--- A --->
<cfif myD.type EQ "Dir" AND  DirectoryExists("//server/A/B/C/skupics/#trim(myD.name)#/")>
<cfdirectory directory="//server/A/B/C/skupics/#trim(myD.name)#/" action="list" name="myD2" filter="a*.jpg|b*.jpg|c*.jpg|d*.jpg|e*.jpg|f*.jpg|g*.jpg|h*.jpg|i*.jpg|j*.jpg|k*.jpg|l*.jpg|m*.jpg|n*.jpg|o*.jpg|p*.jpg|q*.jpg|r*.jpg|s*.jpg|t*.jpg|u*.jpg|v*.jpg|w*.jpg|x*.jpg|y*.jpg|z*.jpg" recurse="no" type="File"> ***** anyone know of a better way to do this ? ****
<CFLOOP query = "myD2" > <!--- B --->
<CFSET variables.nlen = #len(myd2.name)#>
<CFIF nlen GTE 6>
<CFQUERY datasource="famousviews" name="instcfd">
insert into ws_pics (num, picfile) values (0, '#trim(myd2.name)#')
<cfflush> ***print to browser ***
</CFLOOP> <!--- B --->
</CFLOOP> <!--- A --->
Question by:ckercher
  • 3
  • 2
LVL 16

Expert Comment

ID: 22856074
I think this
can just be rewritten as
unless your intention is to exclude files name 1.jpg etc?  

take out your cfflush to speed things up a bit (although I'm guessing you've kept it in as a tracer while trying to debug this?)

avoid setting an unnecessary variable
<CFSET variables.nlen = #len(myd2.name)#>
<CFIF nlen GTE 6>
<CFIF len(myd2.name) GTE 6>

also note you don't need the # # around your function call inside a cfset.

in your inner cfdirectory, try setting listinfo="name", as you're only wanting the filenames, although read this article:
you could probably just use LIstLast(name, "/") to then extract just the filename part.

in your outer cfdirectory, try setting type="dir", then remove your cfif myD.type EQ "Dir" AND  DirectoryExists if clause.  

Could you redo the whole thing as one cfdirectory with recurse=true, applying your filter to just get the jpgs?  saves you having to do nested cfdirectory.

Author Comment

ID: 22866994
Hi duncan...
Thanks for the reply. There are folders and files under /a/b/c that I do not want to process, hence the long filter and multiple cfdirectory loops. Its too bad cfdirectory can not process regex for the filter :(  I'll have to play with the new cfdirectory fields in cf8 to make it more efficient. The odd thing is the cf8 monitor did showed the jvm memory did not fill up which is what I would have expected if that was the issue. Any other ideas aer appreciated.
LVL 62

Expert Comment

ID: 22867631
What is java version - i.e java -version?

-XX: parameter may not perform as expected when another JVM is used.

MaxPermSize and PermSize should be same due to bugs in old JVMs (if you use SUN's JVM)

Does it work when you get rid of parameter?
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!


Author Comment

ID: 22868117
Something I just found to be curious is it always bombs on the insertion of the same file. db spaces and file systems are not filling up.

The JVM is Sun's 1.6.0_04
I have not tried getting rid of the MaxPermSize but that will be my next step.

LVL 62

Expert Comment

ID: 22868279
Is there any Informix session identifiable? onstat -g ses

You should be running later JDK http://secunia.com/advisories/product/12878/?task=advisories_2008

I do not see valid reason for -XX tuning unless system does not cope with JVM's memory use well.

Setting -Xms and -Xmx to same value offloads memory management to JVM - which usually is more optimal that one provided by OS.

Accepted Solution

ckercher earned 0 total points
ID: 22880783
It turns out that the cfquery was using the native cf8 Informix DB drivers. When I switched it to use the downloaded IBM JDBC drivers it works to completion. Unfortunately I don't have time to dig into why the issue with the native drivers. Thanks for your suggestions.

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

How much do you know about the future of data centers? If you're like 50% of organizations, then it's probably not enough. Read on to get up to speed on this emerging field.
The following information will get you familiar with your new DV server, including the (mt) Account Center, the Plesk Control Panel, our world-renowned support department and the rest of the (mt) tools that come with your new service.
With just a little bit of  SQL and VBA, many doors open to cool things like synchronize a list box to display data relevant to other information on a form.  If you have never written code or looked at an SQL statement before, no problem! ...  give i…
The Relationships Diagram is a good way to get an overall view of what a database is keeping track of. It is also where relationships are defined. A relationship specifies how two tables connect to each other. As you build tables in Microsoft Ac…
Suggested Courses

621 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question