Solved

COldfusion: Memory allocation failed during query processing: error on CF8 but not on CF7

Posted on 2008-10-31
6
1,003 Views
Last Modified: 2013-12-24
Hello,
I am having problems performing a task on a CF8 server that works on a CF7 server. The CF7 & CF8 servers are both windows. What I am trying to do is use cfdirectory to get a list of picture files (about 15,000) then insert a record for each file into a table. When I run the cfm template on the CF7 server it eventually finishes, slow but it finishes. When I run the same template on the cf8 server it eventually comes back with this error

[Macromedia][Informix JDBC Driver][Informix]Memory allocation failed during query processing

I looked in the cf8 monitor and the jvm memory peaks to about 120M out of 512. Here are some cf8 settings which, btw, are the same as on the cf7 server

Maximum number of running JRun threads : 50
Maximum number of queued JRun Threads : 1000
Java Virtual Machine Path : D:/ColdFusion8/runtime/jre
Minimum JVM Heap Size (MB): 512        Maximum JVM Heap Size (MB) : 512
ColdFusion Class Path:  "c:\Program Files\IBM\Informix JDBC Driver 2.21.JC6\lib","D:\cfsftp\cfsftp.jar"
JVM Arguments: -server -Dsun.io.useCanonCaches=false -XX:MaxPermSize=128m -Dcoldfusion.rootDir={application.home}/../ -Dcoldfusion.libPath={application.home}/../lib

Any help or ideas is appreciated!
Regards,
Joe

Here is the code

<CFQUERY datasource="foo" name="delcfd">
      delete from ws_pics
</CFQUERY>

<cfif DirectoryExists("//server/A/B/C/skupics/")>
<cfdirectory directory="//server/A/B/C/skupics/" action="list" name="myD" recurse="no">
<CFLOOP query="myD"> <!--- A --->
<cfif myD.type EQ "Dir" AND  DirectoryExists("//server/A/B/C/skupics/#trim(myD.name)#/")>
<cfdirectory directory="//server/A/B/C/skupics/#trim(myD.name)#/" action="list" name="myD2" filter="a*.jpg|b*.jpg|c*.jpg|d*.jpg|e*.jpg|f*.jpg|g*.jpg|h*.jpg|i*.jpg|j*.jpg|k*.jpg|l*.jpg|m*.jpg|n*.jpg|o*.jpg|p*.jpg|q*.jpg|r*.jpg|s*.jpg|t*.jpg|u*.jpg|v*.jpg|w*.jpg|x*.jpg|y*.jpg|z*.jpg" recurse="no" type="File"> ***** anyone know of a better way to do this ? ****
<CFLOOP query = "myD2" > <!--- B --->
<CFSET variables.nlen = #len(myd2.name)#>
<CFIF nlen GTE 6>
:#myD.name#/#myd2.name#:
<CFQUERY datasource="famousviews" name="instcfd">
insert into ws_pics (num, picfile) values (0, '#trim(myd2.name)#')
</CFQUERY>
</CFIF>
<cfflush> ***print to browser ***
</CFLOOP> <!--- B --->
</cfif>
</CFLOOP> <!--- A --->
...
0
Comment
Question by:ckercher
  • 3
  • 2
6 Comments
 
LVL 16

Expert Comment

by:duncancumming
ID: 22856074
I think this
filter="a*.jpg|b*.jpg|c*.jpg|d*.jpg|e*.jpg|f*.jpg|g*.jpg|h*.jpg|i*.jpg|j*.jpg|k*.jpg|l*.jpg|m*.jpg|n*.jpg|o*.jpg|p*.jpg|q*.jpg|r*.jpg|s*.jpg|t*.jpg|u*.jpg|v*.jpg|w*.jpg|x*.jpg|y*.jpg|z*.jpg"
can just be rewritten as
filter="*.jpg"
unless your intention is to exclude files name 1.jpg etc?  

take out your cfflush to speed things up a bit (although I'm guessing you've kept it in as a tracer while trying to debug this?)

avoid setting an unnecessary variable
<CFSET variables.nlen = #len(myd2.name)#>
<CFIF nlen GTE 6>
becomes
<CFIF len(myd2.name) GTE 6>

also note you don't need the # # around your function call inside a cfset.

in your inner cfdirectory, try setting listinfo="name", as you're only wanting the filenames, although read this article:
http://www.bennadel.com/blog/886-Learning-ColdFusion-8-CFDirectory-Improvements.htm
you could probably just use LIstLast(name, "/") to then extract just the filename part.

in your outer cfdirectory, try setting type="dir", then remove your cfif myD.type EQ "Dir" AND  DirectoryExists if clause.  

Could you redo the whole thing as one cfdirectory with recurse=true, applying your filter to just get the jpgs?  saves you having to do nested cfdirectory.
0
 

Author Comment

by:ckercher
ID: 22866994
Hi duncan...
Thanks for the reply. There are folders and files under /a/b/c that I do not want to process, hence the long filter and multiple cfdirectory loops. Its too bad cfdirectory can not process regex for the filter :(  I'll have to play with the new cfdirectory fields in cf8 to make it more efficient. The odd thing is the cf8 monitor did showed the jvm memory did not fill up which is what I would have expected if that was the issue. Any other ideas aer appreciated.
Joe
0
 
LVL 62

Expert Comment

by:gheist
ID: 22867631
What is java version - i.e java -version?

-XX: parameter may not perform as expected when another JVM is used.

MaxPermSize and PermSize should be same due to bugs in old JVMs (if you use SUN's JVM)

Does it work when you get rid of parameter?
0
Create the perfect environment for any meeting

You might have a modern environment with all sorts of high-tech equipment, but what makes it worthwhile is how you seamlessly bring together the presentation with audio, video and lighting. The ATEN Control System provides integrated control and system automation.

 

Author Comment

by:ckercher
ID: 22868117
Something I just found to be curious is it always bombs on the insertion of the same file. db spaces and file systems are not filling up.

The JVM is Sun's 1.6.0_04
I have not tried getting rid of the MaxPermSize but that will be my next step.

Thanks!
Joe
0
 
LVL 62

Expert Comment

by:gheist
ID: 22868279
Is there any Informix session identifiable? onstat -g ses

You should be running later JDK http://secunia.com/advisories/product/12878/?task=advisories_2008

I do not see valid reason for -XX tuning unless system does not cope with JVM's memory use well.

Setting -Xms and -Xmx to same value offloads memory management to JVM - which usually is more optimal that one provided by OS.
0
 

Accepted Solution

by:
ckercher earned 0 total points
ID: 22880783
It turns out that the cfquery was using the native cf8 Informix DB drivers. When I switched it to use the downloaded IBM JDBC drivers it works to completion. Unfortunately I don't have time to dig into why the issue with the native drivers. Thanks for your suggestions.
Joe
0

Featured Post

Microsoft Certification Exam 74-409

Veeam® is happy to provide the Microsoft community with a study guide prepared by MVP and MCT, Orin Thomas. This guide will take you through each of the exam objectives, helping you to prepare for and pass the examination.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Shadow IT is coming out of the shadows as more businesses are choosing cloud-based applications. It is now a multi-cloud world for most organizations. Simultaneously, most businesses have yet to consolidate with one cloud provider or define an offic…
Many companies are looking to get out of the datacenter business and to services like Microsoft Azure to provide Infrastructure as a Service (IaaS) solutions for legacy client server workloads, rather than continuing to make capital investments in h…
Video by: Steve
Using examples as well as descriptions, step through each of the common simple join types, explaining differences in syntax, differences in expected outputs and showing how the queries run along with the actual outputs based upon a simple set of dem…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

733 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question