[Last Call] Learn how to a build a cloud-first strategyRegister Now

x
?
Solved

ColdFusion 7 crashing when looping over large chunk of text

Posted on 2011-05-12
23
Medium Priority
?
594 Views
Last Modified: 2012-06-27
I have an application that needs to send out emails. For each email I need to insert custom tracking ids to the URLs in the email message. The HTML content for the email is around 44K and I have tried multiple methods to loop over these records but it eventually crashed the CF server after 4000 or so iterations.

If I try sending the emails immediately CF crashes even faster. I tried storing all of the content in the database putting the complete html code for the emails in a largetext field and that only made MySQL run out of memory as well. I was trying to create the HTML content once and then just swap out the tracking info per user. I thought that would elleviate some stress from the server but it did not improve performance. When I watch the Task Manager in Windows when I start this routine CF memory usage jumps up quickly and in large increments 20K-30K a second.

Basically what I need is for this code to be able to process 50,000+ emails without crashing the server. Suggestions to reduce this apparent memory leak would be greatly appreciated.

      <cfsavecontent variable="emailBody">
        <!--- created the intial email with a new variable to swap out --->
         <cfinclude template="/intranet/event_rollup2.cfm">
      </cfsavecontent>
      <cfset session.emailAddTracking = emailBody>

  
    <cfloop query="getFollowers">
      <!--- create tracking record --->
      <cfset temp = mkt.createTrackingRecord(val(marketId),val(getFollowers.prospectId),val(getFollowers.mailid),"email")>
      <cfset em.trackingId = "#val(marketId)#-#val(getFollowers.prospectId)#-#val(getFollowers.mailid)#:email">
      <cfset em.trackingId = URLencodedFormat(toBase64(em.trackingId))>
      
      <cfset curEmail = getFollowers.email>
    
      <cfset emailBody = replaceNoCase(session.emailAddTracking,"@@@track@@@",em.trackingId,"ALL")>
      <cffile action="write" file="D:\temp\mailid_#getFollowers.mailid#_marketid_#marketid#.html" output="#emailBody#">
      
      <!--- add all followers to spool table --->
      <cfquery name="addToSpool" datasource="mailspooler">
        INSERT INTO mailspool (datecreated, mailid, emailto, emailfrom, emailsubject, marketid)
        VALUES (now(), #getFollowers.mailid#, '#curEmail#', '#getSiteDomainPre.systemAdminEmailAddress#', '#getEvents.emailtitle#', #marketid#)
      </cfquery>
			
      <cfcontent reset="true">
  <cfloop>

Open in new window


0
Comment
Question by:mackaboogie
  • 8
  • 6
  • 5
  • +2
23 Comments
 
LVL 39

Accepted Solution

by:
gdemaria earned 668 total points
ID: 35754682
Try dividing your send into batches.   You can add a flag to your mailSpool table to indicate whether or not the email has been sent.   Grab 500 records at a time, loop through those 500 sending each one.  Repeat every 10-15 minutes.  

Note that the mass mailing companies do it this way.  That is why it takes hours to send out emails.   You could add intelligence to grab no more than 5 email address from any domain during one send.  That will reduce the likihood of being black listed.

0
 
LVL 4

Author Comment

by:mackaboogie
ID: 35755217
I have tried something like that already. That is part of the reason I am trying to create the emails into a spooler and the use another task to send them in batches. The problem is that even a couple dozen iterations over this loop cause the CF server to spike. One the process completes it seems the CF memory use does not drop, so the next time it keeps building and eventually crashes.
0
 
LVL 39

Expert Comment

by:gdemaria
ID: 35755546
This may have nothing to do with it but I would move
<cfcontent reset="true">

outside of your loop.   I don't think you want to run that with every iteration.. once at the should do it

0
Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

 
LVL 4

Author Comment

by:mackaboogie
ID: 35755705
gdemaria - Thanks for the suggestion. I actually added that in there as a last ditch effort to stop the memory build up on the server. It really had no affect.

I also tried playing with different settings in the CF admin and did not really see a change. It there a way to flush all of this data between loops so that it does not build up?
0
 
LVL 39

Expert Comment

by:gdemaria
ID: 35755766
I just noticed this..

<cfset session.emailAddTracking = emailBody>


Why would you use a session variable here?

I would change this to a variable scope variable, I can't think of a reason why you'd want the entire email body to persist in the user's session, and then replace it constantly every use.  That could be the problem right there...

<cfset variables.emailAddTracking = emailBody>
0
 
LVL 4

Author Comment

by:mackaboogie
ID: 35755810
Here is my thinking (probably wrong) I am creating the email once. The email code in the include has multiple queries to generate the layout and design. I figured I would create it once, store it in memory and then reuse it through each loop to just swap out the tracking values. That way I don;t have to re-create the email itself 50,000 times.

I did just come across some info on the Adobe knowledge base about a hotfix for CF 7.0.2 to fix memory leaks in CFCs. I will need to see if this patch exists on this server. I do not usually work on this client server so I am not sure how updated it is (obvious version 7, not too up-to-date).
0
 
LVL 39

Expert Comment

by:gdemaria
ID: 35755868
sorry, I have been in a rush and not focusing well here.   I see now that the session assignment is not inside the loop.   To accomplish your goal, you can use the variable scope because you never leave the CF template.  No difference.  

However, this is also not your issue.
0
 
LVL 52

Expert Comment

by:_agx_
ID: 35755870
mkt.createTrackingRecord

What does the code do? I assume it's some sort of cfc.

Are you sure it's really using a "large chunk of text" that's causing the issue? You are doing 50K loops with multiple db queries on each iteration. Just wonder if it's not something else....
0
 
LVL 39

Expert Comment

by:gdemaria
ID: 35755902

.. and you never use the temp variable created by that cfc, so try removing it and see how things go..

0
 
LVL 36

Assisted Solution

by:SidFishes
SidFishes earned 668 total points
ID: 35755954
you might look at tweaking your jvm settings, especially garbage collection which can become an issue on long running tasks

This is a good resource on tuning & tweaking http://www.trunkful.com/index.cfm/2008/11/25/How-to-Tune-the-JVM-Part-2

0
 
LVL 52

Expert Comment

by:_agx_
ID: 35756168
SidFishes
>> you might look at tweaking your jvm settings      

especially since it's a memory related crash.  

>> VALUES (now(), #getFollowers.mailid#, '#curEmail#', ....)

the jvm settings are more critical. but just fyi, always use cfqueryparam whenever querying w/in a loop. it usually helps boost db performance by promoting reuse of query plans


0
 
LVL 36

Expert Comment

by:SidFishes
ID: 35756368
yes... excellent point agx

cfqueryparam isn't just for security, performance is also improved

http://coldfusion.sys-con.com/node/41712 
0
 
LVL 4

Author Comment

by:mackaboogie
ID: 35756402
I tried the cfquery param and as expected to did speed up the queries in the loop but did nothing for the overall jrun memory usage. I installed hotpatch 3 for 7.0.2 and am updating the JVM to 1.4.2_11. I will then look into the JVM tuning suggested above. I have tried dozens of updates to my code, commenting out sections and adding in others and nothing seems to help much.
0
 
LVL 52

Expert Comment

by:_agx_
ID: 35756561
yeah, I wouldn't expect cfqueryparam to fix a memory issue.  you'd have to look into jvm tuning / monitoring for that.

mkt.createTrackingRecord
what does this do?

I have tried dozens of updates to my code,
did you figure out the minimum code that causes the issue?  ie Does a simple loop with cffile do it?

 <cfloop query="getFollowers">
      <cfset curEmail = getFollowers.email>
   
      <cfset emailBody = replaceNoCase(session.emailAddTracking,"@@@track@@@","xxx","ALL")>
      <cffile action="write" file="D:\temp\mailid_#getFollowers.mailid#_marketid_#marketid#.html" output="#emailBody#">
  <cfloop>

Or does it occur when you add in a cfquery ?

 <cfloop query="getFollowers">
      <cfset curEmail = getFollowers.email>
   
      <cfset emailBody = replaceNoCase(session.emailAddTracking,"@@@track@@@","xxx","ALL")>
      <cffile action="write" file="D:\temp\mailid_#getFollowers.mailid#_marketid_#marketid#.html" output="#emailBody#">

      <!--- add all followers to spool table --->
      <cfquery name="addToSpool" datasource="mailspooler">
        INSERT INTO mailspool (datecreated, mailid, emailto, emailfrom, emailsubject, marketid)
        VALUES (now(), #getFollowers.mailid#, '#curEmail#', '#getSiteDomainPre.systemAdminEmailAddress#', '#getEvents.emailtitle#', #marketid#)
      </cfquery>
  <cfloop>      
0
 
LVL 4

Author Comment

by:mackaboogie
ID: 35756613
It spikes even without a query, but it completes so quick that the memory does not build up that much. I did the JVM tuning and it looked like it was working but instead of crashing at 580Megs of memory usage it crashed at 1.1Gigs. Still only processed the same number records...

I think I may need to go with a smaller chunk of records per transaction.
0
 
LVL 52

Expert Comment

by:_agx_
ID: 35756678
It spikes even without a query,
You mean just the cffile write part (only), nothing else? Weird ...could be all the disk i/o and temp string objects created 50k times.

I think I may need to go with a smaller chunk of records per transaction.
that's what I would do. I always break big tasks into smaller batches.
0
 
LVL 4

Author Comment

by:mackaboogie
ID: 35757047
Thanks everyone. I will work on this some more and post back my final solution. I will pass out some points as soon as I figure this out.
0
 
LVL 36

Expert Comment

by:SidFishes
ID: 35757275
I'm thinking agx has got the right idea about the disk write. I know you are trying to spool but I think you may be causing more problems than you are solving.

I send out a newsletter to about 22,000 people once a month with file sizes ranging from 25-35k and don't have a problem (I'm on CF8 though)

My newsletters are created in my admin cms I built which saves them as plain html to my rss directory. When I do the mailout, I read it once into a variable and manage the rate of send with sleep() (not sure that's available in 7 though)

bare bones code is attached

however ---- Now that I think about it, I did have some pretty consistent out of memory crashes during mailouts when my server had only 2g but since I upgraded to 8g there's been no issue. If your server is managing with only a couple of gigs of ram, you may need to upgrade...

<cffile action="read" file="D:\virtual\webserver\RSS\newsletters\#emailObject#" variable="nlContent">
	<cfloop query="emaillist">
				<cfmail to="#emaillist.emailaddress#" 
					server="xxxxx"
			        username="xxxxxx"
			        password="xxxxx"
					from="xxxxxxxxx"
					subject="#subject#"
						type="text/html">
				   #nlContent#
			   </cfmail>
		<cfset sleep(1000)>
	</cfloop>

Open in new window

0
 
LVL 4

Author Comment

by:mackaboogie
ID: 35757484
Sleep may be what I need. I found this on Pete Freitag's website...

<cfset thread = CreateObject("java", "java.lang.Thread")>
About to sleep for 5 seconds...<cfflush>
<cfset thread.sleep(5000)>

Open in new window


That should work for MX7. It may be that if I slow down the processing the JVM will be able to clean up between routines...
0
 
LVL 52

Assisted Solution

by:_agx_
_agx_ earned 664 total points
ID: 35758286
If you haven't done it already, you should use a jvm monitor/profiler to see if the jvm tuning actually helped the problem or if it still needs adjusting
0
 
LVL 1

Expert Comment

by:witsCOMPUTING
ID: 35768467
Instead of sending bulk emails at once, you should create a schedule job which send few hundred emails e.g. 300 per run.
0
 
LVL 4

Author Comment

by:mackaboogie
ID: 35782007
I got my scheduler working. I had to NOT use CFCs even though I had installed the hot fixes for CF7. I did cancel all variables at the end of the request. <cfset structClear(variables)> as well as a <cfset queryName = ""> for each query and each reused variable. Once I had the code working the memory on the server would max out and not go back down.

I did some more tuning and additionally tested some code to manually flush to GC. Nothing else helped other than clearing the variables at the end of the request.

I ended up using the two scheduled task approach. The first grabs my scheduled emails to go out, renders the email with the tracking info for each user and enters the data into a mailspool table.

My second task runs every 3 minutes, grabs the first 500 records from the mailspool table and send the emails using cfx_mail (which is way faster than cfmail I found).

I will have to split up points for all of the assistance. The only thing that did not help (although I was sure it would) was adding in the cfqueryparams. That just added to the heap.

Thanks Everyone!
0
 
LVL 52

Expert Comment

by:_agx_
ID: 35782481
I will have to split up points for all of the assistance. The only thing that did not help (although I was sure it would) was adding in the cfqueryparams. That just added to the heap.

Understand cfqueryparam's purpose is to help database performance, not jvm issues. As far as the heap, I'm not sure how cfqueryparam effects the jvm. But I would be surprised if it had a significant effect anyway since your memory crashes occurred before you even started using it. If in fact manually clearing the variables is what resolved it.. that too points to a bigger issue with unreleased memory. Possibly a cf issue, but more commonly it's the code itself.  
0

Featured Post

Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The technique is by far very Simple! How we can export the ColdFusion query results to DOC file?  Well before writing this I researched a lot in Internet but did not found a good Answer anyways!  So i thought now i should share my small snippet w…
Recently while working on a project I got a very annoying cfdocument has no body error message. I had never seen this error before. So I checked the code. The code was pretty simple; it was Just showing me the cfdocumnt tag and inside that tag a …
This video shows how to quickly and easily deploy an email signature for all users in Office 365 and prevent it from being added to replies and forwards. (the resulting signature is applied on the server level in Exchange Online) The email signat…
Loops Section Overview
Suggested Courses

834 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question