[x]
Posted via EE Mobile

Search, ask, and monitor your questions on the go with EE Mobile. Visit Experts Exchange from your mobile device and never be out of touch again.
Question
[x]
Attachment Details
[x]
The Solution Rating System

With so many solutions, how can you tell which solutions are most likely to help you and which ones are not? To provide you with a tool to use, we rate our solutions based on various elements that most accurately determine if a solution is a quality solution. To explain what factors affect the solution rating, here are the elements we take into consideration when formulating our solution rating.

  • The Grade of the Solution
  • The Zone Rank of the Expert Providing the Solution
  • The Number of Author and Expert Comments
  • The Number of Experts Contributing
  • The Feedback of the Community

Your Input Matters
Because of the way the system is set up, the most important variable in this equation is you. As a member of Experts Exchange, you are able to cast your vote on the quality of the solutions in regard to how complete, accurate, helpful and easy to understand each solution is. When you provide your feedback, each rating is adjusted accordingly. So, if you see a solution that has a poor rating that you think is a good solution, let us know by rating it. As you do, the rating will be adjusted and will become more accurate for other members of our site.

If you have any suggestions that you would like to make for our rating system, please ask a question in the Suggestions Zone of Community Support.

Thank you!
9.2

java.lang.OutOfMemoryError: Java heap space | Windows CF8 -> Trying to Load / Parse 6mb XML File

Asked by rcbuchanan in ColdFusion Application Server, Extensible Markup Language (XML), MySQL Server

Tags: coldfusion, xml

Can you help?
I have a daily maintenance task I'm trying to run in CF8 that will load a remote XML file (6mb in size) and parse, via ColdFusion8, into a simply MySQL Table.

I'm getting java.lang.OutOfMemoryError: Java heap space ERR 500 message.

Alas I cannot parcel the remote source xml file into smaller pieces (as the file is not mine to parcel).  

Is there a way to incrementally read the XML file and cache / managed / discretely load each XML 'row' into the MySQL database without the memory issue?

The code is pasted below +/- security.

Thanks! This is an essential and I'm stymied.
Richard 
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:
28:
29:
30:
31:
32:
33:
34:
35:
36:
37:
38:
39:
40:
41:
42:
43:
44:
45:
46:
47:
48:
49:
50:
51:
52:
53:
54:
55:
56:
57:
58:
59:

<!--- URL --->
<cfset congress="http://www.govtrack.us/data/us/111/repstats/people.xml">
 
<cfquery name="insertpeople" datasource="blah"  username="blah" password="blah" timeout="60"> 
    delete from congress_people 
</cfquery>
 
<!--- Get it --->
<cfhttp url="#congress#" method="get" resolveurl="yes" throwonerror="yes"/>
<cfset congress_answer = xmlparse(cfhttp.FileContent)>
 
<cfset resources = congress_answer.xmlroot.xmlChildren>
<cfset numresources = ArrayLen(resources)>
 
<cfset numItems = ArrayLen(congress_answer.people.person)>
 
<cfloop index="i" from="1" to="#numItems#">
 
<cfif structKeyExists(congress_answer.people.person[i].XmlAttributes, "district")>
	<cfset var_district=#congress_answer.people.person[i].XmlAttributes.district#>
<cfelse>
	<cfset var_district = ''>
</cfif>
 
<cfif structKeyExists(congress_answer.people.person[i].XmlAttributes, "metavidid")>
	<cfset var_metavidid=#congress_answer.people.person[i].XmlAttributes.metavidid#>
<cfelse>
	<cfset var_metavidid = ''>
</cfif>
 
<cfif structKeyExists(congress_answer.people.person[i].XmlAttributes, "state")>
	<cfset var_state=#congress_answer.people.person[i].XmlAttributes.state#>
<cfelse>
	<cfset var_state = ''>
</cfif>
 
<cfif structKeyExists(congress_answer.people.person[i].XmlAttributes, "title")>
	<cfset var_title=#congress_answer.people.person[i].XmlAttributes.title#>
<cfelse>
	<cfset var_title = ''>
</cfif>
    
	<cfset var_bioguideid=#congress_answer.people.person[i].XmlAttributes.bioguideid#>
	<cfset var_birthday=#congress_answer.people.person[i].XmlAttributes.birthday#>
	<cfset var_firstname=#congress_answer.people.person[i].XmlAttributes.firstname#>
	<cfset var_gender=#congress_answer.people.person[i].XmlAttributes.gender#>
	<cfset var_id=#congress_answer.people.person[i].XmlAttributes.id#>
	<cfset var_lastname=#congress_answer.people.person[i].XmlAttributes.lastname#>
	<cfset var_name=#congress_answer.people.person[i].XmlAttributes.name#>
	<cfset var_osid=#congress_answer.people.person[i].XmlAttributes.osid#>
 
    <cfquery name="insertpeople" datasource="blah"  username="blah" password="blah" timeout="600"> 
    insert into congress_people 
    (bioguideid, birthday, district, firstname, gender, id, lastname, metavidid, name, osid, state, title)  
    values 
    ('#var_bioguideid#', '#var_birthday#', '#var_district#', '#var_firstname#', '#var_gender#', '#var_id#', '#var_lastname#', '#var_metavidid#', '#var_name#', '#var_osid#', '#var_state#', '#var_title#')
    </cfquery>
 
</cfloop>
[+][-]02/03/09 11:40 PM, ID: 23545661Expert Comment

At Experts Exchange, members can ask their questions to thousands of technology professionals, also known as Experts. Experts compete and collaborate to answer those questions by leaving comments like this one.

Start your 30-day free trial to view this Expert Comment or ask the Experts your question.

 
[+][-]02/04/09 11:30 AM, ID: 23552328Author Comment

Often, when Experts are collaborating with members who have asked questions, they will request additional information about the problem. Askers respond with an author comment like this one.

Start your 30-day free trial to view this Author Comment or ask the Experts your question.

 
[+][-]02/04/09 01:30 PM, ID: 23553771Accepted Solution

View this solution now by starting your 30-day free trial. Setting up your free trial is quick, easy, and secure. We will return you to this solution, unlocked, when you're done.

About this solution

Zones: ColdFusion Application Server, Extensible Markup Language (XML), MySQL Server
Tags: coldfusion, xml
Sign Up Now!
Solution Provided By: _agx_
Participating Experts: 2
Solution Grade: A
 
 
Loading Advertisement...
20091021-EE-VQP-81 - Hierarchy / EE_QW_3_20080625