?
Solved

parse xml feeds into mysql

Posted on 2009-04-24
10
Medium Priority
?
370 Views
Last Modified: 2012-05-06
I am trying to parse an xml feed in to mysql using php code. However, i keep getting the following error messages:

Warning: XMLReader::read() [xmlreader.read]: /home/icanget/public_html/getmein.xml:79: parser error : xmlParseEntityRef: no name in /home/icanget/public_html/getmein.php on line 12

Warning: XMLReader::read() [xmlreader.read]: <td>Nine Inch Nails & Jane's Addiction in London, United Kingdom</td> in /home/icanget/public_html/getmein.php on line 12

Warning: XMLReader::read() [xmlreader.read]: ^ in /home/icanget/public_html/getmein.php on line 12

Warning: XMLReader::read() [xmlreader.read]: An Error Occured while reading in /home/icanget/public_html/getmein.php on line 12

Can someone please point out what i have don incorrectly please?
<?php require_once ('connect.php'); 
db_connect();
 
$xmlReader = new XMLReader();
 
$filename = "getmein.xml";
$url = "http://feeds.perfb.com/index.php/download?OEMAIL=&PX=d0310005b5e00cc254c6bc3c1fe075f0&DISPLAYFORMAT=HEAD&REVERSEMAPXML=yes&PRODUCTDB_ID=357";
file_put_contents($filename, file_get_contents($url));
 
$xmlReader->open($filename); 
 
while ($xmlReader->read()) 
{
}
switch ($xmlReader->name)
{
case 'product':
 
$dom = new DOMDocument();
$domNode = $xmlReader->expand();
$element = $dom->appendChild(domNode);
$domstring = utf8_encode($dom->saveXML($element));
$product = new SimpleXMLElement($domString);
 
$product_code = $product->product_code;
$product_name = $product->product_name;
$level1 = $product->level1;
$level2 = $product->level2;
$description = $product->description;
$buyat_short_deeplink_url = $product->buyat_short_deeplink_url;
 
$image_url = $product->image_url;
$currency = $product->currency;
$price = $product->online_price;
$city = $product->city;
$country = $product->country;
$event_date = $product->event_date;
$event_venue = $product->event_venue;
$number_available = $product->number_available;
 
if (strlen($product_code)> 0)
{
$query = mysql_query("REPLACE INTO icanget_tickets
(product_code, product_name, level1, level2, description, buyat_short_deeplink_url, image_url, currency, price, country, event_date, event_venue, number_available)
 
VALUES ('$product_code', '$product_name', '$level1', '$level2', '$description', '$buyat_short_deeplink_url', '$image_url', '$currency', '$price', '$country', '$event_date', '$event_venue', '$number_available')");
 
echo $product_name . "has been inserted </br>";
}
break;
 
}
 
?>

Open in new window

0
Comment
Question by:rafique12
  • 6
  • 3
10 Comments
 
LVL 6

Expert Comment

by:basic612
ID: 24225842
From the error you are reporting it would seem that the data you are trying to process with XMLreader is not a valid XML document. It appears to be part of an HTML table.

When I attempted to check the URL you have hardcoded in to the script (http://feeds.perfb.com/index.php/download?OEMAIL=&PX=d0310005b5e00cc254c6bc3c1fe075f0&DISPLAYFORMAT=HEAD&REVERSEMAPXML=yes&PRODUCTDB_ID=357) I do not see any XML, rather an error message from buy.at

Can you test this script with a locally hosted copy of an XML document that is in the right format, then once you know that is working you can concentrate on getting the correct XML document in to the script. My guess there is that the document at that URL relies on some session based data that is not available to your script.

HTH
0
 
LVL 6

Expert Comment

by:basic612
ID: 24225883
Also, checking the URL http://feeds.perfb.com/index.php shows me this error / hint as to what parameters you need to provide.
  • OEMAIL is a required parameter
  • OEMAIL must be a valid email address
  • PX is a required parameter
  • You must specify either PRODUCTDB_ID or CUSTOM_FEED_ID
  • DISPLAYFORMAT is a required parameter
  • DISPLAYFORMAT must be one of: XMLGZIP XML CSVGZIP CSV PIPEGZIP PIPE SCSVGZIP SCSV SIMPLECSV HEAD SAMPLE
I see your DISPLAYFORMAT is set to HEAD. Not knowing much about the system but guessing from your requirements, should that not be XML instead?
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 
LVL 111

Expert Comment

by:Ray Paseur
ID: 24226120
@rafique12: When I tried that link, I got an HTML document.
@basic612: When I tried that link, I got a file download that I stopped after 30 MB.  Not sure what it was supposed to do, but it looked like it was out of control - maybe looping?

Is is possible to extract a segment of the XML and post it here?  We can help you parse it if we can get a look at it!

best, ~Ray
0
 
LVL 6

Accepted Solution

by:
basic612 earned 2000 total points
ID: 24226155
@Ray - if you had not have stopped the download you would have ended up with a 92Mb XML doc - which is what I would suggest rafique is trying to create a MySQL table from.

For clarity I have attached the corrected script below
<?php require_once ('connect.php'); 
db_connect();
 
$xmlReader = new XMLReader();
 
$filename = "getmein.xml";
$url = "http://feeds.perfb.com/index.php/download?OEMAIL=danielrafique@vyceversa.co.uk&PX=d0310005b5e00cc254c6bc3c1fe075f0&DISPLAYFORMAT=XML&REVERSEMAPXML=yes&PRODUCTDB_ID=357";
file_put_contents($filename, file_get_contents($url));
 
$xmlReader->open($filename); 
 
while ($xmlReader->read()) 
{
}
switch ($xmlReader->name)
{
case 'product':
 
$dom = new DOMDocument();
$domNode = $xmlReader->expand();
$element = $dom->appendChild(domNode);
$domstring = utf8_encode($dom->saveXML($element));
$product = new SimpleXMLElement($domString);
 
$product_code = $product->product_code;
$product_name = $product->product_name;
$level1 = $product->level1;
$level2 = $product->level2;
$description = $product->description;
$buyat_short_deeplink_url = $product->buyat_short_deeplink_url;
 
$image_url = $product->image_url;
$currency = $product->currency;
$price = $product->online_price;
$city = $product->city;
$country = $product->country;
$event_date = $product->event_date;
$event_venue = $product->event_venue;
$number_available = $product->number_available;
 
if (strlen($product_code)> 0)
{
$query = mysql_query("REPLACE INTO icanget_tickets
(product_code, product_name, level1, level2, description, buyat_short_deeplink_url, image_url, currency, price, country, event_date, event_venue, number_available)
 
VALUES ('$product_code', '$product_name', '$level1', '$level2', '$description', '$buyat_short_deeplink_url', '$image_url', '$currency', '$price', '$country', '$event_date', '$event_venue', '$number_available')");
 
echo $product_name . "has been inserted </br>";
}
break;
 
}
 
?>

Open in new window

0
 

Author Comment

by:rafique12
ID: 24226261
I am now getting this:
Fatal error: Maximum execution time of 30 seconds exceeded in /home/icanget/public_html/getmein.php on line 8
How do i limit the amount of data being downloaded? you are quite right that the feed contains 92mb of data well over 47,000 items...
0
 
LVL 6

Expert Comment

by:basic612
ID: 24226285
Limiting the amount of data downloaded is something that you will have to do by adjusting the $url

I am not sure how you have the system set up but would suggest you would need to be using CUSTOM_FEED_ID instead of PRODUCTDB_ID

Is there a way you can set up a custom feed in your perfb.com prefs?
0
 

Author Comment

by:rafique12
ID: 24226474
Not actually sure...I need to check...I'll accept the solution and try to get this sorted thanks pal
0
 
LVL 6

Expert Comment

by:basic612
ID: 24226488
Great. Glad to help!
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this blog, we’ll look at how improvements to Percona XtraDB Cluster improved IST performance.
Backups and Disaster RecoveryIn this post, we’ll look at strategies for backups and disaster recovery.
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
The viewer will learn how to look for a specific file type in a local or remote server directory using PHP.
Suggested Courses
Course of the Month13 days, 18 hours left to enroll

807 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question