Solved

Pass csv data to query for XML data to combine in csv using PHP

Posted on 2008-10-31
45
495 Views
Last Modified: 2013-12-13
I have figured out various parts to the final outcome online.  I am just beginning to learn PHP.  I am trying to get the value of a field from a csv to pass to the creation of a URL which supplies an XML file and then pull certain information from that xml file to store with each returned value and then save in a csv file that is comma seperated with the fields enclosed in quotes.  The XML file looks like:
http://shopping.yahooapis.com/ShoppingService/V3/productSearch?appid=YahooDemo&class=catalogs&query=32P0768

I have created php to parse and display portions of the xml and to parse and display results from the existing csv but have no idea how to put them together.

Exisitng CSV file that will be read from to pass it's value to get xml data is:
SKU|MANUFACTURER|CATEGORY
32P0768|IBM|
92P1102|Lenovo|
This provides a search box and passes that to create the url, then reads and outputs the XML using magicparser

<?php

  require("MagicParser.php");

  ini_set('user_agent', 'Opera - Opera/9.00 (Windows NT 5.1; U; en)');

  if (!$_GET["q"]) $_GET["q"] = "Xseries";

  print "<h1>Yahoo Shopping API</h1>";

  print "<form method='GET'>";

  print "<input type='text' name='q' value='".htmlentities($_GET["q"])."' /> ";

  print "<input type='submit' value='Search' />";

  print "</form>";

  function myRecordHandler($item)

  {

    print "<h3><a href='".$item["URL"]."'>".$item["SUMMARY"]."</a></h3>";

    print "<img src='".$item["GRIDIMAGE/URL"]."' />";

    print "<p>".$item["PRODUCTNAME"]."</p>";

    print "<p>".$item["DESCRIPTION"]."</p>";

    print "<h4>UPC:".$item["UPC"]."</h4>";

    print "<h4>Manufacturer:".$item["BRAND"]."</h4>";

    print "<h4>Category:".$item["CATEGORY/TITLE"]."</h4>";

    print "<h4>Pricing: $".$item["PRICEFROM"]. " - $".$item["PRICETO"]."</h4>";

	$specificationLabel = "";

    $specificationValue = "";

    foreach($item as $key => $value)

    {

      if (strpos($key,"SPECIFICATIONLABEL")!==FALSE) $specificationLabel = $value;

      if (strpos($key,"SPECIFICATIONVALUE")!==FALSE) $specificationValue = $value;

      if ($specificationLabel && $specificationValue)

      {

        // use $specificationLabel and $specificationValue here

        // reset for next pair

	print "<ul>";

	print "<li>".$specificationLabel." : ";

	print $specificationValue."</li>";

	print "</ul>";

		$specificationLabel = "";

        $specificationValue = "";

      }

    }
 

    print "<br /><br />";

  }

  if ($_GET["q"])

  {

    // construct Yahoo Web Services Query URL

    $url  = "http://shopping.yahooapis.com/ShoppingService/V3/productSearch?";

    $url .= "appid=YahooDemo";

    $url .= "&class=catalogs";

    $url .= "&query=".urlencode($_GET["q"]);

    // fetch the response and parse the resultsz

    MagicParser_parse($url,"myRecordHandler","xml|PRODUCTSEARCH/PRODUCTS/PRODUCT/CATALOG/");

  }

?>
 

This php reads and outputs the existing csv file which I have specified it's format above.
 

<?php
 

// reads a csv file and returns a two-dimensional array of lines/fields

function read_csv($file,$delimiter)

{

 $data_array = file($file);

 for ( $i = 0; $i < count($data_array); $i++ )

 {

  $parts_array[$i] = explode($delimiter,$data_array[$i]);

 }

 return $parts_array;

}
 

// reads a csv file and returns an two-dimensional array of lines/fields

function select_csv($file,$delimiter,$field,$query)

{

 $data_array = file($file);

 for ( $i = 0; $i < count($data_array); $i++ )

 {

  $parts_array[$i] = explode($delimiter,$data_array[$i]);

  if(trim(strtolower($parts_array[$i][$field])) == trim(strtolower($query)))

  {

   $result_array[] = $parts_array[$i];

  }

 }

 return $result_array;

}
 
 

// ------------------- demonstration below --------------------
 
 

// this willl display all records in the csv file

$data = read_csv('read_csv.txt','|');

for ( $i = 0; $i < count($data); $i++ )

{

 for ( $u = 0; $u < count($data[$i]); $u++ )

 {

  echo $data[$i][$u].' ';

  if($data[$i][$u] == end($data[$i]))

  {

   echo '<br>';  

  }

 }

}
 

echo '<p>'; 

 
 
 

// this willl display all records where the value

// of the selected field matches the query

$data = select_csv('read_csv.txt','|','0','32P0768');

for ( $i = 0; $i < count($data); $i++ )

{

 for ( $u = 0; $u < count($data[$i]); $u++ )

 {

  echo $data[$i][$u].' ';

 }

 echo '<br>';  

}
 

?>

Open in new window

0
Comment
Question by:websuperman
  • 27
  • 18
45 Comments
 
LVL 2

Author Comment

by:websuperman
ID: 22853140
Maybe to further clarify, the end result would then be
"SKU", "Manufacturer","Category","ProductName","Summary","Description","GridImageURL"
"32P0768","IBM","Hard Drives","IBM <b>32P0768</b> Hard Drive","73.4 GB - Hot-swap Drive - 15000 rpm - 3.6 ms Access Time - 2Gb Fibre Channel -Package Quantity 1","A higher-performance 2 Gigabit/second (2 Gbps), hot-swappable, 73.4 GB Fiber Channel Disk Drive Module is available for the IBM TotalStorage FAStT Storage Server Family. Operating at 15, 000 (15K) rpm, this disk drive module provides increased performance compared to disk modules of the same capacity that operate at 10, 000 (10K) rpm. <p>When used in FAStT configurations consisting of the FAStT700 Storage Servers and FAStT EXP700 Expansion Units, the 73.4 GB/15K disk module operates up to a full 2 Gbps, end-to-end fiber channel speed. You may also use this disk module in the FAStT200 Storage Servers and FAStT EXP500 Expansion Units. This module provides a higher performance, 73.4 GB disk drive module for these FAStT enclosures operating at 1 Gbps fiber channel speed compared to disk modules operating at 10K rpm.","http://f5c.yahoofs.com/shopping/3053064/simg_t_tsimg_t_oi01190jpg110?rm_____DgMVnwDEp"
0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 22862607
Hi, websuperman.  

This string is what is called "pipe-delimited" because of those vertical bars and their resemblance to pipes.  It's one of several useful delimiters, and one I like because there is almost no use for the pipes in normal text (they get used in logic functions, etc., but not much in conversation or writing).

32P0768|IBM|
92P1102|Lenovo|

You can use the "explode" function to turn a pipe-delimited string into an array, and they you can pluck the fields you need out of the array.  Man Page here:
http://us.php.net/manual/en/function.explode.php

Now if you can help clarify the rest of the question, I'll be glad to try to answer. ~Ray
$string = "32P0768|IBM|";

$array  = explode('|', $string);

$sku = array[0]; // 32P0768

$man = array[1]; // IBM

$cat = array[2]; // [empty] NOTHING AFTER LAST PIPE

Open in new window

0
 
LVL 2

Author Comment

by:websuperman
ID: 22867802
Basically I'll have the pipe delimited file and will need to use the sku in the URL I create to pull the xml data for each part and then capture parts of that xml such as the listed manufacturer, category, description, summary, productname, upc, image, etc.  These fields are not necessarily available for every sku and for some sku's nothing will be available.  I then need to combine the information I had in the existing pipe delimited file with the newly retrieved information in a new file.  
0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 22867866
So long as you have the correct number of delimiters, there is no need to have data in the fields.  This is a perfectly valid string (containing several empty fields):

$string = "32P0768|IBM|||||||";

Following up on y earlier post, see the code snippet.
// GET SOME DATA - MAYBE $string GETS SET INSIDE AN ITERATIVE LOOP

$string = "32P0768|IBM|";

$array  = explode('|', $string);

$sku = array[0]; // 32P0768

$man = array[1]; // IBM

$cat = array[2]; // [empty] NOTHING AFTER LAST PIPE
 

// CREATE A URL WITH THE SKU IN THE GET ARGUMENTS

$url = "http://www.mywebsite.com/mypage.php?sku=" . "$sku";

Open in new window

0
 
LVL 2

Author Comment

by:websuperman
ID: 22868048
I tried using your code and get the following error on line 4:
Parse error: syntax error, unexpected '[', expecting '

I tried changing the [] to ' ' and then get unexpected T_CONSTANT_ENCAPSED_STRING, just copied and pasted the above code and enclosed in php tags.
0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 22868115
You might want to get a book on PHP syntax and have it at hand for a reference.  

Please post the code that gave you the syntax error.  Thanks, ~Ray
0
 
LVL 2

Author Comment

by:websuperman
ID: 22868159
Here is the code I tried.
<?php

// GET SOME DATA - MAYBE $string GETS SET INSIDE AN ITERATIVE LOOP

$string = "32P0768|IBM|";

$array  = explode('|', $string);

$sku = array[0]; // 32P0768

$man = array[1]; // IBM

$cat = array[2]; // [empty] NOTHING AFTER LAST PIPE

 

// CREATE A URL WITH THE SKU IN THE GET ARGUMENTS

$url = "http://www.mywebsite.com/mypage.php?sku=" . "$sku";

print $url;

?>

Open in new window

0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 22868188
Thanks.  Here's a better copy.  Variables all have to start with $ in PHP.  Sorry for my typo!
<?php

// GET SOME DATA - MAYBE $string GETS SET INSIDE AN ITERATIVE LOOP

$string = "32P0768|IBM|";

$array  = explode('|', $string);

$sku = $array[0]; // 32P0768

$man = $array[1]; // IBM

$cat = $array[2]; // [empty] NOTHING AFTER LAST PIPE
 

// CREATE A URL WITH THE SKU IN THE GET ARGUMENTS

$url = "http://www.mywebsite.com/mypage.php?sku=" . "$sku";

print $url;

?>

Open in new window

0
 
LVL 2

Author Comment

by:websuperman
ID: 22868729
I feel like I have the pieces but I'm not sure how to put them together.  The following reads a csv and I can have it display but I have been unable to break out the array into fields.

I commented out the return $parts_array and tried inputting what you supplied and obviously I didn't do it right.
$file = 'read_csv.txt';

$delimiter = '|';

// reads a csv file and returns a two-dimensional array of lines/fields

function read_csv($file,$delimiter)

{

 $data_array = file($file);

 for ( $i = 0; $i < count($data_array); $i++ )

 {

  $parts_array[$i] = explode($delimiter,$data_array[$i]);

 }

//return $parts_array;

$sku = $parts_array[0]; // 32P0768

$man = $parts_array[1]; // IBM

$cat = $parts_array[2]; // [empty] NOTHING AFTER LAST PIPE

}

Open in new window

0
 
LVL 2

Author Comment

by:websuperman
ID: 22871602
This prints the URL to the screen with the data in the proper place but repeats once for each column and I only need it to provide the result once.  $i+1 skips the first line.

$data = read_csv('read_csv.txt','|');

for ( $i = 0; $i < count($data); $i++ )

{

 for ( $u = 0; $u < count($data[$i]); $u++ )

 {

  echo  $url .$data[$i][0].'';

  if($data[$i][$u] == end($data[$i]))

  {

   echo '<br>';  

  }

 }

}

Open in new window

0
 
LVL 2

Author Comment

by:websuperman
ID: 22871638
Sorry, this skips the first line so column headers aren't there, still repeating once for each column

echo  $url .$data[$i+1][0].'';
0
 
LVL 2

Author Comment

by:websuperman
ID: 22871689
Ok, this works better, but providing an extra line at the end.  
Output looks like:
http://shopping.yahooapis.com/ShoppingService/V3/productSearch?appid=YahooDemo&class=catalogs&query=32P0768
http://shopping.yahooapis.com/ShoppingService/V3/productSearch?appid=YahooDemo&class=catalogs&query=92P1102
http://shopping.yahooapis.com/ShoppingService/V3/productSearch?appid=YahooDemo&class=catalogs&query=74P6733
http://shopping.yahooapis.com/ShoppingService/V3/productSearch?appid=YahooDemo&class=catalogs&query=286716-B22
http://shopping.yahooapis.com/ShoppingService/V3/productSearch?appid=YahooDemo&class=catalogs&query=


There is no blank line at the end of the txt file either.
$data = read_csv('read_csv.txt','|');

for ( $i = 0; $i < count($data); $i++ )

{

  echo  $url .$data[$i+1][0].'<br>';

  if($data[$i][$u] == end($data[$i]))

  { 

 }

}

Open in new window

0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 22871880
Maybe something like this?
<?php

$urls   = array();

$my_url = "http://shopping.yahooapis.com/ShoppingService/V3/productSearch?appid=YahooDemo&class=catalogs&query=";

$data   = file("read_csv.txt");

$delim  = '|'; // PIPE
 

// PROCESS EACH OF THE LINES OF THE FILE

foreach ($data as $line) {

   $x = explode($delim, $line);

   if (empty($x)) continue;

   $sku = $x[0];

   $urls[] = "$my_url" . "$sku";

}
 

var_dump($urls);

?>

Open in new window

0
 
LVL 2

Author Comment

by:websuperman
ID: 22872478
That does work and should take care of step 1 of the process.  The way I see it is.
Step 1. Make URLs where XML resides
Step 2. Read the XML from URLs provided in Step 1
Step 3. "Insert" desired data from Step 2 into each "record"
Step 4. Create CSV file containing desired data.
0
 
LVL 2

Author Comment

by:websuperman
ID: 22873703
I'm almost there, my big hangup right now is not being able to use $sku outside the foreach statement, I tried first declaring
global $sku;
but that didn't seem to work.
0
 
LVL 2

Author Comment

by:websuperman
ID: 22878719
I am almost there, I am able to pull the xml data and write it to a file, although it throws an error, it still writes the file.  If you can help me get the $sku from the foreach statement so I can use in a seperate function I think I will have the solution.  

I am using the following to parse the XML and write to a file.  I can't seem to get the sku that was passed to the url to make getting the information possible, however.

The Error:
Warning: fwrite(): supplied argument is not a valid stream resource
Again, it gives an error, but does write the file.

Also I am able to get the sku out as an array by adding to your most recent supplied code:
$skus   = array();
$skus[] = "$sku";

I am able to see the results by doing the var_dump($skus); but have not been able, still, to use it in creating the file.

function myRecordHandler($item)

{

/* This writes to a file but I need the sku that was used to create the URL that this function is pulling data from, also 

throws an error saying the fwrite line is not a valid stream resource but it creates the file

*/

   global $outfile;

    fwrite($outfile,$item["UPC"] . "|" . $item["CATEGORY/TITLE"] . "|" . $item["BRAND"] . "\r");

    $outfile = fopen("new.txt",'a'); 

  if (!$outfile) die("Could not create output file");
 

  }

fclose($outfile); 

Open in new window

0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 22878823
Check that you have used fopen() correctly.  I can't see where $outfile came from.

Also, you MUST test the return values from PHP commands.  A statement like fwrite() returns information to your program. If you do not test that, you do not know whether your program worked!

You can find the man page for fwrite, as well as user-contributed notes on the PHP web site.  All the functions are documented that way - very helpful.
http://us3.php.net/manual/en/function.fwrite.php
0
 
LVL 2

Author Comment

by:websuperman
ID: 22880662
Is this the part you're talking about:
  $outfile = fopen("new.txt",'a');

The rest I just don't comprehend.  I have tried everything that makes sense to me with my limited knowledge of PHP.  I tried putting the fwrite within a foreach and iterative loop and each time the results are not correct.  I have read the fopen() man page several times and I just don't see anything that I am missing. I still get the error but the file is written so I'm not sure what to do about that.  If I loop the file to get the sku again and try to pass that along with the xm fields, when it works it seems to give try to apply all the xml fields to each part number thus multiplying the lines and providing unusable data.
0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 22881383
Sorry- I have been busy with "reall work" but let me see if I am understanding this...

Your code snippet above seems to have the fopen() statement after the fwrite() statement.  Those should be reversed, I think.  Also, I'm not sure I understand why $outfile would be a global variable, since it's the file handle.  Is it used outside of the function?  

Here is how I might write the function... and the calling sequence.
function myRecordHandler($item) { // APPEND ELEMENTS FROM THE $item ARRAY TO A FILE
 

// CONSTRUCT THE DATA STRING

   $string = $item["UPC"] . "|" . $item["CATEGORY/TITLE"] . "|" . $item["BRAND"] . "\r";

   $str_length = strlen($string);
 

// TRY TO OPEN THE FILE

   if (!$outfile = fopen("new.txt",'a')) {

return FALSE;

}
 

// WRITE TO THE FILE

   if (!$str_count = fwrite($outfile, $string) {

return FALSE;

}
 

// CORRECT NUMBER OF BYTES?

   if ($str_count != $str_length) {

return FALSE;

}
 

   fclose($outfile);

return TRUE;

} // END FUNCTION
 

//

// CALLING SEQUENCE

//
 

/* PREPARE $item ARRAY */

if (!myRecordHandler($item)) { die("myRecordHandler failed"); }

Open in new window

0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 22881410
Also, one other thing... You should set up a rigid test-case environment.  This means that you start with 100% new files for everything.  Delete ALL the old files every time you start a test.  That way you won't look at your test output and think that nothing changed from last time.  If you don't do this, you are at risk of having your test fail and leave the old data in the output files.  You can be looking at the output of the last test, but you won't know it!
0
 
LVL 2

Author Comment

by:websuperman
ID: 22888597
Certainly.  I really appreciate your help.  Is there any way to pass the sku from this function:
// PROCESS EACH OF THE LINES OF THE FILE
foreach ($data as $line) {
   $x = explode($delim, $line);
   if (empty($x)) continue;
   $sku = $x[0];
   $mfg = $x[1];

to this part of the next function:
$string = $item["UPC"] . "|" . $item["CATEGORY/TITLE"] . "|" . $item["BRAND"] . "\r";
I need to reference the sku along with it's matching upc, category, and brand.  Also, the file created creates one additional line that just contains pipes and one line after that which is blank.  I do appreciate your assistance.
0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 22890269
You can pass arguments between functions by putting the arguments into the function definitions or making the variables global.  

I really think you might benefit from this book.  It has a lot of help on the kinds of questions you are asking.
http://www.sitepoint.com/books/phpmysql1/
<?php
 

// EXAMPLE WITH GLOBAL VARIABLE

function echox(){

   global $x;

   echo $x;

}

$x = 'Hello';

echox();
 

// EXAMPLE WITH PASSED ARGUMENT

function echox($var){

   echo $var;

}

$x = 'World';

echox($x);

Open in new window

0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 2

Author Comment

by:websuperman
ID: 22891254
Thanks for the link to the book.  It looks very helpful and I will be getting it.  I tried to utilize the latest code provided integrated into what I've put together thus far without success.  Could this be because the $sku variable is being set in a foreach that is reading from the txt file?  I tried declaring the function inside and outside the foreach and both versions supplied without success.  I'll post the applicable code in the snippet box for easier reading.
// This gets the data to pass to the url, not complete

foreach ($data as $line) {

   $x = explode($delim, $line);

   if (empty($x)) continue;

	$sku = $x[0];

}

/* This is using the URL via MagicParser to pull the XML and outputs

properly other than not being able to get the sku */

string = $sku . "|" . $item["UPC"] . "|" . $item["CATEGORY/TITLE"] . "|" . $item["BRAND"] . "\r";

/*Obviously $sku isn't getting anything and I have removed my failed attempts at utilizing the new code */

Open in new window

0
 
LVL 108

Accepted Solution

by:
Ray Paseur earned 500 total points
ID: 22891690
There is not enough information in the snippet box to tell what is happening.  Where did $data come from? What's in $delim?  Without knowing these answers, I would only be speculating.

As a matter of habit and good coding practice, you should declare all your functions at the top or bottom of your script so you (or your successor) can find them.  You can use them anywhere they make sense, but they need to be declared all in one place.

It looks like you are dealing with a data-dependent issue.  You might want to sprinkle var_dump() commands at appropriate intervals throughout you code,  Read up on var_dump() here and try it as shown in the code snippet.

http://us.php.net/manual/en/function.var-dump.php


// This gets the data to pass to the url, not complete

foreach ($data as $line) {

   $x = explode($delim, $line);

   var_dump($x); // WHAT IS IN THIS ARRAY?

   if (empty($x)) continue;

	$sku = $x[0];

}

/* This is using the URL via MagicParser to pull the XML and outputs

properly other than not being able to get the sku */

string = $sku . "|" . $item["UPC"] . "|" . $item["CATEGORY/TITLE"] . "|" . $item["BRAND"] . "\r";

/*Obviously $sku isn't getting anything and I have removed my failed attempts at utilizing the new code */

Open in new window

0
 
LVL 2

Author Comment

by:websuperman
ID: 22925480
Here is the full code, provided by you, modified by me.  Under // CONSTRUCT THE DATA STRING you'll see I have inputed $sku to try to get the sku to put into the data file (without success) this is the issue I am having at the moment.  
<?php

  require("MagicParser.php");

  ini_set('user_agent', 'Opera - Opera/9.00 (Windows NT 5.1; U; en)');

$urls   = array();

$skus   = array();

$mfgs   = array();

$my_url = "http://shopping.yahooapis.com/ShoppingService/V3/productSearch?appid=YahooDemo&class=catalogs&query=";

$data   = file("read_csv.txt");

$delim  = '|'; // PIPE

// PROCESS EACH OF THE LINES OF THE FILE

foreach ($data as $line) {

   $x = explode($delim, $line);

   if (empty($x)) continue;

	$sku = $x[0];

   $mfg = $x[1];	

   $skus[] = "$sku";	

   $mfgs[] = "$mfg";

MagicParser_parse($urls[] = "$my_url" . "$sku" . "&category=Computers%20%26%20Software&refinement=4168-Brand=" ."$mfg","myRecordHandler","xml|PRODUCTSEARCH/PRODUCTS/PRODUCT/CATALOG/");

}

function myRecordHandler($item) { // APPEND ELEMENTS FROM THE $item ARRAY TO A FILE

 

// CONSTRUCT THE DATA STRING

   $string = $sku . "|" . $item["UPC"] . "|" . $item["CATEGORY/TITLE"] . "|" . $item["BRAND"] . "\r";

   $str_length = strlen($string);

 

// TRY TO OPEN THE FILE

   if (!$outfile = fopen("new.txt",'a')) {

return FALSE;

}

 

// WRITE TO THE FILE

   if (!$str_count = fwrite($outfile, $string)) {

return FALSE;

}

 

// CORRECT NUMBER OF BYTES?

   if ($str_count != $str_length) {

return FALSE;

}

 

   fclose($outfile);

return TRUE;

} // END FUNCTION

 

//

// CALLING SEQUENCE

//

 

/* PREPARE $item ARRAY */

if (!myRecordHandler($item)) { die("myRecordHandler failed"); }

?>

Open in new window

0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 22925535
I don't know what that Magic Parser thing is, but I'm wondering what you find when you look in the array named $urls.  Try printing that out.

Also, where is the $item variable getting set?  Are you using it?
0
 
LVL 2

Author Comment

by:websuperman
ID: 22926270
$item variable is part of magicparser and is required to be followed by the "field" containing the needed data from the xml file.   var_dump($urls) produces the following
array(1) { [0]=> string(224) "http://shopping.yahooapis.com/ShoppingService/V3/productSearch?appid=YahooDemo&class=catalogs&query=32P0768&category=Computers%20%26%20Software&refinement=4168-Brand=IBM" } array(2) { [0]=> string(224) "http://shopping.yahooapis.com/ShoppingService/V3/productSearch?appid=YahooDemo&class=catalogs&query=32P0768&category=Computers%20%26%20Software&refinement=4168-Brand=IBM" [1]=> string(227) "http://shopping.yahooapis.com/ShoppingService/V3/productSearch?appid=YahooDemo&class=catalogs&query=92P1102&category=Computers%20%26%20Software&refinement=4168-Brand=Lenovo" } array(3) { [0]=> string(224) "http://shopping.yahooapis.com/ShoppingService/V3/productSearch?appid=YahooDemo&class=catalogs&query=32P0768&category=Computers%20%26%20Software&refinement=4168-Brand=IBM" [1]=> string(227) "http://shopping.yahooapis.com/ShoppingService/V3/productSearch?appid=YahooDemo&class=catalogs&query=92P1102&category=Computers%20%26%20Software&refinement=4168-Brand=Lenovo" [2]=> string(224) "http://shopping.yahooapis.com/ShoppingService/V3/productSearch?appid=YahooDemo&class=catalogs&query=74P6733&category=Computers%20%26%20Software&refinement=4168-Brand=IBM" } array(4) { [0]=> string(224) "http://shopping.yahooapis.com/ShoppingService/V3/productSearch?appid=YahooDemo&class=catalogs&query=32P0768&category=Computers%20%26%20Software&refinement=4168-Brand=IBM" [1]=> string(227) "http://shopping.yahooapis.com/ShoppingService/V3/productSearch?appid=YahooDemo&class=catalogs&query=92P1102&category=Computers%20%26%20Software&refinement=4168-Brand=Lenovo" [2]=> string(224) "http://shopping.yahooapis.com/ShoppingService/V3/productSearch?appid=YahooDemo&class=catalogs&query=74P6733&category=Computers%20%26%20Software&refinement=4168-Brand=IBM" [3]=> string(226) "http://shopping.yahooapis.com/ShoppingService/V3/productSearch?appid=YahooDemo&class=catalogs&query=286716-B22&category=Computers%20%26%20Software&refinement=4168-Brand=HP" } 

Open in new window

0
 
LVL 2

Author Comment

by:websuperman
ID: 22926297
I should point out that the var_dump was done inside the foreach statement where the data seems readily available.  Outside the foreach the result is:
NULL NULL NULL NULL NULL
Herein lies the problem, I think, as I need the sku from each "loop" in the foreach outside the foreach later on in the function myRecordHandler($item)
0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 22926801
One more time... where is the $item variable getting set?  Are you using it?
0
 
LVL 2

Author Comment

by:websuperman
ID: 22927602
$item is part of the magicparser requirements.  Anything put in the function used by MagicParser would be set up in the way seen above.
function myRecordHandler($item)
requires that the fields I want to use are prefaced by whatever variable is contained in the function so if the above was instead
function myRecordHandler($somethingdifferent)
then within the function I would use
$somethingdifferent["UPC"]
this magic is all handled by magicparser.  http://www.magicparser.com
So as far as I know it is being set to the value of the referenced xml field, which, again, is working fine.  The issue is somehow also pulling the $sku we got in the foreach statement to create the URL that we are passing to magicparser to get the xml files and put that $sku inline with the data we are pulling from those xml files.

The short of it is it is being set by magicparser and I am using it.
0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 22938766
Looks like I'll need to spend some time with the Magic Parser.  But before I do, please tell me what version of PHP you are running?  You can find this (and a lot more) with phpinfo().

Also, from the code above, it looks like you are not testing the return code from Magic Parser.  You should add a test to see if it returns True or False, and you should use the error function documented here: http://www.magicparser.com/node/20

Once you've added the test and the code to evaluate MagicParser_getErrorMessage(), please post the new code again, so I can copy it and try to run it on my own system.

Thanks, ~Ray
<?php phpinfo(); ?>

Open in new window

0
 
LVL 2

Author Comment

by:websuperman
ID: 22940602
PHP Version 5.2.6, adding MagicParser_getErrorMessage() just before the ?> allows the script to run and no error messages are produced.  I don't believe that MagicParser is producing any errors, just that I'm not properly getting the $sku into the function myRecordHandler($item).  I assume this is because $sku is set in a foreach and the function is outside of this foreach.  I started thinking that the likely correction for this is to have a seperate function that receives the URL and SKU and then pass from that function to the function myRecordHandler($item) as the foreach is passing what I ask it to during the foreach but outside of the foreach the data isn't available.  Does that make sense?
0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 22940699
Did you consider using the simpleXML functions?  If not, can you tell us why not?

But to this statement:" I don't believe that MagicParser is producing any errors" you should be using an IF statement to test the return.

Let's try this one again (seems to be a pattern of incomplete responses to EE requests?)

Once you've added the test and the code to evaluate MagicParser_getErrorMessage(), please post the new code again, so I can copy it and try to run it on my own system.
0
 
LVL 2

Author Comment

by:websuperman
ID: 22944218
I apologize if my responses aren't complete.  It's not intentional, I am just beginning my journey into PHP and likely am guilty of doing to much "troubleshooting" from my limited knowledge.  I tried the getErrorMessage() in the foreach and at the end with the same results, no errors.  Here is the code modified to how I believe you are asking me to modify
<?php

  require("MagicParser.php");

  ini_set('user_agent', 'Opera - Opera/9.00 (Windows NT 5.1; U; en)');

$urls   = array();

$skus   = array();

$mfgs   = array();

$my_url = "http://shopping.yahooapis.com/ShoppingService/V3/productSearch?appid=YahooDemo&class=catalogs&query=";

$data   = file("read_csv.txt");

$delim  = '|'; // PIPE

// PROCESS EACH OF THE LINES OF THE FILE

foreach ($data as $line) {

   $x = explode($delim, $line);

   if (empty($x)) continue;

	$sku = $x[0];

   $mfg = $x[1];	

   $skus[] = "$sku";	

   $mfgs[] = "$mfg";

MagicParser_parse($urls[] = "$my_url" . "$sku" . "&category=Computers%20%26%20Software&refinement=4168-Brand=" ."$mfg","myRecordHandler","xml|PRODUCTSEARCH/PRODUCTS/PRODUCT/CATALOG/");

}

function myRecordHandler($item) { // APPEND ELEMENTS FROM THE $item ARRAY TO A FILE

 var_dump($urls);

// CONSTRUCT THE DATA STRING

   $string = $sku . "|" . $item["UPC"] . "|" . $item["CATEGORY/TITLE"] . "|" . $item["BRAND"] . "\r";

   $str_length = strlen($string);

 

// TRY TO OPEN THE FILE

   if (!$outfile = fopen("new.txt",'a')) {

return FALSE;

} 

 

// WRITE TO THE FILE

   if (!$str_count = fwrite($outfile, $string)) {

return FALSE;

}

 

// CORRECT NUMBER OF BYTES?

   if ($str_count != $str_length) {

return FALSE;

}

 

   fclose($outfile);

return TRUE;

} // END FUNCTION

 

//

// CALLING SEQUENCE

//
 

/* PREPARE $item ARRAY */

if (!myRecordHandler($item)) { die("myRecordHandler failed"); }

if (!$result)

  { 

    print MagicParser_getErrorMessage();

  }

?>

Open in new window

0
 
LVL 2

Author Comment

by:websuperman
ID: 22944342
Only thing missing is the sku in the file, current file created looks like:
|087944839477|Hard Drives|IBM

|000435609328|Laptop Batteries|Lenovo

|087944862765|More Computer Accessories|Lenovo

|613326447765|Hard Drives|HP

|||

Open in new window

0
 
LVL 2

Author Comment

by:websuperman
ID: 22944707
Corrected code below which would actually show results for the errors if they exist and prints the $skus from vardump but not if it is located inside the function, so there must be a way to break these skus from the array and push them into the MyRecordHandler function?  I did notice the string number changes for the last result, not sure if that means anyting, here's the vardump output as well:
array(4) { [0]=> string(7) "32P0768" [1]=> string(7) "92P1102" [2]=> string(7) "74P6733" [3]=> string(10) "286716-B22" }
<?php

  require("MagicParser.php");

  ini_set('user_agent', 'Opera - Opera/9.00 (Windows NT 5.1; U; en)');

$urls   = array();

$skus   = array();

$mfgs   = array();

$my_url = "http://shopping.yahooapis.com/ShoppingService/V3/productSearch?appid=YahooDemo&class=catalogs&query=";

$data   = file("read_csv.txt");

$delim  = '|'; // PIPE

// PROCESS EACH OF THE LINES OF THE FILE

foreach ($data as $line) {

   $x = explode($delim, $line);

   if (empty($x)) continue;

	$sku = $x[0];

   $mfg = $x[1];	

   $skus[] = "$sku";	

   $mfgs[] = "$mfg";

$result = MagicParser_parse($urls[] = "$my_url" . "$sku" . "&category=Computers%20%26%20Software&refinement=4168-Brand=" ."$mfg","myRecordHandler","xml|PRODUCTSEARCH/PRODUCTS/PRODUCT/CATALOG/");

} 

var_dump($skus);

function myRecordHandler($item) { // APPEND ELEMENTS FROM THE $item ARRAY TO A FILE
 

// CONSTRUCT THE DATA STRING

   $string = $sku . "|" . $item["UPC"] . "|" . $item["CATEGORY/TITLE"] . "|" . $item["BRAND"] . "\r";

   $str_length = strlen($string);

 

// TRY TO OPEN THE FILE

   if (!$outfile = fopen("new.txt",'a')) {

return FALSE;

} 

 

// WRITE TO THE FILE

   if (!$str_count = fwrite($outfile, $string)) {

return FALSE;

}

 

// CORRECT NUMBER OF BYTES?

   if ($str_count != $str_length) {

return FALSE;

}

 

   fclose($outfile);

return TRUE;

} // END FUNCTION

 

//

// CALLING SEQUENCE

//
 

/* PREPARE $item ARRAY */

if (!myRecordHandler($item)) { die("myRecordHandler failed"); }

if (!$result)

  { 

    print MagicParser_getErrorMessage();

  }

?>

Open in new window

0
 
LVL 2

Author Comment

by:websuperman
ID: 22960915
I figured it out.   Found this which led the way:
 Every function that needs access to a global variable must have that variable declared as global, otherwise you will only be accessing a local copy.

Now the only issue I have is that the last line gets repeated and is definitely code related as the source file is accurate, does not have a blank line at the end or anything but when the file is written the last $sku gets repeated with no other information and then a blank line.  Any ideas?  Output below
Part1|087944839477|Hard Drives|IBM

Part2|000435609328|Laptop Batteries|Lenovo

Part3|087944862765|More Computer Accessories|Lenovo

Part4|613326447765|Hard Drives|HP

Part4|||

<then an extra blank line here>

Open in new window

0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 22962607
"Every function that needs access to a global variable must have that variable declared as global,..."

Right.  It is one of the most basic concepts of programming, documented here: http://us.php.net/global

But I'm still confused by your posts - you posted this, two posts above:
here's the vardump output as well: array(4) { [0]=> string(7) "32P0768" [1]=> string(7) "92P1102" [2]=> string(7) "74P6733" [3]=> string(10) "286716-B22" }

Then you posted this, one post above:
Part1|087944839477|Hard Drives|IBM
Part2|000435609328|Laptop Batteries|Lenovo
Part3|087944862765|More Computer Accessories|Lenovo
Part4|613326447765|Hard Drives|HP
Part4|||
<then an extra blank line here>

So it looks like the test data set has changed from one iteration to the next, and the most recently posted code segment has not got the word "global" anywhere in it.  When the code and data change, you might want to post the revised code and data.  If you don't do that, it makes it nearly impossible for us to help you - we can't keep guessing at what's happening on your end.

If you want to remove a blank line, you need to know where the blank line is coming from, and when it appears.  You can test for it in the right place and remove it there.  Similarly, you can test for the presence of a triple-pipe (indicative of empty fields) and remove that line.  You can use strpos and/or REGEX matching to find these sorts of things.

HTH, ~Ray
0
 
LVL 2

Author Comment

by:websuperman
ID: 22963417
Sorry, I changed the part numbers in the post simply to be more generic, the post with | is the output of the text file while the other is obviously the array.

I figured out how to get the last part number from repeating by unsetting the variable at the end of the foreach, however, it still repeats one line with nothing but ||| pipes and then a blank line.

I am fairly certain that this is because the last result returned by the foreach is NULL which is getting passed to the function writing the file, so it is in essence writing NULL|NULL|NULL|NULL.  

Based on this I am fairly certain the issue is having the foreach stopp when null is returned, but I am unsure how to do this properly.  I'll post the code where the trobule lkely lies below, here is what I think is happening.

It reads through gets the part numbers, everything is good, but then when it gets to NULL the explode is also pulling the delimiter and thus it's not totally null? so the if statement doesn't see null and runs that last time?  Thus the function writing to the file prints NULL and the pipes and the new line as instructed which is where the line of pipes and new line comes from.
   $x = explode($delim, $line);

   if (empty($x)) die;

Open in new window

0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 22963479
Please post the code as it is now, and I can have a look.  I'm highly suspicious that an array created by explode will not be empty.  Why not just test $line?
0
 
LVL 2

Author Comment

by:websuperman
ID: 22963902
Here's the complete code, changing the if to test $line did not change the output.
<?php

  require("MagicParser.php");

  ini_set('user_agent', 'Opera - Opera/9.00 (Windows NT 5.1; U; en)');

$urls   = array();

$skus   = array();

$mfgs   = array();

$my_url = "http://shopping.yahooapis.com/ShoppingService/V3/productSearch?appid=YahooDemo&class=catalogs&query=";

$data   = file("read_csv.txt");

$delim  = '|'; // PIPE

// PROCESS EACH OF THE LINES OF THE FILE

foreach ($data as $line) {

global $sku;

   $x = explode($delim, $line);

   if (empty($line)) die;

	$sku = $x[0];

   $mfg = $x[1];	

   $skus[] = "$sku";	

   $mfgs[] = "$mfg";

$result = MagicParser_parse($urls[] = "$my_url" . "$sku" . "&category=Computers%20%26%20Software&refinement=4168-Brand=" ."$mfg","myRecordHandler","xml|PRODUCTSEARCH/PRODUCTS/PRODUCT/CATALOG/");

unset($sku);

} 

function myRecordHandler($item) { // APPEND ELEMENTS FROM THE $item ARRAY TO A FILE

global $sku;

var_dump($sku);

// CONSTRUCT THE DATA STRING

   $string = $sku . "|" . $item["UPC"] . "|" . $item["CATEGORY/TITLE"] . "|" . $item["BRAND"] . "\r";

   $str_length = strlen($string);

 

// TRY TO OPEN THE FILE

   if (!$outfile = fopen("new.txt",'a')) {

return FALSE;

} 

 

// WRITE TO THE FILE

   if (!$str_count = fwrite($outfile, $string)) {

return FALSE;

}

 

// CORRECT NUMBER OF BYTES?

   if ($str_count != $str_length) {

return FALSE;

}

 

   fclose($outfile);

return TRUE;

} // END FUNCTION

 

//

// CALLING SEQUENCE

//
 

/* PREPARE $item ARRAY */

if (!myRecordHandler($item)) { die("myRecordHandler failed"); }

if (!$result)

  { 

    print MagicParser_getErrorMessage();

  }

?>

Open in new window

0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 22963933
Thanks.  Now, please post the contents of the input file - read_csv.txt
0
 
LVL 2

Author Comment

by:websuperman
ID: 22965768
Contents of the input file:
32P0768|IBM

92P1102|Lenovo

74P6733|IBM

286716-B22|HP

Open in new window

0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 22968048
This is not the data that caused the extra blank line.  You showed us an extra blank line with this data:

Part1|087944839477|Hard Drives|IBM
Part2|000435609328|Laptop Batteries|Lenovo
Part3|087944862765|More Computer Accessories|Lenovo
Part4|613326447765|Hard Drives|HP
Part4|||
<then an extra blank line here>

Honestly, I wonder why you keep doing this?  You seem to have great difficulty keeping the code and test data organized.  I don't think I have time to work on this any more.  Good luck, and please consider hiring a programmer to do this work, if it is at all important to you. ~Ray
0
 
LVL 2

Author Comment

by:websuperman
ID: 22968543
If you had read my post above responding to you regarding the data changing, I stated that I changed the part numbers in my post.  I did so in an effort to make it more general and to simply get the point across.  I appreciate all your help.  The data above is indeed the input file that produces the empty line.  And, as stated, I figured out how to get the last part number from repeating and am down to just the last line of pipes and a blank line.  The out put now is different and wasn't posted because I assumed that you had read my posts and were aware the output has changed because the code has changed.  I also assumed this is why you wanted to see the new code.  Again, I appreciate your help, your frustration is understood.  Hopefully someone else will chime in.  The output now, as described looks like:
32P0768|087944839477|Hard Drives|IBM

92P1102|000435609328|Laptop Batteries|Lenovo

74P6733|087944862765|More Computer Accessories|Lenovo

286716-B22|613326447765|Hard Drives|HP

|||

<then an extra blank line here>

Open in new window

0

Featured Post

Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

Join & Write a Comment

Suggested Solutions

I was working on a PowerPoint add-in the other day and a client asked me "can you implement a feature which processes a chart when it's pasted into a slide from another deck?". It got me wondering how to hook into built-in ribbon events in Office.
This article discusses how to create an extensible mechanism for linked drop downs.
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
The viewer will learn how to count occurrences of each item in an array.

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now