Solved

Skipping lines of a CSV file import in PHP

Posted on 2011-09-23
14
218 Views
Last Modified: 2012-05-12
I am importing a set of orders from a CSV file.  I want to remove certain rows from the rest of the script based on the value of a certain field.  

Would I first load the entire CSV file into an array?  How would I test and remove the rows I do not want to show up?
0
Comment
Question by:hydrazi
  • 6
  • 4
  • 3
  • +1
14 Comments
 
LVL 3

Assisted Solution

by:dkellner
dkellner earned 167 total points
Comment Utility
No, loading is not necessary.
Follow a logic like this (DON'T copy/paste this, it's only a symbolic thing):

$output=""
$f = fopen(..............)or die()
while(!feof($f)) {
   $s = fgets($f,9999); // max line size
   if(needThisLine($s)) $output.=$s;
}
fclose($f)

Write the function needThisLine to make the decision about one single CSV row (could be only 2 lines, one with explode() and another for the condition), it should return true/false - and there, you have the $output which can then be written to a file.

Ask some more if it's not clear.
0
 
LVL 3

Expert Comment

by:dkellner
Comment Utility
If your file is REALLY big, you can do it with 2 files open: one for reading, another for writing, and instead of filling a variable like "$output", write to file2 so that you only need memory space for a single line, not the whole output.
0
 

Author Comment

by:hydrazi
Comment Utility
Alright, let me see if I can figure that out.  I'm a bit of a newbie!  Many thanks for this!
0
 

Author Comment

by:hydrazi
Comment Utility
The file is pretty small.  Maybe a max of 40 rows.  
0
 
LVL 3

Expert Comment

by:dkellner
Comment Utility
If you give me some details (like the file itself) I can do it for you.  But try by yourself coz that's how newbies become pros!
0
 

Author Comment

by:hydrazi
Comment Utility
I am unsure what logic to use for the function.  If I could get a hint...  
0
 
LVL 108

Accepted Solution

by:
Ray Paseur earned 167 total points
Comment Utility
Open the file
http://us.php.net/manual/en/function.fopen.php

While the file is open and you have not found the end of file, you will have a loop
http://us.php.net/manual/en/function.feof.php

Read each line of the file into an array
http://us.php.net/manual/en/function.fgetcsv.php

Compare the fields to see if this is data you want to keep
http://us.php.net/manual/en/control-structures.if.php

If you do not want to keep this line, skip to the end of the while loop
http://us.php.net/manual/en/control-structures.continue.php

If you want to keep this line, add it to an array, something like this:
$arr[] = $line;

At the end of the while() loop you will have an array of arrays.  Each of the "sub-arrays" contains a line from the original file in the correct order.

Then you can use a foreach() iterator to get each line from the master array, and you can process them in any way you want, one line at a time.
http://us.php.net/manual/en/control-structures.foreach.php
0
Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 
LVL 33

Assisted Solution

by:Slick812
Slick812 earned 166 total points
Comment Utility
greetings hydrazi, I have looked at your question and I think that you should  "first load the entire CSV file into an array"  and then use a for loop to go through your conditions for excluding the csv  value and remove it from the array, , However, the way you might test for the removal "number" as an array element would greatly depend on what you had as the data container for the excluded array element number.
Below is some code that may help you get started,
I use a simple array to hold the values I need removed as the  $doNotUse  array
this is made to be simple to get you started, but it really depends on data set data container for the excluded array element numbers.

please ask questions if you need more information.
<?php

$doNotUse = array(2,3,6,7,12);// these are the array "rows" to delete
$aryCSV = file_get_contents('test.csv');
$aryCSV = explode(',', $aryCSV);, makes string into an array
$arySize=count($aryCSV);
for ($i = 0; $i < $arySize; ++$i){
    // use in_array() to see if the value is in the $doNotUse array
    if (in_array($i, $doNotUse)) unset($aryCSV[$i]);
    // unset() will remove the array element
    }
echo 'Comma Separated Values after Removal= ';
foreach ($aryCSV as $val) echo $val,',';
echo '<br />';

?>

Open in new window

0
 
LVL 33

Expert Comment

by:Slick812
Comment Utility
oh almost forgot, this is a plain text file for the  file  test.csv, it is One Line of text as =
0 ,1 ,2 ,3 ,4 ,5 ,6 ,7 ,8 ,9 ,10 ,11 ,12 ,13 ,14 ,15 ,16 ,17 ,18 ,19
0
 

Author Comment

by:hydrazi
Comment Utility
Alrighty..... working on this again!  Sorry for the delay!
0
 

Author Closing Comment

by:hydrazi
Comment Utility
I DID IT!  Many thanks for all the pointers!
0
 

Author Comment

by:hydrazi
Comment Utility
ooops..... ok, I tried Slick812's code, just on it's own to understand it better.  Then I tried it with our CSV file that is delimited with a "|" which I changed in the code.  However, it did not remove the offending rows.  Not even the offending field.  Here is what I changed it to, but it still prints the row I want removed:


<?php
$doNotUse = array("ppfc","plot plan");
$aryCSV = file_get_contents("orders.csv");
$aryCSV = explode('|', $aryCSV);//makes string into an array
$arySize=count($aryCSV);
for ($i = 0; $i < $arySize; ++$i){
    // use in_array() to see if the value is in the $doNotUse array
    if (in_array($i, $doNotUse)) unset($aryCSV[$i]);
    // unset() will remove the array element
    }
echo 'Comma Separated Values after Removal= ';
foreach ($aryCSV as $val) echo $val,'|';
echo '<br />';

?>

Open in new window

0
 
LVL 33

Expert Comment

by:Slick812
Comment Utility
OK, you do not seem to understand the workings of the code I supplied, because I did not know how your CSV was used as text, I used "Numbers" as 0, 1, 2, 3 as the
$doNotUse = array(2,3,6,7,12);
array,
but for you you have changed it to -
$doNotUse = array("ppfc","plot plan");

which will not work for this or any other CSV file that you use -
$aryCSV = explode('|', $aryCSV);
to get an array. The $aryCSV array will always have Numbered elements as -
$aryCSV[0] , $aryCSV[1] , $aryCSV[2],
and never any as -
$aryCSV["ppfc"], $aryCSV["plot plan"]

as I said before "However, the way you might test for the removal "number" as an array element would greatly depend on what you had as the data container for the excluded array element number."

this is the line of code that "TESTS" for a correct or incorrect element key or value in the $aryCSV -
if (in_array($i, $doNotUse)) unset($aryCSV[$i]);

you will need to change this to your own way of testing for what you want to leave in or take out of the $aryCSV array,

I suspect that the  "ppfc" and "plot plan" are values in the array and not keys, is this correct ?

0
 
LVL 33

Expert Comment

by:Slick812
Comment Utility
OK, I did something that will test for Values, instead of Keys in the CSV array, I use the PHP function array_search( ) which you can see what it does here -
http://www.php.net/manual/en/function.array-search.php

I changed the FOR loop to go through the shorter (I guess, but it may not be true depending on the way you fill the $doNotUse) $doNotUse array


The 'test.csv' file is ONE line of text as-
first 0|second 1|third 2|fourth 3|fifth 4|sixth 5|seven 6|last 7




I hope this will help, but your test to unset( ) will depend on what is in the $doNotUse  array
$doNotUse = array('second 1','bogus 99','fifth 5');
$aryCSV = file_get_contents('test.csv');
$aryCSV = explode('|', $aryCSV);
//$arySize=count($aryCSV); // old way
$arySize=count($doNotUse);// LOOK this has changed
for ($i = 0; $i < $arySize; ++$i){
	//if (in_array($i, $doNotUse)) unset($aryCSV[$i]);// old way
	$key = array_search($doNotUse[$i], $aryCSV);// LOOK this is different
	if($key) unset($aryCSV[$key]);
	}
echo 'Comma Separated Values after Removal= ';
foreach ($aryCSV as $val) echo $val,',';
echo '<br />';

Open in new window

0

Featured Post

Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

Join & Write a Comment

Author Note: Since this E-E article was originally written, years ago, formal testing has come into common use in the world of PHP.  PHPUnit (http://en.wikipedia.org/wiki/PHPUnit) and similar technologies have enjoyed wide adoption, making it possib…
Nothing in an HTTP request can be trusted, including HTTP headers and form data.  A form token is a tool that can be used to guard against request forgeries (CSRF).  This article shows an improved approach to form tokens, making it more difficult to…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
The viewer will learn how to look for a specific file type in a local or remote server directory using PHP.

728 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now