Solved

Skipping lines of a CSV file import in PHP

Posted on 2011-09-23
14
223 Views
Last Modified: 2012-05-12
I am importing a set of orders from a CSV file.  I want to remove certain rows from the rest of the script based on the value of a certain field.  

Would I first load the entire CSV file into an array?  How would I test and remove the rows I do not want to show up?
0
Comment
Question by:hydrazi
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 4
  • 3
  • +1
14 Comments
 
LVL 3

Assisted Solution

by:dkellner
dkellner earned 167 total points
ID: 36588279
No, loading is not necessary.
Follow a logic like this (DON'T copy/paste this, it's only a symbolic thing):

$output=""
$f = fopen(..............)or die()
while(!feof($f)) {
   $s = fgets($f,9999); // max line size
   if(needThisLine($s)) $output.=$s;
}
fclose($f)

Write the function needThisLine to make the decision about one single CSV row (could be only 2 lines, one with explode() and another for the condition), it should return true/false - and there, you have the $output which can then be written to a file.

Ask some more if it's not clear.
0
 
LVL 3

Expert Comment

by:dkellner
ID: 36588288
If your file is REALLY big, you can do it with 2 files open: one for reading, another for writing, and instead of filling a variable like "$output", write to file2 so that you only need memory space for a single line, not the whole output.
0
 

Author Comment

by:hydrazi
ID: 36588373
Alright, let me see if I can figure that out.  I'm a bit of a newbie!  Many thanks for this!
0
NFR key for Veeam Backup for Microsoft Office 365

Veeam is happy to provide a free NFR license (for 1 year, up to 10 users). This license allows for the non‑production use of Veeam Backup for Microsoft Office 365 in your home lab without any feature limitations.

 

Author Comment

by:hydrazi
ID: 36588385
The file is pretty small.  Maybe a max of 40 rows.  
0
 
LVL 3

Expert Comment

by:dkellner
ID: 36588452
If you give me some details (like the file itself) I can do it for you.  But try by yourself coz that's how newbies become pros!
0
 

Author Comment

by:hydrazi
ID: 36588464
I am unsure what logic to use for the function.  If I could get a hint...  
0
 
LVL 110

Accepted Solution

by:
Ray Paseur earned 167 total points
ID: 36590725
Open the file
http://us.php.net/manual/en/function.fopen.php

While the file is open and you have not found the end of file, you will have a loop
http://us.php.net/manual/en/function.feof.php

Read each line of the file into an array
http://us.php.net/manual/en/function.fgetcsv.php

Compare the fields to see if this is data you want to keep
http://us.php.net/manual/en/control-structures.if.php

If you do not want to keep this line, skip to the end of the while loop
http://us.php.net/manual/en/control-structures.continue.php

If you want to keep this line, add it to an array, something like this:
$arr[] = $line;

At the end of the while() loop you will have an array of arrays.  Each of the "sub-arrays" contains a line from the original file in the correct order.

Then you can use a foreach() iterator to get each line from the master array, and you can process them in any way you want, one line at a time.
http://us.php.net/manual/en/control-structures.foreach.php
0
 
LVL 34

Assisted Solution

by:Slick812
Slick812 earned 166 total points
ID: 36593423
greetings hydrazi, I have looked at your question and I think that you should  "first load the entire CSV file into an array"  and then use a for loop to go through your conditions for excluding the csv  value and remove it from the array, , However, the way you might test for the removal "number" as an array element would greatly depend on what you had as the data container for the excluded array element number.
Below is some code that may help you get started,
I use a simple array to hold the values I need removed as the  $doNotUse  array
this is made to be simple to get you started, but it really depends on data set data container for the excluded array element numbers.

please ask questions if you need more information.
<?php

$doNotUse = array(2,3,6,7,12);// these are the array "rows" to delete
$aryCSV = file_get_contents('test.csv');
$aryCSV = explode(',', $aryCSV);, makes string into an array
$arySize=count($aryCSV);
for ($i = 0; $i < $arySize; ++$i){
    // use in_array() to see if the value is in the $doNotUse array
    if (in_array($i, $doNotUse)) unset($aryCSV[$i]);
    // unset() will remove the array element
    }
echo 'Comma Separated Values after Removal= ';
foreach ($aryCSV as $val) echo $val,',';
echo '<br />';

?>

Open in new window

0
 
LVL 34

Expert Comment

by:Slick812
ID: 36593434
oh almost forgot, this is a plain text file for the  file  test.csv, it is One Line of text as =
0 ,1 ,2 ,3 ,4 ,5 ,6 ,7 ,8 ,9 ,10 ,11 ,12 ,13 ,14 ,15 ,16 ,17 ,18 ,19
0
 

Author Comment

by:hydrazi
ID: 36910289
Alrighty..... working on this again!  Sorry for the delay!
0
 

Author Closing Comment

by:hydrazi
ID: 36910381
I DID IT!  Many thanks for all the pointers!
0
 

Author Comment

by:hydrazi
ID: 36912058
ooops..... ok, I tried Slick812's code, just on it's own to understand it better.  Then I tried it with our CSV file that is delimited with a "|" which I changed in the code.  However, it did not remove the offending rows.  Not even the offending field.  Here is what I changed it to, but it still prints the row I want removed:


<?php
$doNotUse = array("ppfc","plot plan");
$aryCSV = file_get_contents("orders.csv");
$aryCSV = explode('|', $aryCSV);//makes string into an array
$arySize=count($aryCSV);
for ($i = 0; $i < $arySize; ++$i){
    // use in_array() to see if the value is in the $doNotUse array
    if (in_array($i, $doNotUse)) unset($aryCSV[$i]);
    // unset() will remove the array element
    }
echo 'Comma Separated Values after Removal= ';
foreach ($aryCSV as $val) echo $val,'|';
echo '<br />';

?>

Open in new window

0
 
LVL 34

Expert Comment

by:Slick812
ID: 36918419
OK, you do not seem to understand the workings of the code I supplied, because I did not know how your CSV was used as text, I used "Numbers" as 0, 1, 2, 3 as the
$doNotUse = array(2,3,6,7,12);
array,
but for you you have changed it to -
$doNotUse = array("ppfc","plot plan");

which will not work for this or any other CSV file that you use -
$aryCSV = explode('|', $aryCSV);
to get an array. The $aryCSV array will always have Numbered elements as -
$aryCSV[0] , $aryCSV[1] , $aryCSV[2],
and never any as -
$aryCSV["ppfc"], $aryCSV["plot plan"]

as I said before "However, the way you might test for the removal "number" as an array element would greatly depend on what you had as the data container for the excluded array element number."

this is the line of code that "TESTS" for a correct or incorrect element key or value in the $aryCSV -
if (in_array($i, $doNotUse)) unset($aryCSV[$i]);

you will need to change this to your own way of testing for what you want to leave in or take out of the $aryCSV array,

I suspect that the  "ppfc" and "plot plan" are values in the array and not keys, is this correct ?

0
 
LVL 34

Expert Comment

by:Slick812
ID: 36919851
OK, I did something that will test for Values, instead of Keys in the CSV array, I use the PHP function array_search( ) which you can see what it does here -
http://www.php.net/manual/en/function.array-search.php

I changed the FOR loop to go through the shorter (I guess, but it may not be true depending on the way you fill the $doNotUse) $doNotUse array


The 'test.csv' file is ONE line of text as-
first 0|second 1|third 2|fourth 3|fifth 4|sixth 5|seven 6|last 7




I hope this will help, but your test to unset( ) will depend on what is in the $doNotUse  array
$doNotUse = array('second 1','bogus 99','fifth 5');
$aryCSV = file_get_contents('test.csv');
$aryCSV = explode('|', $aryCSV);
//$arySize=count($aryCSV); // old way
$arySize=count($doNotUse);// LOOK this has changed
for ($i = 0; $i < $arySize; ++$i){
	//if (in_array($i, $doNotUse)) unset($aryCSV[$i]);// old way
	$key = array_search($doNotUse[$i], $aryCSV);// LOOK this is different
	if($key) unset($aryCSV[$key]);
	}
echo 'Comma Separated Values after Removal= ';
foreach ($aryCSV as $val) echo $val,',';
echo '<br />';

Open in new window

0

Featured Post

Edgartown IT Case Study

Learn about Edgartown's quest to ensure the safety and security of the entire town's employee and citizen data. Read the case study!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article discusses how to create an extensible mechanism for linked drop downs.
While rebooting windows server 2003 server , it's showing "active directory rebuilding indices please wait" at startup. It took a little while for this process to complete and once we logged on not all the services were started so another reboot is …
The viewer will learn how to dynamically set the form action using jQuery.
The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…

730 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question