?
Solved

function to clean csv data

Posted on 2009-03-31
10
Medium Priority
?
712 Views
Last Modified: 2012-05-06
I have a csv file comma delimited. Some of the data is double quote encaps where address field has comma caused by user input from web form.

Ie. 12345,"235 main street, apt3",boston,ma,12431

When i try to use putcsv function and import into sql the extra comma that is encaps is causing apt3 to write into the city column and city into the state column, etc.

How can i clean this up before importing?

I tried this but dont work.
<?
 
$file_path = "/my/path";
$file = $file_path . "filename.csv";
 
$handle = fopen("$file", "r"); 
if ($handle)
{
$array = explode("\n", fread($handle, filesize("$file")));
}
 
$total_array = count($array);
$i = 0;
 
$handle2 = fopen('filename2.csv', 'w');
 
while($i < $total_array)
{
	
$data = explode(",", $array[$i]);
 
fputcsv($handle2, $data, $sSeparator='|', $sEnclosure='"');
 
$i++;
}
 
fclose($handle); 
 
?>

Open in new window

0
Comment
Question by:pdheady
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
  • 3
10 Comments
 
LVL 3

Accepted Solution

by:
garlix earned 2000 total points
ID: 24032310
The code below will find all , within " and " and replace them with a ..

Hope that will be usefull!

$test = '12345,"235 main street, apt3",boston,ma,12431';
$csvOk = preg_replace_callback('/"(.*)"/', create_function('$matches',
            'return \'"\' . preg_replace(\'/,/\', \'.\', $matches[1]) . \'"\';'), $test);

Open in new window

0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 24032378
@pdheady:

What if you decided to do without commas in the address?  The USPS does not use, or need, them.
0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 24032471
... and for that matter, the Geocoders from Google and Yahoo do not need commas inside the address components, either.  Just a thought. ~Ray
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 1

Author Comment

by:pdheady
ID: 24032510
I have no control over someone entering in a comma in the address field at the moment. Another company handles the front end web form.

I will strip it out when they post to our system but for now need to clean up existing data. I need something that reads the csv and locates the double quote field and pulls the comma out, then exports to new csv file.
0
 
LVL 1

Author Comment

by:pdheady
ID: 24032516
I was able to do a search and replace using excel feature but would prefer using php to do the work.
0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 24032545
Understood - havent' tested it , but garlix answer can probably be useful.  Good luck with it, ~Ray
0
 
LVL 1

Author Comment

by:pdheady
ID: 24032621
Thanks, I guess based on that my question is how to implement garlix suggestion to strip from ARRAY $data
0
 
LVL 3

Assisted Solution

by:garlix
garlix earned 2000 total points
ID: 24032710
If you got an array where each post is a row of csv data. Then the following ought to do the trick :-)

Good luck!

function alterRow(&$item, $key) {
    $item = preg_replace_callback('/"(.*)"/', create_function('$matches',
                'return \'"\' . preg_replace(\'/,/\', \'.\', $matches[1]) . \'"\';'), $item);
}
 
array_walk($rows, 'alterRow');

Open in new window

0
 
LVL 3

Expert Comment

by:garlix
ID: 24037060
Did you get it to work? :)
0

Featured Post

VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I imagine that there are some, like me, who require a way of getting currency exchange rates for implementation in web project from time to time, so I thought I would share a solution that I have developed for this purpose. It turns out that Yaho…
Many old projects have bad code, but the budget doesn't exist to rewrite the codebase. You can update this code to be safer by introducing contemporary input validation, sanitation, and safer database queries.
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
The viewer will learn how to count occurrences of each item in an array.
Suggested Courses

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question