Solved

Import CSV and UPDATE the data into mySQL

Posted on 2014-01-21
6
499 Views
Last Modified: 2014-01-27
Hi, I have some data in Excel (CSV format) that looks like this:

Material........Component.......Quantity
123456.........aaa.....................1
123456.........bbb....................2
123456.........ccc.....................1
555555.........ddd....................1

My mySQL database rows look like this (I inherited this structure):

Material....Component1...Component2...Component3...Quantity1...Quantity2...Quantity3
123456..............................................................................................................................
555555..............................................................................................................................

What I'm trying to do is import the CSV and UPDATE the data into the relevant columns, so that it looks like:

Material....Component1...Component2...Component3...Quantity1...Quantity2...Quantity3
123456.....aaa....................bbb....................ccc.....................1...................2..................1
555555.....ddd........................................................................1........................................

Can someone please give some examples on how the PHP should look?

Thank you
0
Comment
Question by:kbit
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
6 Comments
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 39796531
You should redesign the data base!  What if you get a Material that has four Components?  And what if there is only one Component?  What would you put into the other columns?

But that aside, I think I can show you something that could translate the CSV into some kind of query.  I'll see if I can come up with a code sample and I'll post back in a little while.
0
 

Author Comment

by:kbit
ID: 39796541
Thanks very much Ray, that would be much appreciated.

Yes the structure is a nightmare...I've posted the stripped down version there...you dont want to see the real thing!
0
 
LVL 110

Accepted Solution

by:
Ray Paseur earned 500 total points
ID: 39796716
This code snippet may transform the original test data into something that is usable for a data base update, but the data structure and process is so fraught with risk, that I'm almost reluctant to post it here.  Issues include...

1. What if there are more or fewer Components per Material?
2. What if there is a duplicate Material in the input file?
3. What if Component 1 should be empty, with the Quantity applicable to Component 2?
4. What if the input CSV is not sorted by Material?
5. What if there is a Material without any Component?
6. What if there is a mismatch in the count of Components and Materials?

I could go on, but I'm sure you get the point.  This would be a good time to get a DBA involved to help design a table structure that will not suck.

<?php // RAY_temp_kbit.php
error_reporting(E_ALL);
echo '<pre>';


// SEE http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/Q_28343790.html#a39796531


/*
Material.......Component.......Quantity
123456.........aaa..............1
123456.........bbb..............2
123456.........ccc..............1
555555.........ddd..............1

Material...Component1...Component2...Component3...Quantity1...Quantity2...Quantity3
123456.....aaa..........bbb..........ccc...........1...........2...........1
555555.....ddd..........NULL.........NULL..........1...........NULL........NULL
*/

Class MaterialRows
{
    public function __construct()
    {
        $this->rows = array();
    }
    public function inject(array $row)
    {
        $mat = $row[0];
        $com = $row[1];
        $qty = $row[2];
        $this->rows[$mat]['com'][] = $com;
        $this->rows[$mat]['qty'][] = $qty;
    }
    public function collapse()
    {
        ksort($this->rows);
        reset($this->rows);
        var_dump($this->rows);
        foreach ($this->rows as $key => $row)
        {
            // PROCESS THE COMPONENTS
            $com = implode(',', $row['com']);

            // IF FEWER THAN 3 ADD EMPTY POSITIONS
            $xxx = 3 - count($row['com']);
            if ($xxx) $com .= str_repeat(',', $xxx);

            // PROCESS THE QUANTITIES
            $qty = implode(',', $row['qty']);

            // IF FEWER THAN 3 ADD EMPTY POSITIONS
            $xxx = 3 - count($row['qty']);
            if ($xxx) $qty .= str_repeat(',', $xxx);

            $out[]
            = $key
            . ','
            . $com
            . ','
            . $qty
            ;
        }
        var_dump($out);
    }
}

// TEST DATA SIMULATING WHAT YOU GET WITH FGETCSV()
$rows = array
( array(123456, 'aaa', 1)
, array(123456, 'bbb', 2)
, array(123456, 'ccc', 1)
, array(555555, 'ddd', 1)
, array(555555, 'eee', 3)
)
;

$aggregate = new MaterialRows;
foreach ($rows as $row)
{
    $aggregate->inject($row);
}
$aggregate->collapse();

Open in new window

0
Revamp Your Training Process

Drastically shorten your training time with WalkMe's advanced online training solution that Guides your trainees to action.

 

Assisted Solution

by:kbit
kbit earned 0 total points
ID: 39797524
Thanks a lot Ray, I think you're right, its a method that has a few ways of going wrong although some of your questions will not be issues in reality as the Excel file will be built by SAP.

However that piece of code will be very useful for a different import I'll have to do after this one.

For this one, I've put together a workaround as follows:

1) Import the file as-is into a temporary table (which is wiped with each run)
2) PHP reads this table and for each change in "Material", it does a loop, building a long UPDATE statement.
3) The UPDATE statement is then run which updates the target table
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 39797560
I think I like your way better!
0
 

Author Closing Comment

by:kbit
ID: 39811484
The expert approved the method I used
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Popularity Can Be Measured Sometimes we deal with questions of popularity, and we need a way to collect opinions from our clients.  This article shows a simple teaching example of how we might elect a favorite color by letting our clients vote for …
This article discusses how to implement server side field validation and display customized error messages to the client.
The viewer will learn how to look for a specific file type in a local or remote server directory using PHP.
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.

724 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question