Import CSV and UPDATE the data into mySQL

Hi, I have some data in Excel (CSV format) that looks like this:

Material........Component.......Quantity
123456.........aaa.....................1
123456.........bbb....................2
123456.........ccc.....................1
555555.........ddd....................1

My mySQL database rows look like this (I inherited this structure):

Material....Component1...Component2...Component3...Quantity1...Quantity2...Quantity3
123456..............................................................................................................................
555555..............................................................................................................................

What I'm trying to do is import the CSV and UPDATE the data into the relevant columns, so that it looks like:

Material....Component1...Component2...Component3...Quantity1...Quantity2...Quantity3
123456.....aaa....................bbb....................ccc.....................1...................2..................1
555555.....ddd........................................................................1........................................

Can someone please give some examples on how the PHP should look?

Thank you

kbit

Asked: 2014-01-21

Advertise Here

2 Solutions

LVL 111

Ray PaseurCommented: 2014-01-21

You should redesign the data base! What if you get a Material that has four Components? And what if there is only one Component? What would you put into the other columns?

But that aside, I think I can show you something that could translate the CSV into some kind of query. I'll see if I can come up with a code sample and I'll post back in a little while.

kbitAuthor Commented: 2014-01-21

Thanks very much Ray, that would be much appreciated.

Yes the structure is a nightmare...I've posted the stripped down version there...you dont want to see the real thing!

LVL 111

Ray PaseurCommented: 2014-01-21

This code snippet may transform the original test data into something that is usable for a data base update, but the data structure and process is so fraught with risk, that I'm almost reluctant to post it here. Issues include...

1. What if there are more or fewer Components per Material?
2. What if there is a duplicate Material in the input file?
3. What if Component 1 should be empty, with the Quantity applicable to Component 2?
4. What if the input CSV is not sorted by Material?
5. What if there is a Material without any Component?
6. What if there is a mismatch in the count of Components and Materials?

I could go on, but I'm sure you get the point. This would be a good time to get a DBA involved to help design a table structure that will not suck.

<?php // RAY_temp_kbit.php
error_reporting(E_ALL);
echo '<pre>';


// SEE http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/Q_28343790.html#a39796531


/*
Material.......Component.......Quantity
123456.........aaa..............1
123456.........bbb..............2
123456.........ccc..............1
555555.........ddd..............1

Material...Component1...Component2...Component3...Quantity1...Quantity2...Quantity3
123456.....aaa..........bbb..........ccc...........1...........2...........1
555555.....ddd..........NULL.........NULL..........1...........NULL........NULL
*/

Class MaterialRows
{
    public function __construct()
    {
        $this->rows = array();
    }
    public function inject(array $row)
    {
        $mat = $row[0];
        $com = $row[1];
        $qty = $row[2];
        $this->rows[$mat]['com'][] = $com;
        $this->rows[$mat]['qty'][] = $qty;
    }
    public function collapse()
    {
        ksort($this->rows);
        reset($this->rows);
        var_dump($this->rows);
        foreach ($this->rows as $key => $row)
        {
            // PROCESS THE COMPONENTS
            $com = implode(',', $row['com']);

            // IF FEWER THAN 3 ADD EMPTY POSITIONS
            $xxx = 3 - count($row['com']);
            if ($xxx) $com .= str_repeat(',', $xxx);

            // PROCESS THE QUANTITIES
            $qty = implode(',', $row['qty']);

            // IF FEWER THAN 3 ADD EMPTY POSITIONS
            $xxx = 3 - count($row['qty']);
            if ($xxx) $qty .= str_repeat(',', $xxx);

            $out[]
            = $key
            . ','
            . $com
            . ','
            . $qty
            ;
        }
        var_dump($out);
    }
}

// TEST DATA SIMULATING WHAT YOU GET WITH FGETCSV()
$rows = array
( array(123456, 'aaa', 1)
, array(123456, 'bbb', 2)
, array(123456, 'ccc', 1)
, array(555555, 'ddd', 1)
, array(555555, 'eee', 3)
)
;

$aggregate = new MaterialRows;
foreach ($rows as $row)
{
    $aggregate->inject($row);
}
$aggregate->collapse();

Open in new window

kbitAuthor Commented: 2014-01-21

Thanks a lot Ray, I think you're right, its a method that has a few ways of going wrong although some of your questions will not be issues in reality as the Excel file will be built by SAP.

However that piece of code will be very useful for a different import I'll have to do after this one.

For this one, I've put together a workaround as follows:

1) Import the file as-is into a temporary table (which is wiped with each run)
2) PHP reads this table and for each change in "Material", it does a loop, building a long UPDATE statement.
3) The UPDATE statement is then run which updates the target table

LVL 111

Ray PaseurCommented: 2014-01-21

I think I like your way better!

kbitAuthor Commented: 2014-01-27

The expert approved the method I used

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Expand your skills with this month's free Premium course.

Import CSV and UPDATE the data into mySQL

Join & Write a Comment Already a member? Login.

Featured Post