Link to home
Start Free TrialLog in
Avatar of metropia
metropiaFlag for United States of America

asked on

import excel data into sql server using SSIS

hello, i need to get some opinions and recommendations regarding how to import data that is maintained in an excel file, and uploaded to a share-point site that also needs to be imported into a sql server table. the excel file has multiple tabs, but i need to read the data from only one of the tabs.

i'd like to use a ssis package but i would like to get expert advice, on how to do this, and if there are better, simpler ways to do this.

my specifications are:

buyer keeps market price information on excel file, every week he needs to use that data to update market prices in a sql server table. the excel file is saved on a network location (share-point)

\\companynetwork\sites\buyersite\buyer\byrdocs\level1\General\excel.xlsm

i am using sql server 2012, excel 2013

any ideas are welcome. thank you very much.
SOLUTION
Avatar of plummet
plummet
Flag of United Kingdom of Great Britain and Northern Ireland image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Avatar of Chris M
Chris M
Flag of Uganda image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
ASKER CERTIFIED SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of metropia

ASKER

First of all, thank you very much for each one of your comments. They are highly appreciated.

Will it be possible to read three columns from the excel file and using those 3 columns retrieve one more column from sql server?

Example, excel file contains:

item number
date
description
market price

From excel file I need item number, date (these two would make the unique key, or identifier for each row, ideal), and market price. The description comes from the item table, because the excel file could potentially have misspelling errors in the description, or invalid lengths.

Is it possible to do this via SSIS?

Thank you.
Yes, you can load the excel file into a temporary table which you can then use in a SQL query to pull the description. All of this within the SSIS package.

Cheers
John
Is there any tutorial that I can read where a task similar to what I need is done?
I am not familiar with using temporary tables. Will I need to use a stored procedure?
What other modules I need to add to the data flow? So much it can be done with this tool but not knowing how to correctly use its power, it is frustrating even the most simple task.
Thank you.
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
John, I was able to load the values to the new table MyTemp

Question, why to create a new table when I already have a destination table created?
My challenge resides in getting the Item Description from the Item table, using the Item Number from the excel file. I was reading that an option I might consider is using Lookup?

Is it possible you show me how to add the description column from another table, using a value from the excel file, on my data flow?

Thank you.
Hi

Sorry I did n't realise you had a target table already. In that case specify your table instead of the temp table and make sure the column mappings in the destination object are correct.

Then we can add an execute SQL task to run the SQL to update the Description column. If you give that a try we're really getting somewhere.

Regards
John
Hi John, I replace the temp table and I'm using the table I already had created. It worked well.
I am trying to figure this out, I read that using a look-up module can help me get the description column from the sql table. That part is not working.

If you could teach me how to use the execute SQL task to do the update, that would be great.

User generated image
Now it is working. I had to change the default error configuration of the Lookup task "Fail Component"; it would fail if a no match occurs. Changed to 'Redirect row'.

I have a question John, the combination of item number and Date, make each row unique.
Is it possible to check if a row with ItemNumber and Date combination already exists in the database table? If so, how? Thank you very much.
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Hi John, yes I did changed the error column and got around the issue. What was happening is that since I am using a development database, some of the values in the excel file (which uses live data) did not have a match in the dev db.

Would you be willing to teach me how to check for duplicates? my task flow looks exactly as the screen shot I uploaded a couple comments above.
Hi,

I'll have to do this later, in a few hours. Sorry about that!
hopefully there is another expert available. thanks.
i do need help with this still. if you are available please let me know. thank you.
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
i created a staging table. i do not know how to go from the staging table to the final table. please help i am so frustrated...thank you


User generated image
I've requested that this question be deleted for the following reason:

poor help. really not a helpful site anymore.
Looks like metropia doesn't see the difference between asking a question and getting a consultant to implement a project. Therefore I object, there's at least one expert around that deserves some recognition for his effort...
Please ValentinoV if I missed that post, just point me to the comment id and I read it and determine if it actually contributed to helping me figuring out the solution to my problem. Thank you for your feedback.
that got your attention. you should have instead make a post to help me out figure out last step ;)
that got your attention. you should have instead make a post to help me out figure out last step ;)
That is perfectly infantile, let alone unprofessional.  I am not sure if you realize we are all volunteers here.  We all have our jobs and our lives and they do not revolve around you.  We are here to help, but we are not at you beck and call.  The part you are missing is that you will eventually may want to post another question in this site, nearly 25% of all your questions have been deleted, faced with that record there is no incentive to help out.  I do understand that this is against EE Guidelines, but I would not be surprised if you opened a new account.

I wish you the best of luck.
acperkins, i am not with intention to start a debate here. thank you for sharing your opinions about me, you have helped me in the past and i appreciate that. i do not think of me as being special or selfish in regards my expectations of how fast or soon a question should received attention, really. i opened question a couple of days ago, and the feedback has been poor, that is true. i also have tried to learn how to figure it out on my own, but sadly with some frustration and limitations. if i have deleted questions in the past, i am not aware of what is the percentage of that, and also i do delete questions that i figure on my own, or questions that never get a response. i also, before deleting a question, express the reason why i am deleting it. as i did for this one. if experts on ee will hold a grudge for this action, then by all means, that is your right. thank you and good day.
if experts on ee will hold a grudge for this action, then by all means, that is your right.
You are clearly missing the point and as the French say le raison d'être of this site.  Holding a grudge would be infantile, don't you think?  Which is exactly the opposite of the principles found on this site.  We are here to learn and share.  What would be the point of sharing with you, if there is no chance we can learn?

I don't know how else to explain it to you.  Perhaps with time you can learn to appreciate by giving as much as you have received and you may understand better.
Hi Metropia

I am sorry you consider all the time I spent trying to help you as "poor". I am an IT contractor in the UK and I work full time so I can't spend a lot of time on EE. I believe I more than answered your original, quite non-specific, question fully and as you asked further questions I went into detail providing you with step by step instructions. I think the purpose of this website is to help where someone needs assistance but not to do somebody's job for them.

I was happy to try to help but as at 22.30 last night I wasn't able to spend any more time. It did seem that you were getting there with your SSIS package and I hope you'll get it all working.

Please bear in mind we all do this out of a wish to help out other people, and also that the assignment of points here doesn't reflect the amount of time and effort that I put into your replies.

Regards
John
Thank you John. I kindly appreciate the help. I was able to complete the SSIS Package yesterday. Next time I'll try to be more assertive when wording my thoughts. Once again, thank you.