Solved

Need to split one huge CSV file into 5 files

Posted on 2013-06-25
8
1,104 Views
Last Modified: 2016-02-11
Please help with the SSIS package, thanks
0
Comment
Question by:sqlcurious
8 Comments
 
LVL 4

Expert Comment

by:thelug
ID: 39275958
CSV Splitter.  Someone I knew in a previous "life" used it all the time.  Free utility.  Get it here:

http://erdconcepts.com/dbtoolbox.html
0
 

Author Comment

by:sqlcurious
ID: 39276169
Thelug, Thanks for the answer but I needto do it via SSIS, please suggest.
I know we could do using multicast or conditional split but not sure about the exact process
0
 
LVL 45

Expert Comment

by:aikimark
ID: 39322298
1. Are you splitting an input file or an output file?
2. Are you splitting based on row/records or by columns?
3. What kind of logic are you using to do the split?
4. When you say "huge", just how big is the file?

============
have you looked at Powershell?
http://www.codeproject.com/Articles/70319/Splitting-CSV-file-based-on-content-in-one-line-us
http://gallery.technet.microsoft.com/scriptcenter/PowerShell-Split-large-log-6f2c4da0
http://michielw.blogspot.com/2012/05/powershell-split-text-file-in-multiple.html

============
free CSV splitter to use while you are working on a complete solution
http://www.mydigitallife.info/download-free-csv-splitter-to-split-csv-files-easily/
0
DevOps Toolchain Recommendations

Read this Gartner Research Note and discover how your IT organization can automate and optimize DevOps processes using a toolchain architecture.

 
LVL 12

Accepted Solution

by:
Saurabh Bhadauria earned 500 total points
ID: 39322764
yes you can do it SSIS..
you have to use single  Flat file source and multiple  flat file destinations

if you want to split data based on row values then use Conditional Split ..
check  below link....
http://henkvandervalk.com/speeding-up-ssis-bulk-inserts-into-sql-server


If you want to Split data base on colums... then use multicast..
check below link..
http://stackoverflow.com/questions/6213847/using-multi-cast-transformation-for-destinations-with-different-columns 

Thanks,
Saurabh
0
 

Author Comment

by:sqlcurious
ID: 39334914
Thanks a lot Saurabh I will give it a try
0
 

Author Closing Comment

by:sqlcurious
ID: 39625802
thanks
0

Featured Post

Microsoft Certification Exam 74-409

Veeam® is happy to provide the Microsoft community with a study guide prepared by MVP and MCT, Orin Thomas. This guide will take you through each of the exam objectives, helping you to prepare for and pass the examination.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

JSON is being used more and more, besides XML, and you surely wanted to parse the data out into SQL instead of doing it in some Javascript. The below function in SQL Server can do the job for you, returning a quick table with the parsed data.
Ever needed a SQL 2008 Database replicated/mirrored/log shipped on another server but you can't take the downtime inflicted by initial snapshot or disconnect while T-logs are restored or mirror applied? You can use SQL Server Initialize from Backup…
This video shows, step by step, how to configure Oracle Heterogeneous Services via the Generic Gateway Agent in order to make a connection from an Oracle session and access a remote SQL Server database table.
Via a live example, show how to backup a database, simulate a failure backup the tail of the database transaction log and perform the restore.

863 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now