Solved

Need to split one huge CSV file into 5 files

Posted on 2013-06-25
8
1,279 Views
Last Modified: 2016-02-11
Please help with the SSIS package, thanks
0
Comment
Question by:sqlcurious
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
8 Comments
 
LVL 4

Expert Comment

by:thelug
ID: 39275958
CSV Splitter.  Someone I knew in a previous "life" used it all the time.  Free utility.  Get it here:

http://erdconcepts.com/dbtoolbox.html
0
 

Author Comment

by:sqlcurious
ID: 39276169
Thelug, Thanks for the answer but I needto do it via SSIS, please suggest.
I know we could do using multicast or conditional split but not sure about the exact process
0
 
LVL 45

Expert Comment

by:aikimark
ID: 39322298
1. Are you splitting an input file or an output file?
2. Are you splitting based on row/records or by columns?
3. What kind of logic are you using to do the split?
4. When you say "huge", just how big is the file?

============
have you looked at Powershell?
http://www.codeproject.com/Articles/70319/Splitting-CSV-file-based-on-content-in-one-line-us
http://gallery.technet.microsoft.com/scriptcenter/PowerShell-Split-large-log-6f2c4da0
http://michielw.blogspot.com/2012/05/powershell-split-text-file-in-multiple.html

============
free CSV splitter to use while you are working on a complete solution
http://www.mydigitallife.info/download-free-csv-splitter-to-split-csv-files-easily/
0
Revamp Your Training Process

Drastically shorten your training time with WalkMe's advanced online training solution that Guides your trainees to action.

 
LVL 12

Accepted Solution

by:
Saurabh Bhadauria earned 500 total points
ID: 39322764
yes you can do it SSIS..
you have to use single  Flat file source and multiple  flat file destinations

if you want to split data based on row values then use Conditional Split ..
check  below link....
http://henkvandervalk.com/speeding-up-ssis-bulk-inserts-into-sql-server


If you want to Split data base on colums... then use multicast..
check below link..
http://stackoverflow.com/questions/6213847/using-multi-cast-transformation-for-destinations-with-different-columns 

Thanks,
Saurabh
0
 

Author Comment

by:sqlcurious
ID: 39334914
Thanks a lot Saurabh I will give it a try
0
 

Author Closing Comment

by:sqlcurious
ID: 39625802
thanks
0

Featured Post

NFR key for Veeam Backup for Microsoft Office 365

Veeam is happy to provide a free NFR license (for 1 year, up to 10 users). This license allows for the non‑production use of Veeam Backup for Microsoft Office 365 in your home lab without any feature limitations.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Moving away from Access 2003 adp files 4 48
What does "Between" mean? 6 48
TDE for SQL Web Edition 1 42
T-SQL Query 9 37
Ever wondered why sometimes your SQL Server is slow or unresponsive with connections spiking up but by the time you go in, all is well? The following article will show you how to install and configure a SQL job that will send you email alerts includ…
I have a large data set and a SSIS package. How can I load this file in multi threading?
Using examples as well as descriptions, and references to Books Online, show the different Recovery Models available in SQL Server and explain, as well as show how full, differential and transaction log backups are performed
Viewers will learn how to use the UPDATE and DELETE statements to change or remove existing data from their tables. Make a table: Update a specific column given a specific row using the UPDATE statement: Remove a set of values using the DELETE s…

752 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question