Solved

not all  rows are loaded into table

Posted on 2011-09-19
8
185 Views
Last Modified: 2012-05-12
When I load my  data is only getting 3829 loaded out of 15913 rows of data
SELECT     COUNT(*) AS Expr1
FROM         load_into_wf 

Open in new window


using System;
using System.Data;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data.OleDb;
using System.Data.SqlClient;
using System.IO;





namespace import_from_excel_1
{
    class Program
    {
        static void Main(string[] args)
        {

            
SqlConnection con = new SqlConnection("Data Source=USVAVI; Initial Catalog=ANDREWS_DB_REPVIEW; User Id=sa; Password= azkaban09$");
            string filepath = "o:\\EE_Roster_ALL.csv";
            StreamReader sr = new StreamReader(filepath);
            string line = sr.ReadLine();
            string[] value = line.Split(',');
            DataTable dt = new DataTable();
            DataRow row;
            foreach (string dc in value)
            {
                dt.Columns.Add(new DataColumn(dc));
            }

            while ( !sr.EndOfStream )
            {
                value = sr.ReadLine().Split(',');
                if(value.Length == dt.Columns.Count)
                {
                    row = dt.NewRow();
                    row.ItemArray = value;
                    dt.Rows.Add(row);
                }
            }
            SqlBulkCopy bc = new SqlBulkCopy(con.ConnectionString, SqlBulkCopyOptions.TableLock);
            bc.DestinationTableName = "load_into_wf";
            bc.BatchSize = dt.Rows.Count;
            con.Open();
            bc.WriteToServer(dt);
            bc.Close();
            con.Close(); 
 










                }
            }
        }
    

 

Open in new window


EE-Roster-ALL.csv
0
Comment
Question by:jaypappas
  • 3
  • 3
8 Comments
 
LVL 44

Accepted Solution

by:
AndyAinscow earned 250 total points
ID: 36560640
Your line 37 :                 if(value.Length == dt.Columns.Count)
looks suspicious.  This would result in rows NOT being added (your bobserved behaviour) if the statement was not true.
0
 

Author Comment

by:jaypappas
ID: 36560765
Your line 37 :                 if(value.Length == dt.Columns.Count)
looks suspicious.  This would result in rows NOT being added (your bobserved behaviour) if the statement was not true.  


 it does not know what to do with lines with quotes in them.

example:

ANV,403605,353605,Batte,Alonzo Gene,A,H,P,9/8/2008,3020SF,San Francisco Office,NPUNI,Unarmed Security Officer,334159,"Mitchell,Richard D",3500
0
 
LVL 44

Expert Comment

by:AndyAinscow
ID: 36560856
Do you mean the quote marks are the cause of your problem ?


ps.  I hope that isn't real information detailing who an 'unarmed security officer' is (name + address....).
0
Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 

Author Comment

by:jaypappas
ID: 36561599
Yes the quotes are causing the problem.  This data is sample data not real data.
0
 

Author Comment

by:jaypappas
ID: 36574212
I've requested that this question be deleted for the following reason:

No timely responses
0
 
LVL 44

Expert Comment

by:AndyAinscow
ID: 36574213
Comment #36560640 points out the problem why not all rows are being added.
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

Extention Methods in C# 3.0 by Ivo Stoykov C# 3.0 offers extension methods. They allow extending existing classes without changing the class's source code or relying on inheritance. These are static methods invoked as instance method. This…
Introduction Although it is an old technology, serial ports are still being used by many hardware manufacturers. If you develop applications in C#, Microsoft .NET framework has SerialPort class to communicate with the serial ports.  I needed to…
Sending a Secure fax is easy with eFax Corporate (http://www.enterprise.efax.com). First, Just open a new email message.  In the To field, type your recipient's fax number @efaxsend.com. You can even send a secure international fax — just include t…
Excel styles will make formatting consistent and let you apply and change formatting faster. In this tutorial, you'll learn how to use Excel's built-in styles, how to modify styles, and how to create your own. You'll also learn how to use your custo…

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now