Solved

not all  rows are loaded into table

Posted on 2011-09-19
8
189 Views
Last Modified: 2012-05-12
When I load my  data is only getting 3829 loaded out of 15913 rows of data
SELECT     COUNT(*) AS Expr1
FROM         load_into_wf 

Open in new window


using System;
using System.Data;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data.OleDb;
using System.Data.SqlClient;
using System.IO;





namespace import_from_excel_1
{
    class Program
    {
        static void Main(string[] args)
        {

            
SqlConnection con = new SqlConnection("Data Source=USVAVI; Initial Catalog=ANDREWS_DB_REPVIEW; User Id=sa; Password= azkaban09$");
            string filepath = "o:\\EE_Roster_ALL.csv";
            StreamReader sr = new StreamReader(filepath);
            string line = sr.ReadLine();
            string[] value = line.Split(',');
            DataTable dt = new DataTable();
            DataRow row;
            foreach (string dc in value)
            {
                dt.Columns.Add(new DataColumn(dc));
            }

            while ( !sr.EndOfStream )
            {
                value = sr.ReadLine().Split(',');
                if(value.Length == dt.Columns.Count)
                {
                    row = dt.NewRow();
                    row.ItemArray = value;
                    dt.Rows.Add(row);
                }
            }
            SqlBulkCopy bc = new SqlBulkCopy(con.ConnectionString, SqlBulkCopyOptions.TableLock);
            bc.DestinationTableName = "load_into_wf";
            bc.BatchSize = dt.Rows.Count;
            con.Open();
            bc.WriteToServer(dt);
            bc.Close();
            con.Close(); 
 










                }
            }
        }
    

 

Open in new window


EE-Roster-ALL.csv
0
Comment
Question by:jaypappas
  • 3
  • 3
8 Comments
 
LVL 44

Accepted Solution

by:
AndyAinscow earned 250 total points
ID: 36560640
Your line 37 :                 if(value.Length == dt.Columns.Count)
looks suspicious.  This would result in rows NOT being added (your bobserved behaviour) if the statement was not true.
0
 

Author Comment

by:jaypappas
ID: 36560765
Your line 37 :                 if(value.Length == dt.Columns.Count)
looks suspicious.  This would result in rows NOT being added (your bobserved behaviour) if the statement was not true.  


 it does not know what to do with lines with quotes in them.

example:

ANV,403605,353605,Batte,Alonzo Gene,A,H,P,9/8/2008,3020SF,San Francisco Office,NPUNI,Unarmed Security Officer,334159,"Mitchell,Richard D",3500
0
 
LVL 44

Expert Comment

by:AndyAinscow
ID: 36560856
Do you mean the quote marks are the cause of your problem ?


ps.  I hope that isn't real information detailing who an 'unarmed security officer' is (name + address....).
0
Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

 

Author Comment

by:jaypappas
ID: 36561599
Yes the quotes are causing the problem.  This data is sample data not real data.
0
 

Author Comment

by:jaypappas
ID: 36574212
I've requested that this question be deleted for the following reason:

No timely responses
0
 
LVL 44

Expert Comment

by:AndyAinscow
ID: 36574213
Comment #36560640 points out the problem why not all rows are being added.
0

Featured Post

Announcing the Most Valuable Experts of 2016

MVEs are more concerned with the satisfaction of those they help than with the considerable points they can earn. They are the types of people you feel privileged to call colleagues. Join us in honoring this amazing group of Experts.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Extention Methods in C# 3.0 by Ivo Stoykov C# 3.0 offers extension methods. They allow extending existing classes without changing the class's source code or relying on inheritance. These are static methods invoked as instance method. This…
This article introduced a TextBox that supports transparent background.   Introduction TextBox is the most widely used control component in GUI design. Most GUI controls do not support transparent background and more or less do not have the…
In an interesting question (https://www.experts-exchange.com/questions/29008360/) here at Experts Exchange, a member asked how to split a single image into multiple images. The primary usage for this is to place many photographs on a flatbed scanner…

856 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question