Celebrate National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

C# Collection Class - Generic List

Posted on 2013-07-01
5
Medium Priority
?
610 Views
Last Modified: 2013-07-02
I am working on importing a csv file to sql database. I have attached the sample of csv file which tells, how data inside the csv file is looks like.

This csv file has multiple set of Tables. At present,i am pushing each set of table into a datatable by keeping all the column values of string datatype.


Instead of datatable, i prefer to use collection class. Original version of the code is written in VBA Macro code in which they used Structure - Type End Type Syntax inside the Macro code. I am sure Array list will not work. How to define a LinkedList with column and datatype. Does it improve the performance ??





public static DataTable GetInvoiceHeader(string strFileName)
        {
            DataTable dtInvoiceHeader = new DataTable();
            dtInvoiceHeader.Columns.Add("Invoice_Date", typeof(string));
            dtInvoiceHeader.Columns.Add("Invoice_Number", typeof(string));
            dtInvoiceHeader.Columns.Add("Account_Number", typeof(string));
            dtInvoiceHeader.Columns.Add("DEA_Number", typeof(string));
            dtInvoiceHeader.Columns.Add("Sales_Rep", typeof(string));
            try
            {
                using (StreamReader sr = new StreamReader(strFileName))
                {
                    string line;
                    line = sr.ReadLine();
                    while ((line = sr.ReadLine()) != "Invoice Date,Invoice Number,Account Number,DEA Number,Sales Rep,,,,,,,,,,")
                    {                       
                    }
                    // Split on these
                    line = sr.ReadLine() + 1;

                    char[] commadelimiter = new char[] { ',' };   
                    
                    string[] strArray = line.Split(commadelimiter, StringSplitOptions.None);

                    for (int i = 0; i < (strArray.Length + 1) - strArray.Length; i++)
                    {
                        DataRow dr = dtInvoiceHeader.NewRow();

                        dr["Invoice_Date"] = strArray[0];
                        dr["Invoice_Number"] = strArray[1];
                        dr["Account_Number"] = strArray[2];
                        dr["DEA_Number"] = strArray[3];
                        dr["Sales_Rep"] = strArray[4];
                        dtInvoiceHeader.Rows.Add(dr);
                    }                
                    
                }
                    
            }
            catch (Exception ex)
            {
                MessageBox.Show(ex.Message);
            }
            return dtInvoiceHeader;
        }

Open in new window

Test-6619289--5-30-2013.csv
0
Comment
Question by:chokka
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
5 Comments
 
LVL 11

Accepted Solution

by:
lenordiste earned 2000 total points
ID: 39291259
why don't you simply use a generic list ? create an object to store your columns (properties), let's say Invoice, and then you can use something like:

//instantiate list
var invoices=new List<Invoice>();
//add to list
invoices.add(new Invoice{Invoice_Date=strArray[0],Invoice_Number=strArray[1]});

Open in new window

0
 

Author Comment

by:chokka
ID: 39291272
i dont know the syntax. but thanks your syntax would help me.

and also does it improve the performance ?? in the end, once i created multiple generic list, either i should export to one single dataset or a sql table. so its possible to do, am i right ??
0
 
LVL 11

Expert Comment

by:lenordiste
ID: 39291346
it's hard to compare the performance of generic lists against datatables without a clear context... but for what you seem to be doing I believe it should perform better though the difference might not be noticeable.

However I don't know your current code architecture but if you are using LinqToSQL (outdated...) or EntityFramework to interact with the database than generic lists will work very well. If however you are not using that framework you might be better off with dataTables to minimize code rewrite.

what performance issues are you having exactly? One thing you could improve (whether using DataTables or Generic lists), is to define an initial capacity for your datastructure. For datatables:
http://msdn.microsoft.com/en-us/library/system.data.datatable.minimumcapacity.aspx

by the way, the for loop looks wrong:
(strArray.Length + 1) - strArray.Length
this is the same thing as
strArray.Length - strArray.Length +1 = 1

I am sure that looping until you reach 1 is not the intended purpose :-)
0
 

Author Comment

by:chokka
ID: 39291379
1) Thank you for for loop syntax correction

2) I am not using Linq to SQL. This is simple one page code which pushes 10 to 20 CSV files to one sql table. Every CSV File will multiple table set of records. You can see the sample csv file which i attached in the original posting.

3) i personally felt getting the data to datatable might affect performance as compared to generic list.

I have attached the whole page ( one page ) of my code. Still yet to write the SQL Connection Syntax part.



using System;
using System.Collections;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.IO;

namespace CSVImportSQL_CSharp
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }
       
        private void Form1_Load(object sender, EventArgs e)
        {
            ImportCSV();
        }
       
        public void ImportCSV()
        {
            try
            {

                string strFile = string.Empty;
                strFile = @"C:\FinalCSVReader_src\Test-6619289--5-30-2013.csv";
                
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
            }
        
        }

        public static DataTable GetInvoiceHeader(string strFileName)
        {
            DataTable dtInvoiceHeader = new DataTable();
            dtInvoiceHeader.Columns.Add("Invoice_Date", typeof(string));
            dtInvoiceHeader.Columns.Add("Invoice_Number", typeof(string));
            dtInvoiceHeader.Columns.Add("Account_Number", typeof(string));
            dtInvoiceHeader.Columns.Add("DEA_Number", typeof(string));
            dtInvoiceHeader.Columns.Add("Sales_Rep", typeof(string));
            try
            {
                using (StreamReader sr = new StreamReader(strFileName))
                {
                    string line;
                    line = sr.ReadLine();
                    while ((line = sr.ReadLine()) != "Invoice Date,Invoice Number,Account Number,DEA Number,Sales Rep,,,,,,,,,,")
                    {                       
                    }
                    // Split on these
                    line = sr.ReadLine() + 1;

                    char[] commadelimiter = new char[] { ',' };   
                    
                    string[] strArray = line.Split(commadelimiter, StringSplitOptions.None);

                    for (int i = 0; i < (strArray.Length + 1) - strArray.Length; i++)
                    {
                        DataRow dr = dtInvoiceHeader.NewRow();

                        dr["Invoice_Date"] = strArray[0];
                        dr["Invoice_Number"] = strArray[1];
                        dr["Account_Number"] = strArray[2];
                        dr["DEA_Number"] = strArray[3];
                        dr["Sales_Rep"] = strArray[4];
                        dtInvoiceHeader.Rows.Add(dr);
                    }                
                    
                }
                    
            }
            catch (Exception ex)
            {
                MessageBox.Show(ex.Message);
            }
            return dtInvoiceHeader;
        }


        public static DataTable GetInvoiceDetail(string strFileName)
        {
            DataTable dtInvoiceDetail = new DataTable();
            dtInvoiceDetail.Columns.Add("Ship_Qty", typeof(string));
            dtInvoiceDetail.Columns.Add("Order_Qty", typeof(string));
            dtInvoiceDetail.Columns.Add("Code", typeof(string));
            dtInvoiceDetail.Columns.Add("UM", typeof(string));
            dtInvoiceDetail.Columns.Add("Blank1", typeof(string));
            dtInvoiceDetail.Columns.Add("Description", typeof(string));
            dtInvoiceDetail.Columns.Add("Blank2", typeof(string));
            dtInvoiceDetail.Columns.Add("Blank3", typeof(string));
            dtInvoiceDetail.Columns.Add("Blank4", typeof(string));
            dtInvoiceDetail.Columns.Add("CIN", typeof(string));
            dtInvoiceDetail.Columns.Add("Due_Date", typeof(string));
            dtInvoiceDetail.Columns.Add("DEA_Class", typeof(string));
            dtInvoiceDetail.Columns.Add("Unit_Price", typeof(string));
            dtInvoiceDetail.Columns.Add("Extension", typeof(string));
            try
            {

                using (StreamReader sr = new StreamReader(strFileName))
                {

                    string line;
                    line = sr.ReadLine();
                 

                    while ((line = sr.ReadLine()) != "Ship Qty,Order Qty,Code,UM,,Description,,,,CIN,PO Number,Due Date,DEA Class,Unit Price,Extension")
                    {
                    }

                    while (!sr.EndOfStream)
                    {
                        // Split on these
                        line = sr.ReadLine() + 1;

                        char[] commadelimiter = new char[] { ',' };

                        string[] strArray = line.Split(commadelimiter, StringSplitOptions.None);
                        if (strArray[0].TrimEnd() == "Report Total:")
                            break;

                        for (int i = 0; i < (strArray.Length + 1) - strArray.Length; i++)
                        {
                            DataRow dr = dtInvoiceDetail.NewRow();

                            dr["Ship_Qty"] = strArray[0];
                            dr["Order_Qty"] = strArray[1];
                            dr["Code"] = strArray[2];
                            dr["UM"] = strArray[3];
                            dr["Blank1"] = strArray[4];
                            dr["Description"] = strArray[5];
                            dr["Blank2"] = strArray[6];
                            dr["Blank3"] = strArray[7];
                            dr["Blank4"] = strArray[8];
                            dr["CIN"] = strArray[9];
                            dr["Due_Date"] = strArray[10];
                            dr["DEA_Class"] = strArray[11];
                            dr["Unit_Price"] = strArray[12];
                            dr["Extension"] = strArray[13];
                            dtInvoiceDetail.Rows.Add(dr);
                        }
                        
                    }
                    

                }
            }

            catch (Exception ex)
            {
                MessageBox.Show(ex.Message);
            }
            return dtInvoiceDetail;
        }

    }
}

Open in new window

0
 
LVL 11

Expert Comment

by:lenordiste
ID: 39291403
then go the generic list route, having a strongly typed data structure will help you debugging too :-) !!! Also, for performance considerations avoid casting everything as a string (expensive type). Do not hesitate to use int, double, float, whenever possible. This will definitely improve performance.
0

Featured Post

Windows Server 2016: All you need to know

Learn about Hyper-V features that increase functionality and usability of Microsoft Windows Server 2016. Also, throughout this eBook, you’ll find some basic PowerShell examples that will help you leverage the scripts in your environments!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Performance in games development is paramount: every microsecond counts to be able to do everything in less than 33ms (aiming at 16ms). C# foreach statement is one of the worst performance killers, and here I explain why.
This article aims to explain the working of CircularLogArchiver. This tool was designed to solve the buildup of log file in cases where systems do not support circular logging or where circular logging is not enabled
In this video, Percona Solutions Engineer Barrett Chambers discusses some of the basic syntax differences between MySQL and MongoDB. To learn more check out our webinar on MongoDB administration for MySQL DBA: https://www.percona.com/resources/we…
How to fix incompatible JVM issue while installing Eclipse While installing Eclipse in windows, got one error like above and unable to proceed with the installation. This video describes how to successfully install Eclipse. How to solve incompa…

730 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question