Solved

Null Values in Datatable - VB.Net

Posted on 2016-08-07
3
94 Views
Last Modified: 2016-08-09
Hello everyone,
So I am getting a weird anomaly while using a datatable in vb.net. Every so often I get nulls in my datatable where they do not exist in SQL. I know "how" to compensate for them, but that isn't my issue, they should not be there, and it just makes the integrity of results bad..

I am using the following code to populate the table
            Try
                Using Processing_LoadData As New SqlDataAdapter("SELECT [TrackingID],[SAMAccountName],[DisplayName],[givenName],[middleName],[sn],[employeeNumber],[employeeID],[EmployeeType],[Personal_Phone],[Personal_EMail],[roomNumber],[accountExpires],[Creator],[Location_Friendly],[LocationID],[Building_DN],[Address1],[Address2],[City],[State],[Zip],[Description],[Office],[Department],[Company],[ProfilePath],[HomePath],[HomeDrive],[Record_Source],[INSERT_TIME],[MailTO],[Auto_Expiration_Enable],[Auto_Expiration_Int],[Auto_Expiration_Peroid],[IP],[Justification],[141],[ADDN],[Mail],[Live_Object_Source] FROM [AMSv1].[dbo].[AMS_Processing_INPROGRESS_ADWrite];", Processing_SQLConnection)
                    Try
                        Processing_DataResults = New DataTable
                        Processing_LoadData.Fill(Processing_DataResults)
                    Catch ex As Exception
                        WinEventLog.WriteEntry("Account Processing: Failed to populate datatable. " & ex.ToString, EventLogEntryType.Warning, 1503)
                        GoTo Processing_Exit
                    End Try
                End Using
            Catch ex As Exception
                WinEventLog.WriteEntry("Account Processing: Failed to bind to SQL", EventLogEntryType.Warning, 1502)
                GoTo Processing_Exit
            End Try

Open in new window


Reason I am using a database instead of direct SQL reads is because I am spinning up multiple threads to process the changes in parallel. and using a datatable you can easily divide up the load.

executing the same statement in sql, I have no nulls.

Has anyone else had this problem / seen this?
I am not going to lie, it is a LOT of data, 40 columns and between 9,000 to 150,000 rows.
0
Comment
Question by:JesNoFear
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
3 Comments
 
LVL 9

Accepted Solution

by:
Trideep Patel earned 500 total points
ID: 41746673
Have you put a break point and checked that Processing_DataResults contains null values after Processing_LoadData.Fill(Processing_DataResults) line?
I am just asking it because it might be possible due to parallel for loop or threading.
1
 
LVL 2

Author Comment

by:JesNoFear
ID: 41748008
So, directly after this is when the loop's start. it does the load and then spins up the multi-threads.

So, what i have done since this post was in each thread, it is given a range of rows to process, as it goes through the for next loop, i have it checking all the fields for nulls, if it finds a null that is unexpected, it will drop that job and use a continue for and denote in sql to re-process that entry after the cycle is done.

and with that being said, since that change, neither the safeguard check before it does the work or while the work is being done have null's come up. i added a break point if it were to find a null, as well as logging it to windows even log.

so it is still a mystery, i have been flooding it with sets of 9,000 and 15,000 transactions all day. I am about to do another load test with 150,000 transactions and see if it comes back or not, and/or that the safeguard is working.
0
 
LVL 2

Author Closing Comment

by:JesNoFear
ID: 41749507
the issue has disappeared since i added the pre-checking.
0

Featured Post

NFR key for Veeam Agent for Linux

Veeam is happy to provide a free NFR license for one year.  It allows for the non‑production use and valid for five workstations and two servers. Veeam Agent for Linux is a simple backup tool for your Linux installations, both on‑premises and in the public cloud.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Wouldn’t it be nice if you could test whether an element is contained in an array by using a Contains method just like the one available on List objects? Wouldn’t it be good if you could write code like this? (CODE) In .NET 3.5, this is possible…
Calculating holidays and working days is a function that is often needed yet it is not one found within the Framework. This article presents one approach to building a working-day calculator for use in .NET.
There are cases when e.g. an IT administrator wants to have full access and view into selected mailboxes on Exchange server, directly from his own email account in Outlook or Outlook Web Access. This proves useful when for example administrator want…
Monitoring a network: why having a policy is the best policy? Michael Kulchisky, MCSE, MCSA, MCP, VTSP, VSP, CCSP outlines the enormous benefits of having a policy-based approach when monitoring medium and large networks. Software utilized in this v…

636 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question