Solved

VB.Net - Oracle BulkCopy from CSV Date Format

Posted on 2016-11-30
7
45 Views
Last Modified: 2016-12-01
I am reading off a CSV file with some hours, but this is a string.  The "Labor" column have values such as followed:

:15
2:30
4:00
:00
etc...

When I do my test in SQL Developer, I can convert my string into date, and then get it to decimal.  However, when I put this in my VB.net application so the user do the BulkCopy, I am getting the following error:

"Undefined function 'TO_DATE' in expression."

Here's my VB.Net code.
Dim cmd As New OleDb.OleDbCommand("SELECT [Name], [ORDER], [JOB],  " _
	& "ROUND((TO_DATE(LPAD(NVL([Duration], '00:00'), 5, '0'), 'HH24:MI')-TRUNC(TO_DATE(LPAD(NVL([Duration], '00:00'), 5, '0'), 'HH24:MI'), 'DD'))*24, 2) FROM [" + csvFileName + "]", excelstrCon)
Dim reader As OleDb.OleDbDataReader = cmd.ExecuteReader

Open in new window


Can I not use these functions?  Or do I need to do SQL function instead?
0
Comment
Question by:holemania
7 Comments
 
LVL 76

Expert Comment

by:slightwv (䄆 Netminder)
ID: 41907874
TO_DATE is an Oracle function.  If you are using OleDB to read a file and not running the query through an Oracle database engine, then you cannot use Oracle functions.

You would need an OleDB specific function when not connecting to an Oracle database.

That I cannot help with.
0
 

Author Comment

by:holemania
ID: 41907944
Do you know if there's a sql query that can take the following date string and convert into decimal?

Example:
:15
1:30
4:00

When convert into Decimal should be:
.25
1.50
4.00

I think if I can get a straight sql query to convert that and would work with Oracle, it should solve my issue.
0
 
LVL 76

Expert Comment

by:slightwv (䄆 Netminder)
ID: 41907951
Are you reading from an Oracle database or from a CSV file on disk?
0
PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

 
LVL 33

Expert Comment

by:it_saige
ID: 41907974
You would read your dataset as is and then parse the field in question using the TimeSpan.TryParseExact method that takes a list (or array) of formats, if the string to parse does not match one of the formats, the parse fails, by using TryParseExact you can opt to use a default value and issue an error; e.g. -
Imports System.ComponentModel
Imports System.Runtime.CompilerServices
Imports System.Threading

Module Module1
	Sub Main()
		Dim [data] = New List(Of Data)() From {
			New Data() With {.Name = "Job1", .Order = "12345", .Job = "1", .Labor = ":15"}, _
			New Data() With {.Name = "Job2", .Order = "98765", .Job = "2", .Labor = "2:30"}, _
			New Data() With {.Name = "Job3", .Order = "19283", .Job = "3", .Labor = "4:00"}, _
			New Data() With {.Name = "Job4", .Order = "56473", .Job = "4", .Labor = ":00"}
			}
		Dim table = [data].ConvertToDataTable()
		Dim formats = New String() {"%h", "%h\:%m", "\:%m"}
		Dim result = TimeSpan.MinValue
		For Each row In table.Rows
			If TimeSpan.TryParseExact(row("Labor"), formats, Thread.CurrentThread.CurrentCulture, result) Then
				Console.WriteLine("{0} - Labor: {1}", row("Name"), Convert.ToDecimal(result.TotalHours))
			Else
				Console.WriteLine("Parse failed for - {0} - Labor: {1}", row("Name"), row("Labor"))
			End If
		Next
		Console.ReadLine()
	End Sub
End Module

Class Data
	Public Property Name() As String
	Public Property Order() As String
	Public Property Job() As String
	Public Property Labor() As String
End Class

Module Extensions
	<Extension()> _
	Public Function ConvertToDataTable(Of T)(ByVal source As IEnumerable(Of T)) As DataTable
		Dim properties As PropertyDescriptorCollection = TypeDescriptor.GetProperties(GetType(T))
		Dim table As DataTable = New DataTable()

		For i As Integer = 0 To properties.Count - 1
			Dim [property] As PropertyDescriptor = properties(i)
			If [property].PropertyType.IsGenericType AndAlso [property].PropertyType.GetGenericTypeDefinition().Equals(GetType(Nullable)) Then
				table.Columns.Add([property].Name, [property].PropertyType.GetGenericArguments()(0))
			Else
				table.Columns.Add([property].Name, [property].PropertyType)
			End If
		Next

		Dim values(properties.Count - 1) As Object
		For Each item As T In source
			For i As Integer = 0 To properties.Count - 1
				values(i) = properties(i).GetValue(item)
			Next
			table.Rows.Add(values)
		Next

		Return table
	End Function
End Module

Open in new window

Which produces the following output -Capture.JPG
-saige-
0
 

Author Comment

by:holemania
ID: 41908087
I am reading a CSV file and doing a bulkcopy into Oracle.  However, the "Duration" field is not a string and I need to convert it into decimal before I can dump it.  Was hoping I can convert it from my example I provided with the original thread.

This is my code snippit to read from CSV file and doing the Bulkcopy.
            Dim excelCon As String
            Dim conn As String = ConnectionString()

            excelCon = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + csvFilePath + ";Extended Properties=""TEXT;HDR=YES;FMT=Delimited;Characterset=ANSI;"""

            Dim excelstrCon As New OleDb.OleDbConnection(excelCon)
            excelstrCon.Open()


            Dim cmd As New OleDb.OleDbCommand("SELECT [Name], [ORDER], [JOB], [DURATION] FROM [" + csvFileName + "]", excelstrCon)
            Dim reader As OleDb.OleDbDataReader = cmd.ExecuteReader

            Dim dbCon As OracleConnection = New OracleConnection(conn)

            dbCon.Open()

            Dim bulkCopy As OracleBulkCopy = New OracleBulkCopy(dbCon)

            bulkCopy.DestinationTableName = "LABOR"
            bulkCopy.BulkCopyTimeout = 500
            bulkCopy.WriteToServer(reader)
            reader.Close()

Open in new window

0
 
LVL 37

Accepted Solution

by:
Geert Gruwez earned 500 total points
ID: 41908279
if you can't process it in 1 go, use a staging table
it's sometimes easier to upload a line of a csv, skipping the headers, into 1 column of a table
1 line in csv = 1 record in staging table

then use a procedure to process the staging table and move the data to the final table

if have problems in converting, with a record, log it in another separate table
afterwards you can fine tune the process

if no errors occur, fine
if someone invents a new format in the csv, you can evaluate it and take actions as needed
0
 

Author Closing Comment

by:holemania
ID: 41908860
This is what I end up doing is using a staging table.
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Configuring and using Oracle Database Gateway for ODBC Introduction First, a brief summary of what a Database Gateway is.  A Gateway is a set of driver agents and configurations that allow an Oracle database to communicate with other platforms…
Using SQL Scripts we can save all the SQL queries as files that we use very frequently on our database later point of time. This is one of the feature present under SQL Workshop in Oracle Application Express.
This video shows setup options and the basic steps and syntax for duplicating (cloning) a database from one instance to another. Examples are given for duplicating to the same machine and to different machines
This video explains what a user managed backup is and shows how to take one, providing a couple of simple example scripts.

911 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now