Solved

VB.Net - Oracle BulkCopy from CSV Date Format

Posted on 2016-11-30
7
22 Views
Last Modified: 2016-12-01
I am reading off a CSV file with some hours, but this is a string.  The "Labor" column have values such as followed:

:15
2:30
4:00
:00
etc...

When I do my test in SQL Developer, I can convert my string into date, and then get it to decimal.  However, when I put this in my VB.net application so the user do the BulkCopy, I am getting the following error:

"Undefined function 'TO_DATE' in expression."

Here's my VB.Net code.
Dim cmd As New OleDb.OleDbCommand("SELECT [Name], [ORDER], [JOB],  " _
	& "ROUND((TO_DATE(LPAD(NVL([Duration], '00:00'), 5, '0'), 'HH24:MI')-TRUNC(TO_DATE(LPAD(NVL([Duration], '00:00'), 5, '0'), 'HH24:MI'), 'DD'))*24, 2) FROM [" + csvFileName + "]", excelstrCon)
Dim reader As OleDb.OleDbDataReader = cmd.ExecuteReader

Open in new window


Can I not use these functions?  Or do I need to do SQL function instead?
0
Comment
Question by:holemania
7 Comments
 
LVL 76

Expert Comment

by:slightwv (䄆 Netminder)
Comment Utility
TO_DATE is an Oracle function.  If you are using OleDB to read a file and not running the query through an Oracle database engine, then you cannot use Oracle functions.

You would need an OleDB specific function when not connecting to an Oracle database.

That I cannot help with.
0
 

Author Comment

by:holemania
Comment Utility
Do you know if there's a sql query that can take the following date string and convert into decimal?

Example:
:15
1:30
4:00

When convert into Decimal should be:
.25
1.50
4.00

I think if I can get a straight sql query to convert that and would work with Oracle, it should solve my issue.
0
 
LVL 76

Expert Comment

by:slightwv (䄆 Netminder)
Comment Utility
Are you reading from an Oracle database or from a CSV file on disk?
0
Top 6 Sources for Identifying Threat Actor TTPs

Understanding your enemy is essential. These six sources will help you identify the most popular threat actor tactics, techniques, and procedures (TTPs).

 
LVL 32

Expert Comment

by:it_saige
Comment Utility
You would read your dataset as is and then parse the field in question using the TimeSpan.TryParseExact method that takes a list (or array) of formats, if the string to parse does not match one of the formats, the parse fails, by using TryParseExact you can opt to use a default value and issue an error; e.g. -
Imports System.ComponentModel
Imports System.Runtime.CompilerServices
Imports System.Threading

Module Module1
	Sub Main()
		Dim [data] = New List(Of Data)() From {
			New Data() With {.Name = "Job1", .Order = "12345", .Job = "1", .Labor = ":15"}, _
			New Data() With {.Name = "Job2", .Order = "98765", .Job = "2", .Labor = "2:30"}, _
			New Data() With {.Name = "Job3", .Order = "19283", .Job = "3", .Labor = "4:00"}, _
			New Data() With {.Name = "Job4", .Order = "56473", .Job = "4", .Labor = ":00"}
			}
		Dim table = [data].ConvertToDataTable()
		Dim formats = New String() {"%h", "%h\:%m", "\:%m"}
		Dim result = TimeSpan.MinValue
		For Each row In table.Rows
			If TimeSpan.TryParseExact(row("Labor"), formats, Thread.CurrentThread.CurrentCulture, result) Then
				Console.WriteLine("{0} - Labor: {1}", row("Name"), Convert.ToDecimal(result.TotalHours))
			Else
				Console.WriteLine("Parse failed for - {0} - Labor: {1}", row("Name"), row("Labor"))
			End If
		Next
		Console.ReadLine()
	End Sub
End Module

Class Data
	Public Property Name() As String
	Public Property Order() As String
	Public Property Job() As String
	Public Property Labor() As String
End Class

Module Extensions
	<Extension()> _
	Public Function ConvertToDataTable(Of T)(ByVal source As IEnumerable(Of T)) As DataTable
		Dim properties As PropertyDescriptorCollection = TypeDescriptor.GetProperties(GetType(T))
		Dim table As DataTable = New DataTable()

		For i As Integer = 0 To properties.Count - 1
			Dim [property] As PropertyDescriptor = properties(i)
			If [property].PropertyType.IsGenericType AndAlso [property].PropertyType.GetGenericTypeDefinition().Equals(GetType(Nullable)) Then
				table.Columns.Add([property].Name, [property].PropertyType.GetGenericArguments()(0))
			Else
				table.Columns.Add([property].Name, [property].PropertyType)
			End If
		Next

		Dim values(properties.Count - 1) As Object
		For Each item As T In source
			For i As Integer = 0 To properties.Count - 1
				values(i) = properties(i).GetValue(item)
			Next
			table.Rows.Add(values)
		Next

		Return table
	End Function
End Module

Open in new window

Which produces the following output -Capture.JPG
-saige-
0
 

Author Comment

by:holemania
Comment Utility
I am reading a CSV file and doing a bulkcopy into Oracle.  However, the "Duration" field is not a string and I need to convert it into decimal before I can dump it.  Was hoping I can convert it from my example I provided with the original thread.

This is my code snippit to read from CSV file and doing the Bulkcopy.
            Dim excelCon As String
            Dim conn As String = ConnectionString()

            excelCon = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + csvFilePath + ";Extended Properties=""TEXT;HDR=YES;FMT=Delimited;Characterset=ANSI;"""

            Dim excelstrCon As New OleDb.OleDbConnection(excelCon)
            excelstrCon.Open()


            Dim cmd As New OleDb.OleDbCommand("SELECT [Name], [ORDER], [JOB], [DURATION] FROM [" + csvFileName + "]", excelstrCon)
            Dim reader As OleDb.OleDbDataReader = cmd.ExecuteReader

            Dim dbCon As OracleConnection = New OracleConnection(conn)

            dbCon.Open()

            Dim bulkCopy As OracleBulkCopy = New OracleBulkCopy(dbCon)

            bulkCopy.DestinationTableName = "LABOR"
            bulkCopy.BulkCopyTimeout = 500
            bulkCopy.WriteToServer(reader)
            reader.Close()

Open in new window

0
 
LVL 36

Accepted Solution

by:
Geert Gruwez earned 500 total points
Comment Utility
if you can't process it in 1 go, use a staging table
it's sometimes easier to upload a line of a csv, skipping the headers, into 1 column of a table
1 line in csv = 1 record in staging table

then use a procedure to process the staging table and move the data to the final table

if have problems in converting, with a record, log it in another separate table
afterwards you can fine tune the process

if no errors occur, fine
if someone invents a new format in the csv, you can evaluate it and take actions as needed
0
 

Author Closing Comment

by:holemania
Comment Utility
This is what I end up doing is using a staging table.
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

From implementing a password expiration date, to datatype conversions and file export options, these are some useful settings I've found in Jasper Server.
If you need to start windows update installation remotely or as a scheduled task you will find this very helpful.
Via a live example show how to connect to RMAN, make basic configuration settings changes and then take a backup of a demo database
This video shows how to copy a database user from one database to another user DBMS_METADATA.  It also shows how to copy a user's permissions and discusses password hash differences between Oracle 10g and 11g.

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now