Solved

VB.Net - Oracle BulkCopy from CSV Date Format

Posted on 2016-11-30
7
97 Views
Last Modified: 2016-12-01
I am reading off a CSV file with some hours, but this is a string.  The "Labor" column have values such as followed:

:15
2:30
4:00
:00
etc...

When I do my test in SQL Developer, I can convert my string into date, and then get it to decimal.  However, when I put this in my VB.net application so the user do the BulkCopy, I am getting the following error:

"Undefined function 'TO_DATE' in expression."

Here's my VB.Net code.
Dim cmd As New OleDb.OleDbCommand("SELECT [Name], [ORDER], [JOB],  " _
	& "ROUND((TO_DATE(LPAD(NVL([Duration], '00:00'), 5, '0'), 'HH24:MI')-TRUNC(TO_DATE(LPAD(NVL([Duration], '00:00'), 5, '0'), 'HH24:MI'), 'DD'))*24, 2) FROM [" + csvFileName + "]", excelstrCon)
Dim reader As OleDb.OleDbDataReader = cmd.ExecuteReader

Open in new window


Can I not use these functions?  Or do I need to do SQL function instead?
0
Comment
Question by:holemania
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
7 Comments
 
LVL 77

Expert Comment

by:slightwv (䄆 Netminder)
ID: 41907874
TO_DATE is an Oracle function.  If you are using OleDB to read a file and not running the query through an Oracle database engine, then you cannot use Oracle functions.

You would need an OleDB specific function when not connecting to an Oracle database.

That I cannot help with.
0
 

Author Comment

by:holemania
ID: 41907944
Do you know if there's a sql query that can take the following date string and convert into decimal?

Example:
:15
1:30
4:00

When convert into Decimal should be:
.25
1.50
4.00

I think if I can get a straight sql query to convert that and would work with Oracle, it should solve my issue.
0
 
LVL 77

Expert Comment

by:slightwv (䄆 Netminder)
ID: 41907951
Are you reading from an Oracle database or from a CSV file on disk?
0
Online Training Solution

Drastically shorten your training time with WalkMe's advanced online training solution that Guides your trainees to action. Forget about retraining and skyrocket knowledge retention rates.

 
LVL 34

Expert Comment

by:it_saige
ID: 41907974
You would read your dataset as is and then parse the field in question using the TimeSpan.TryParseExact method that takes a list (or array) of formats, if the string to parse does not match one of the formats, the parse fails, by using TryParseExact you can opt to use a default value and issue an error; e.g. -
Imports System.ComponentModel
Imports System.Runtime.CompilerServices
Imports System.Threading

Module Module1
	Sub Main()
		Dim [data] = New List(Of Data)() From {
			New Data() With {.Name = "Job1", .Order = "12345", .Job = "1", .Labor = ":15"}, _
			New Data() With {.Name = "Job2", .Order = "98765", .Job = "2", .Labor = "2:30"}, _
			New Data() With {.Name = "Job3", .Order = "19283", .Job = "3", .Labor = "4:00"}, _
			New Data() With {.Name = "Job4", .Order = "56473", .Job = "4", .Labor = ":00"}
			}
		Dim table = [data].ConvertToDataTable()
		Dim formats = New String() {"%h", "%h\:%m", "\:%m"}
		Dim result = TimeSpan.MinValue
		For Each row In table.Rows
			If TimeSpan.TryParseExact(row("Labor"), formats, Thread.CurrentThread.CurrentCulture, result) Then
				Console.WriteLine("{0} - Labor: {1}", row("Name"), Convert.ToDecimal(result.TotalHours))
			Else
				Console.WriteLine("Parse failed for - {0} - Labor: {1}", row("Name"), row("Labor"))
			End If
		Next
		Console.ReadLine()
	End Sub
End Module

Class Data
	Public Property Name() As String
	Public Property Order() As String
	Public Property Job() As String
	Public Property Labor() As String
End Class

Module Extensions
	<Extension()> _
	Public Function ConvertToDataTable(Of T)(ByVal source As IEnumerable(Of T)) As DataTable
		Dim properties As PropertyDescriptorCollection = TypeDescriptor.GetProperties(GetType(T))
		Dim table As DataTable = New DataTable()

		For i As Integer = 0 To properties.Count - 1
			Dim [property] As PropertyDescriptor = properties(i)
			If [property].PropertyType.IsGenericType AndAlso [property].PropertyType.GetGenericTypeDefinition().Equals(GetType(Nullable)) Then
				table.Columns.Add([property].Name, [property].PropertyType.GetGenericArguments()(0))
			Else
				table.Columns.Add([property].Name, [property].PropertyType)
			End If
		Next

		Dim values(properties.Count - 1) As Object
		For Each item As T In source
			For i As Integer = 0 To properties.Count - 1
				values(i) = properties(i).GetValue(item)
			Next
			table.Rows.Add(values)
		Next

		Return table
	End Function
End Module

Open in new window

Which produces the following output -Capture.JPG
-saige-
0
 

Author Comment

by:holemania
ID: 41908087
I am reading a CSV file and doing a bulkcopy into Oracle.  However, the "Duration" field is not a string and I need to convert it into decimal before I can dump it.  Was hoping I can convert it from my example I provided with the original thread.

This is my code snippit to read from CSV file and doing the Bulkcopy.
            Dim excelCon As String
            Dim conn As String = ConnectionString()

            excelCon = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + csvFilePath + ";Extended Properties=""TEXT;HDR=YES;FMT=Delimited;Characterset=ANSI;"""

            Dim excelstrCon As New OleDb.OleDbConnection(excelCon)
            excelstrCon.Open()


            Dim cmd As New OleDb.OleDbCommand("SELECT [Name], [ORDER], [JOB], [DURATION] FROM [" + csvFileName + "]", excelstrCon)
            Dim reader As OleDb.OleDbDataReader = cmd.ExecuteReader

            Dim dbCon As OracleConnection = New OracleConnection(conn)

            dbCon.Open()

            Dim bulkCopy As OracleBulkCopy = New OracleBulkCopy(dbCon)

            bulkCopy.DestinationTableName = "LABOR"
            bulkCopy.BulkCopyTimeout = 500
            bulkCopy.WriteToServer(reader)
            reader.Close()

Open in new window

0
 
LVL 37

Accepted Solution

by:
Geert Gruwez earned 500 total points
ID: 41908279
if you can't process it in 1 go, use a staging table
it's sometimes easier to upload a line of a csv, skipping the headers, into 1 column of a table
1 line in csv = 1 record in staging table

then use a procedure to process the staging table and move the data to the final table

if have problems in converting, with a record, log it in another separate table
afterwards you can fine tune the process

if no errors occur, fine
if someone invents a new format in the csv, you can evaluate it and take actions as needed
0
 

Author Closing Comment

by:holemania
ID: 41908860
This is what I end up doing is using a staging table.
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Working with Network Access Control Lists in Oracle 11g (part 1) Part 2: http://www.e-e.com/A_9074.html So, you upgraded to a shiny new 11g database and all of a sudden every program that used UTL_MAIL, UTL_SMTP, UTL_TCP, UTL_HTTP or any oth…
Using SQL Scripts we can save all the SQL queries as files that we use very frequently on our database later point of time. This is one of the feature present under SQL Workshop in Oracle Application Express.
This video explains at a high level about the four available data types in Oracle and how dates can be manipulated by the user to get data into and out of the database.
This video shows how to configure and send email from and Oracle database using both UTL_SMTP and UTL_MAIL, as well as comparing UTL_SMTP to a manual SMTP conversation with a mail server.

688 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question