Link to home
Start Free TrialLog in
Avatar of nyee84
nyee84

asked on

txt files reading from directory into arraylist or dataset performance issues!!

Hi Experts,

I have raised this issue to experts in an indirectway in my previous posts but i still need to think of a system and code to avoid the duplication of a file in a predefined directory. Let me explain in detail.

My program need to copy 200  over txt files from c:\source into another  folder c:\destination (2000 txt files) and need to
check whether the same filename exist in destination or not before i copy  the file.

So one expert advised me to use an array list to store all the 2000 files in memory and while to copying the text files into c:\destination each file need to compare with the arraylist and but the perofrmace is very slow.

I also need to take into consideration of the fact that as the times goes by destination files can reach up to 5000.. so storing the file name in memory will make the program less and less effective. One good this i dont need to open to files content, just need to check the filename only, if the file is already exist  in c:\destination i need to move the existing fie in c:\destination into c:\archive and new file can be copied to c:\destination.

Private theFiles As New ArrayList()
    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        Dim temp() As String = System.IO.Directory.GetFiles("c:\B")
        For i As Integer = 0 To temp.Length - 1
            theFiles.Add(temp(i))
        Next
    End Sub

    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        Dim candidateFileToCopy As String = "SD1245-4424.txt"
        If theFiles.Contains(candidateFileToCopy) Then 'This file is in the folder. Son copy it to archive first
            System.IO.File.Copy("c:\b\" & candidateFileToCopy, "c:\archive\" & candidateFileToCopy)
            System.IO.File.Copy("c:\a\" & candidateFileToCopy, "c:\b\" & candidateFileToCopy)
        Else ' there is no file named like that
            System.IO.File.Copy("c:\a\" & candidateFileToCopy, "c:\b\" & candidateFileToCopy)
        End If
    End Sub.

If i store the filename in a database table? Will it improve the performance???

What if, while copying, somebody is using the same file?? concurrency?
pls explain how can we use thread and delegare(too high for a novice programmer like me) for the above scenario? any alteranatice solution for the above scenario.

Regards,
nyee

ASKER CERTIFIED SOLUTION
Avatar of mazkot
mazkot

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of nyee84
nyee84

ASKER

I have used  file.exists method, it very slow when dealing with two thousand over files.