?
Solved

txt files reading from directory into arraylist or dataset performance issues!!

Posted on 2006-05-04
2
Medium Priority
?
160 Views
Last Modified: 2010-04-23
Hi Experts,

I have raised this issue to experts in an indirectway in my previous posts but i still need to think of a system and code to avoid the duplication of a file in a predefined directory. Let me explain in detail.

My program need to copy 200  over txt files from c:\source into another  folder c:\destination (2000 txt files) and need to
check whether the same filename exist in destination or not before i copy  the file.

So one expert advised me to use an array list to store all the 2000 files in memory and while to copying the text files into c:\destination each file need to compare with the arraylist and but the perofrmace is very slow.

I also need to take into consideration of the fact that as the times goes by destination files can reach up to 5000.. so storing the file name in memory will make the program less and less effective. One good this i dont need to open to files content, just need to check the filename only, if the file is already exist  in c:\destination i need to move the existing fie in c:\destination into c:\archive and new file can be copied to c:\destination.

Private theFiles As New ArrayList()
    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        Dim temp() As String = System.IO.Directory.GetFiles("c:\B")
        For i As Integer = 0 To temp.Length - 1
            theFiles.Add(temp(i))
        Next
    End Sub

    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        Dim candidateFileToCopy As String = "SD1245-4424.txt"
        If theFiles.Contains(candidateFileToCopy) Then 'This file is in the folder. Son copy it to archive first
            System.IO.File.Copy("c:\b\" & candidateFileToCopy, "c:\archive\" & candidateFileToCopy)
            System.IO.File.Copy("c:\a\" & candidateFileToCopy, "c:\b\" & candidateFileToCopy)
        Else ' there is no file named like that
            System.IO.File.Copy("c:\a\" & candidateFileToCopy, "c:\b\" & candidateFileToCopy)
        End If
    End Sub.

If i store the filename in a database table? Will it improve the performance???

What if, while copying, somebody is using the same file?? concurrency?
pls explain how can we use thread and delegare(too high for a novice programmer like me) for the above scenario? any alteranatice solution for the above scenario.

Regards,
nyee

0
Comment
Question by:nyee84
2 Comments
 
LVL 3

Accepted Solution

by:
mazkot earned 1000 total points
ID: 16612498
You could use the file.exists to check without using a database or array...

If system.io.file.exists("c:\b\" & candidateFileToCopy) Then 'This file is in the folder. Son copy it to archive first
            System.IO.File.Copy("c:\b\" & candidateFileToCopy, "c:\archive\" & candidateFileToCopy)
            System.IO.File.Copy("c:\a\" & candidateFileToCopy, "c:\b\" & candidateFileToCopy)
        Else ' there is no file named like that
            System.IO.File.Copy("c:\a\" & candidateFileToCopy, "c:\b\" & candidateFileToCopy)
        End If
0
 

Author Comment

by:nyee84
ID: 16623529
I have used  file.exists method, it very slow when dealing with two thousand over files.

 
0

Featured Post

Important Lessons on Recovering from Petya

In their most recent webinar, Skyport Systems explores ways to isolate and protect critical databases to keep the core of your company safe from harm.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Article by: jpaulino
XML Literals are a great way to handle XML files and the community doesn’t use it as much as it should.  An XML Literal is like a String (http://msdn.microsoft.com/en-us/library/system.string.aspx) Literal, only instead of starting and ending with w…
1.0 - Introduction Converting Visual Basic 6.0 (VB6) to Visual Basic 2008+ (VB.NET). If ever there was a subject full of murkiness and bad decisions, it is this one!   The first problem seems to be that people considering this task of converting…
We’ve all felt that sense of false security before—locking down external access to a database or component and feeling like we’ve done all we need to do to secure company data. But that feeling is fleeting. Attacks these days can happen in many w…
Please read the paragraph below before following the instructions in the video — there are important caveats in the paragraph that I did not mention in the video. If your PaperPort 12 or PaperPort 14 is failing to start, or crashing, or hanging, …
Suggested Courses

839 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question