We help IT Professionals succeed at work.

Get List of Specific Amount of Files From Directory

angus_young_acdc
angus_young_acdc used Ask the Experts™
on
Hello Experts,

I have to copy 75GB worth of files from various sub folders in a directory, of course this would take a right ole age to actually complete, and as our companies network is notoriously bad the idea is to potentially copy X GB worth of the data at a time overnight.

My question is this, how exactly can I implement something to get X GB worth of files?  I've had a look at Directory.GetFiles and DirectoryInfo GetFiles methods, but neither of them appear to have a limit.  Ideally I would like something, similar to SQL, where I could just have (obviously in C# not SQL :-P ) SELECT TOP 10000 and get a set amount of data.

Is this possible?  And if so, how can I do it?
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Kamal KhaleefaInformation Security Specialist

Commented:
hi
i suggest on you to manually separate the files you have into many subfolders
and manually put in each folder amount of 10 GB for exampe
then in your code create an array of the paths to these folders
and make a loop through them

also
another solution is
writte a code to read file by file and store the length in a variable and when this variable is about 10 GB make a copy


some helpful code may be this


  Dim files() As String
          
                files = Directory.GetFiles("\\path to your files")
           

          

            For i As Integer = 0 To files.Count - 1
                Dim File_Name As String = ""
                File_Name = files(i).Remove(0, files(i).LastIndexOf("\") + 1)

                Dim beg As Integer = File_Name.IndexOf(".")
                Dim lngth As Integer = File_Name.Length
                If beg <> -1 Then
                    File_Name = File_Name.Remove(beg, lngth - beg)  

                 

Open in new window

this code get a list of nGig pack of files: ***in C# syntax***

List<List<string>> Files= new List<List<string>>();
      long maxlength = 0;
      foreach (string filepath in Directory.GetFiles("your folder path", "*", SearchOption.AllDirectories))
      {
        FileInfo file = new FileInfo(filepath);
        if (maxlength - file.Length < 0)
        {
          Files.Add(new List<string>());
          maxlength = 1000000;
        }

        Files[Files.Count - 1].Add(filepath);
        maxlength -= file.Length;
      }

Open in new window

If C# is not a requirement - you could use ROBOCOPY in 2 different ways:
1. One suggestion is to use the /MIR parameter (for mirroring that 2 "roots") and if "move" is needed you can delete the "source" folder when mirroring is complete
2. You cand copy files based on their "age" so you could move the files based on their "creation time" with /MAXAGE /MINAGE parameters

Or a 3rd approach - you can use ROBOCOPY with multiple passes to copy large files "over-night" and small files whenever you want so finally you'll have all the files on the new location...

Commented:
>>but neither of them appear to have a limit.
So, you want a limit?
Limit input sample 75.1 GB,  because you want 75.GB
And after limit has reach, you want an alert?
I just ended up copying them all, however this would appear to provide a solution in C#