• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 235
  • Last Modified:

Splitting the Huge file into multiple files

Hi All,

    For some reason, I am splitting the binary file into multiple files.    But I think I made a mistake in below code and I am not able to figure it out. Please help, by knowing me where I made a mistake in the code,

If you feel below code is not a right idea, I would like to have an alternate solution in sp;itting the files

Thanks,
Sudhakar  


       long x=0;
      while(true)
      {
            BYTE* buf=new BYTE[200000];
            long n=file.Read(buf,200000);
             x=x+n;
            if(n<=0)
            {
                  
                  break;
            }
            CString str;
            str.Format("%ld",n);
            CString strFile;
            strFile.Format("C:\\Files\\%d.msg",count);
            CFile file1;
            file1.Open(strFile,CFile::modeCreate|CFile::modeReadWrite|CFile::shareDenyNone|CFile::typeBinary);
            file1.Write(buf,n);
            count++;
            file1.Close();
            
            file.Seek(x,CFile::begin);
            Sleep(2);
      }
      file.Close();
0
sudhakar_koundinya
Asked:
sudhakar_koundinya
  • 4
  • 4
  • 2
2 Solutions
 
BarthaxCommented:
Personally, I would put all your variable declarations outside the loop at the top of the function - especially the BYTE* buf=new BYTE[20000]; statement as that is never released from memory, but is re-created at each loop of the while().

Your use of the Seek is fairly redundant as the call to Read will position the file cursor at the next byte anyway.

Additionally the use of while(true) is poor programming practice - you can easily put while(n<=0) if you shift around your use of the first file.Read(buf,200000) to before the loop and the last in the loop:

n=file.Read(buf,200000);
while(n<=0)
{
  x=x+n;
  str.Format(...
...
  n=file.Read(buf,200000);
}
...

Don't forget to destroy the buf variable's memory at the end.
0
 
sudhakar_koundinyaAuthor Commented:
Hi

Thank You for your comments.

I change the code as you mentioned. But I am still getting wrong set of data.

Actually the file is of size 35 mb. When I try my code/your code I am getting total of 968 KB of data only i.e. it is able to split 10 files only. Actually it should split arround 185 files. I used both read and readHuge methods. but no use. I need your help more

Thanks
Sudhakar

                int count=0;
            BYTE* buf=new BYTE[200000];
            long n=file.ReadHuge(buf,200000);
            CString strFile;
            CString str;
            CFile file1;
      while(n>0)
      {
            str.Format("%ld",n);            
            strFile.Format("C:\\DerexlFiles\\%d.txt",count);                     file1.Open(strFile,CFile::modeCreate|CFile::modeReadWrite|CFile::shareDenyNone|CFile::typeBinary);
            file1.WriteHuge(buf,n);
            AfxMessageBox(strFile+" "+str);
            count++;
            file1.Close();            
            Sleep(2);
            n=file.ReadHuge(buf,200000);            
      }
      delete []buf;
0
 
BarthaxCommented:
You need to start adding some error handling to the code.  The CFile::Open returns a false if it fails.  If it does fail, then the Write/WriteHuge will throw an exception - you may be inadvertently capturing the exception elsewhere.  I'd also try stepping through the code - especially if you are not getting enough message boxes from your AfxMessageBox call confirming that each write has occured.
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
sudhakar_koundinyaAuthor Commented:
Hi,

I was able to open the file perfectly. for 10 iterationsof while loop, it has created 10 files and I am getting the data. But at 11 th iteration ReadHuge(buf,200000) is returning 0. So it is coming out of loop. And also I am not getting any exception while reading from file or writing to the file

Thanks
Sudhakar
0
 
BarthaxCommented:
Is the last file of the correct size (200000) or is it smaller?  If it is smaller, then that would suggest you did hit the end of the file and your posted code is correctly working - but your unposted code is probably setting the file pointer to a latter location than you might expect.
0
 
sudhakar_koundinyaAuthor Commented:
>> Is the last file of the correct size (200000) or is it smaller??

It is 200000 then in next iteration ReadHuge is returning 0 bytes
0
 
Julian HansenCommented:
n=file.ReadHuge(buf,200000);

to

n=file.ReadHuge(buf,2000);

Does it get past 11 iterations?
If so does it read the entire file?

This may provide some insight as to why it is failing.
0
 
sudhakar_koundinyaAuthor Commented:
>>Does it get past 11 iterations?
No. At 11 th iteration it is returning out from the method
>>If so does it read the entire file?
No

0
 
BarthaxCommented:
Hmmm.. what happens on a smaller file?  How about a 1k file (where it should read everything in with the first call to ReadHuge)?
0
 
Julian HansenCommented:
Sorry - should have seen this before.

The test I posted above shows that it has more to do with the iterations than then size of the file.

What I noticed is that in your while() loop you are allocating a new buffer on each iteration i.e.
while ( true )
{
  BYTE* buf=new BYTE[200000];
  ...
}

This is not a good idea.

Rathe move the buffer allocation outside the while loop

BYTE* buf=new BYTE[200000];
while ( true )
{
   ...
}

delete [] buf ;

See if that makes a difference.

0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

  • 4
  • 4
  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now