?
Solved

Splitting the Huge file into multiple files

Posted on 2005-03-19
10
Medium Priority
?
230 Views
Last Modified: 2013-11-20
Hi All,

    For some reason, I am splitting the binary file into multiple files.    But I think I made a mistake in below code and I am not able to figure it out. Please help, by knowing me where I made a mistake in the code,

If you feel below code is not a right idea, I would like to have an alternate solution in sp;itting the files

Thanks,
Sudhakar  


       long x=0;
      while(true)
      {
            BYTE* buf=new BYTE[200000];
            long n=file.Read(buf,200000);
             x=x+n;
            if(n<=0)
            {
                  
                  break;
            }
            CString str;
            str.Format("%ld",n);
            CString strFile;
            strFile.Format("C:\\Files\\%d.msg",count);
            CFile file1;
            file1.Open(strFile,CFile::modeCreate|CFile::modeReadWrite|CFile::shareDenyNone|CFile::typeBinary);
            file1.Write(buf,n);
            count++;
            file1.Close();
            
            file.Seek(x,CFile::begin);
            Sleep(2);
      }
      file.Close();
0
Comment
Question by:sudhakar_koundinya
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 4
  • 2
10 Comments
 
LVL 19

Assisted Solution

by:Barthax
Barthax earned 200 total points
ID: 13583402
Personally, I would put all your variable declarations outside the loop at the top of the function - especially the BYTE* buf=new BYTE[20000]; statement as that is never released from memory, but is re-created at each loop of the while().

Your use of the Seek is fairly redundant as the call to Read will position the file cursor at the next byte anyway.

Additionally the use of while(true) is poor programming practice - you can easily put while(n<=0) if you shift around your use of the first file.Read(buf,200000) to before the loop and the last in the loop:

n=file.Read(buf,200000);
while(n<=0)
{
  x=x+n;
  str.Format(...
...
  n=file.Read(buf,200000);
}
...

Don't forget to destroy the buf variable's memory at the end.
0
 
LVL 14

Author Comment

by:sudhakar_koundinya
ID: 13584327
Hi

Thank You for your comments.

I change the code as you mentioned. But I am still getting wrong set of data.

Actually the file is of size 35 mb. When I try my code/your code I am getting total of 968 KB of data only i.e. it is able to split 10 files only. Actually it should split arround 185 files. I used both read and readHuge methods. but no use. I need your help more

Thanks
Sudhakar

                int count=0;
            BYTE* buf=new BYTE[200000];
            long n=file.ReadHuge(buf,200000);
            CString strFile;
            CString str;
            CFile file1;
      while(n>0)
      {
            str.Format("%ld",n);            
            strFile.Format("C:\\DerexlFiles\\%d.txt",count);                     file1.Open(strFile,CFile::modeCreate|CFile::modeReadWrite|CFile::shareDenyNone|CFile::typeBinary);
            file1.WriteHuge(buf,n);
            AfxMessageBox(strFile+" "+str);
            count++;
            file1.Close();            
            Sleep(2);
            n=file.ReadHuge(buf,200000);            
      }
      delete []buf;
0
 
LVL 19

Expert Comment

by:Barthax
ID: 13584987
You need to start adding some error handling to the code.  The CFile::Open returns a false if it fails.  If it does fail, then the Write/WriteHuge will throw an exception - you may be inadvertently capturing the exception elsewhere.  I'd also try stepping through the code - especially if you are not getting enough message boxes from your AfxMessageBox call confirming that each write has occured.
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 
LVL 14

Author Comment

by:sudhakar_koundinya
ID: 13586725
Hi,

I was able to open the file perfectly. for 10 iterationsof while loop, it has created 10 files and I am getting the data. But at 11 th iteration ReadHuge(buf,200000) is returning 0. So it is coming out of loop. And also I am not getting any exception while reading from file or writing to the file

Thanks
Sudhakar
0
 
LVL 19

Expert Comment

by:Barthax
ID: 13589653
Is the last file of the correct size (200000) or is it smaller?  If it is smaller, then that would suggest you did hit the end of the file and your posted code is correctly working - but your unposted code is probably setting the file pointer to a latter location than you might expect.
0
 
LVL 14

Author Comment

by:sudhakar_koundinya
ID: 13589673
>> Is the last file of the correct size (200000) or is it smaller??

It is 200000 then in next iteration ReadHuge is returning 0 bytes
0
 
LVL 58

Expert Comment

by:Julian Hansen
ID: 13590359
n=file.ReadHuge(buf,200000);

to

n=file.ReadHuge(buf,2000);

Does it get past 11 iterations?
If so does it read the entire file?

This may provide some insight as to why it is failing.
0
 
LVL 14

Author Comment

by:sudhakar_koundinya
ID: 13651807
>>Does it get past 11 iterations?
No. At 11 th iteration it is returning out from the method
>>If so does it read the entire file?
No

0
 
LVL 19

Expert Comment

by:Barthax
ID: 13652034
Hmmm.. what happens on a smaller file?  How about a 1k file (where it should read everything in with the first call to ReadHuge)?
0
 
LVL 58

Accepted Solution

by:
Julian Hansen earned 200 total points
ID: 13652517
Sorry - should have seen this before.

The test I posted above shows that it has more to do with the iterations than then size of the file.

What I noticed is that in your while() loop you are allocating a new buffer on each iteration i.e.
while ( true )
{
  BYTE* buf=new BYTE[200000];
  ...
}

This is not a good idea.

Rathe move the buffer allocation outside the while loop

BYTE* buf=new BYTE[200000];
while ( true )
{
   ...
}

delete [] buf ;

See if that makes a difference.

0

Featured Post

Moving data to the cloud? Find out if you’re ready

Before moving to the cloud, it is important to carefully define your db needs, plan for the migration & understand prod. environment. This wp explains how to define what you need from a cloud provider, plan for the migration & what putting a cloud solution into practice entails.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Have you tried to learn about Unicode, UTF-8, and multibyte text encoding and all the articles are just too "academic" or too technical? This article aims to make the whole topic easy for just about anyone to understand.
Ready to get certified? Check out some courses that help you prepare for third-party exams.
This video will show you how to get GIT to work in Eclipse.   It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.
Do you want to know how to make a graph with Microsoft Access? First, create a query with the data for the chart. Then make a blank form and add a chart control. This video also shows how to change what data is displayed on the graph as well as form…
Suggested Courses
Course of the Month7 days, 21 hours left to enroll

765 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question