Concatenating two binary files: c++

by: itkamarajPosted on 2009-09-10 at 06:46:27ID: 25300118

like this ?

#include <iostream>
#include <fstream>
 
using namespace std;
 
int main()
{
  ofstream outFile("FileWrite.out");
 
  if (outFile.fail()) {
    cerr << "Unable to open file for writing." << endl;
    exit(1);
  }
 
  outFile << "Hello!" << endl;
 
  outFile.close();
 
  ofstream appendFile("FileWrite.out", ios_base::app);
 
  if (appendFile.fail()) {
    cerr << "Unable to open file for writing." << endl;
    exit(1);
  }
 
  appendFile << "Append!" << endl;
 
  appendFile.close();
}

1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:
28:
29:

Select all Open in new window

by: itkamarajPosted on 2009-09-10 at 06:47:05ID: 25300124

#include <iostream>
#include <fstream>
#include <ostream>
 
using namespace  std;
 
int main()
{
   ifstream in1("test1.txt", ios::in | ios::binary);
   if(!in1)
   {
      cout << "Cannot open input file 1.\n";
      return 1;
   }
   ifstream in2("test2.txt", ios::in | ios::binary);
   if(!in2)
   {
      cout << "Cannot open input file 2.\n";
      return 1;
   }
   ofstream out("outtest.txt", ios::out | ios::binary);
 
   if(!out)
   {
      cout << "can't open output file ";
      return 1;
   }
 
   out << in1.rdbuf();
   out << " " << flush;
   out << in2.rdbuf();
 
   return 0;
}

1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:
28:
29:
30:
31:
32:
33:
34:

Select all Open in new window

by: wieser-software-ltdPosted on 2009-09-10 at 08:29:48ID: 25301207

You could use the code here:
Appending One File to Another File
http://msdn.microsoft.com/en-us/library/aa363778(VS.85).aspx

by: Infinity08Posted on 2009-09-10 at 08:44:30ID: 25301349

First of all, it depends on what you understand by 'concatenate'. Do you want to create a third file that contains the concatenation of the two other files ? Or do you want to append the second file to the end of the first file ?

In any case, an easy way to copy binary data from one file to another (assuming we're talking about big files), is to have a fixed size buffer in memory, and use 'read' to fill the buffer from one file, and then 'write' to write the buffer to the other file. Repeat this in a loop until the entire file has been copied.

http://cplusplus.com/reference/iostream/istream/read/
http://cplusplus.com/reference/iostream/ostream/write/

To open a file stream for appending, you can use the 'app' flag :

http://cplusplus.com/reference/iostream/fstream/open/

It could be as simple as :

#include <fstream>
 
int main(void) {
  std::ofstream file1("file1.bin", std::ios_base::out | std::ios_base::binary | std::ios_base::app);
  std::ifstream file2("file2.bin", std::ios_base::in | std::ios_base::binary);
 
  if (!file1 || !file2) exit(1);
 
  const size_t bufsize = 1024 * 1024;
  char buf[bufsize]; // 1 MB buffer
  do {
    file2.read(buf, bufsize);
    file1.write(buf, file2.gcount());
  } while (file1 && file2);
 
  file1.close();
  file2.close();
 
  return 0;
}

1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:

Select all Open in new window

by: Subrat2009Posted on 2009-09-10 at 09:03:30ID: 25301553

Other way,

U can use hashing technology and then comapre two files.It'll be faster.

by: evilrixPosted on 2009-09-10 at 09:46:53ID: 25302004

>> U can use hashing technology and then comapre two files.It'll be faster
To concatenate two files? How exactly?

by: peprPosted on 2009-09-10 at 11:40:04ID: 25303025

I suggest the combination of Infinity08 http:#25301349 and itkamaraj http:#25300124. Using the read file .rdbuf() method saves you from allocating buffer and explicit looping. You probably do not want to add a space between the files. Also, flushing is done when the file is closed.

If you do not want to append one existing file to the other but rather create the third file as concatenation of the two, then you have to do something like (no checking here):

std::ofstream fdest("fout.bin", std::ios_base::out | std::ios_base::binary);
std::ifstream fin1("file1.bin", std::ios_base::in | std::ios_base::binary);
std::ifstream fin2("file2.bin", std::ios_base::in | std::ios_base::binary);

fdest << fin1.rdbuf();
fdest << fin2.rdbuf();

fdest.close();
fin1.close();
fin2.close();

#include <fstream>
 
int main(void) {
  std::ofstream file1("file1.bin", std::ios_base::out | std::ios_base::binary | std::ios_base::app);
  std::ifstream file2("file2.bin", std::ios_base::in | std::ios_base::binary);
 
  if (!file1 || !file2) exit(1);
 
  file1 << file2.rdbuf();
 
  file1.close();
  file2.close();
 
  return 0;
}

1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:

Select all Open in new window

by: pgnatyukPosted on 2009-09-10 at 12:09:20ID: 25303263

I'd propose to parabellum to say that each file is about 1-2GB. :)

by: kurojiPosted on 2009-09-10 at 17:28:10ID: 25305864

Hmmm. This looks like a homework question. Tell me I'm wrong parabellum?

by: kurojiPosted on 2009-09-10 at 17:29:51ID: 25305873

It's not exactly wrong to ask such questions here, but it is better for all concerned if you are up front about it. The quality of the responses usually changes to allow you more of an opportunity to LEARN.

by: pgnatyukPosted on 2009-09-10 at 20:56:05ID: 25306616

maybe the answer should contain WinAPI like:

HANDLE hFile1 = CreateFile(,GENERIC_WRITE,);
SeekFile(hFile,); //see to end
HANDLE hFile2 = CreateFile(,GENERIC_READ,);
ReadFile(hFile2,);
WriteFile(hFile1);
CloseHandle(hFile1);
CloseHandle(hFile2);

The question is about Windows.

And this WinAPI way looks for me much better then ofstream appendFile. :)

by: parabellumPosted on 2009-09-10 at 23:44:18ID: 25307112

Well kuroji, this was not a HOMEWORK question. The windows service that i am writing is supposed to concatenate thousands of binary file pairs. And the performance is extremely important.
And i did not have enough time to try different ways to do this operation.

The file sizes are small (20 MB max) but there are thousands of them.

Thanks all for your answers.

by: kurojiPosted on 2009-09-11 at 14:02:10ID: 25313437

My apologies parabellum.
The only additional suggestion I can add to improve performance is to find out the disk sector size and use a file buffer size that is an integer multiple of that value.

DeviceIoControl IOCTL_DISK_GET_DRIVE_GEOMETRY_EX
typedef struct _DISK_GEOMETRY_EX {
DISK_GEOMETRY Geometry;
LARGE_INTEGER DiskSize;
BYTE Data[1];
}
typedef struct _DISK_GEOMETRY {
LARGE_INTEGER Cylinders;
MEDIA_TYPE MediaType;
DWORD TracksPerCylinder;
DWORD SectorsPerTrack;
DWORD BytesPerSector;
}

The only u

by: kurojiPosted on 2009-09-11 at 14:03:51ID: 25313450

The only other suggestion is to investigate using Win32 Overlapped I/O to maximize disk throughput.
That would require another thread which manages the queued up operations.

Question

Concatenating two binary files

Subscribe now for full access to Experts Exchange and get

Instant Access to this Solution

c++

C++ Programming Language

Microsoft Visual C++.Net

Trusted by hundreds of thousands everyday for fast, accurate and reliable tech support.

See what Experts Exchange can do for you.

Got a question?

We've got the answer.

Need individual assistance?

Our experts are ready to help.

Want to learn from the best?

Read articles from industry experts.

Working on a long term project?

Store your work and research.

What Makes Experts Exchange Unique?

Trusted by the world's most respected brands.

Faithfully serving IT professionals since 1996.

Related Solutions

Free Tech Articles

Cloud Class Webinars

Join the Community

Give a Little. Get a Lot.

Answers

3 Ways to Join

The Experts

The Experts

Testimonials

Testimonials

Testimonials

Business Clients

Business Clients

In the Press

In the Press

In the Press