Junk characters while opening a file


I have redirected stdout to a file in C.
This file is being opened/closed at several points of execution.
Only during one particular flow of execution, when I open the file junk characters are written before the first statement is written to the file.

I am wondering if file pointer is corrupted or file buffer has junk at the beginning.

I am using freopen function call.
Please let me know how to track this problem.

Is there a way to purge the file buffer without writing to the file?
I see fpurge in linux but HP-UX I am using doesnt seem to support this.

Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Kent OlsenData Warehouse Architect / DBACommented:

Several "red flags" come up on your query.

1)  Don't open and close stdout "several times" in a program.  It probably won't "hurt", but there are better ways.

2)  freopen() closes the current stream even when opening the new file fails.  Since you're opening an output stream, most failures will be due to read/write conflicts, but they are still possible.  It's possible that opening the new file could have failed and stdout now contains an invalid pointer.  One possible scenario is that stdout is closed but the pointer still points to the old buffer making it LOOK like a stream.  Of course, stdout could also contain NULL.  I don't believe that the stream library checks for NULL so you could be putting random data in some pretty sensitive areas of the O/S.

3)  Check that you're using the correct open modes on the freopen().  Are you trying to create a new file ("w") or append to an existing one ("a")?


MRAAuthor Commented:
Hi Kdo,

Thanks a lot for the reply.
Answering your points,

1)The stdout is being closed/open at several times by already existing code.I will have to change it in that case.

2) Your guess is right!
I ran gdb and found that even though stdout is closed, the file pointer is still having some old values as you said could be old buffer.

3)I need to append to existing file hence using  'a' with freopen.

Also I found that
As soon as I do freopen junk is written to file,
So ,I concluded that junk is in file buffer and file pointer is valid?

Am I right in assuming so?

So I tried the following :

fflush(stdout); // so that junk gets flushed
if ((r_trace_file = freopen(rfname, io_type, stdout)//io_type =a

but dint help.

Please let me know your suggestions to fix this.

Kent OlsenData Warehouse Architect / DBACommented:


My first thought was "are you sure that the 'junk' that you're seeing wasn't in the buffer when the fclose() was executed"?

Upon further review....

stdin, stdout, and stderr are "standard" streams that are manipulated by C's I/O libraries.  Your code closes stdout, and opens the stream r_trace_file.  You could do the same with:

fclose (stdout);
r_trace_file = fopen (rfname, io_type);

Which leads to the question, "Why are you closing stdout"?

Since you haven't changed the stdout pointer, it still points to the old buffer, but the stream is closed.  (A VERY dangerous thing to do!)  Other routines may see the stdout pointer and attempt output.

stdout NEEDS to point to a valid stream, even if the stream is the null device.  On unix systems it's called "/dev/null".  I like to think of it as "write only" memory.  ;)

Try one of these approaches:

stdout = freopen (rfname, io_type, stdout);

This will redirect stdout to "rfname".  Your stream writes will, of course, have to be directed to stdout.

If you've got a lot of code that already uses "r_trace_file" as the stream, a bit of ugliness can get you around this:

1)  r_trace_file = stdout;  /* write to stdout */

2)  r_trace_file = fopen (rtname, io_type);

Before changing the r_trace_file pointer you will need to do this:

  if (r_trace_file && r_trace_file != stdout)
    fclose (r_trace_file);
    r_trace_file = NULL;

If you want to toggle your output between stdout and another file, try this:

AlternateStdout = fopen (rfname, io_type, stdout);

Then just set r_trace_file to the correct stream.

Good Luck,
Keep up with what's happening at Experts Exchange!

Sign up to receive Decoded, a new monthly digest with product updates, feature release info, continuing education opportunities, and more.

MRAAuthor Commented:
Hi Kdo,

Not much luck :-(  I tried all methods and here goes the report.

1)  Tried to use fopen and and then redirecting stdout
  if ((r_trace_file = fopen(rfname, io_type)) == NULL) {

                     //Debug statements
                 r_trace_file = stdout;
                fprintf(r_trace_file, "Begin R Tracing\n");

But the file in rfname remained 0 bytes.
GDB showed that r_tarce_file had all the print statements but it never got into file

2nd method,
I tried to open a temporary junk file so that junk gets written to that file then I can open the required file
But still junk got written to r_trace_file !
Temporary junk.txt remained 0 bytes.

  if ((r_trace_file = freopen("junk.txt", io_type, stdout
)) == NULL) {
                       ///Debug statements
//I expect junk.txt to be closed during second freopen
  if ((r_trace_file = freopen(rfname, io_type,stdout)) == NULL) {
                      ///Debug statements

Also closed r_trace_file in both cases before doing freopen/fopen,
if ((r_trace_file != NULL) && (r_trace_file != stdout
)) {

Please let me know your inputs

Kent OlsenData Warehouse Architect / DBACommented:

I guess my description was lacking... sorry...

1)  'stdout' is not a stream.  It is a pointer to a stream.  When you execute 'r_trace_file = stdout;' you're setting the pointer 'r_trace_file' to ALSO point to the 'stdout' stream.  Since 'r_trace_file' and 'stdout' now point to the same place, the subsequent 'fprintf (r_trace_file,...)' should print on 'stdout'.

In this method, just omit the 'r_trace_file = stdout' and you should at least get something written to the file.

2)  This goes back to the earlier example.

  r_trace_file = freopen("junk.txt", io_type, stdout);

is equivalent to:

  fclose (stdout);
  r_trace_file = fopen ("junk.txt", io_type);

This CLOSES the 'stdout' stream.  It doesn't point it to 'r_trace_file'.  If any of the C library routines (or any of your modules) write to 'stdout' they are now using an invalid pointer and the results are unpredictable.

To have 'stdout' point to your file, you need:

  stdout = freopen ("junk.txt", io_type, stdout);

If you want two pointers to the same stream (file) then use:

  stdout = freopen ("junk.txt", io_type, stdout);
  r_trace_file = stdout;

And of course, you can always leave 'stdout' pointing to the display and write to your own file just by opening a new strean:

  r_trace_file = fopen ("junk.txt", io_type);

Thankfully, you don't have to reset 'stdout' to its original value.  The operating system resets 'stdout' and 'stderr' to the terminal device at each job step.  (Well, not exactly -- but that's a whole explanation by itself.)  If not, little things like the prompt would go to your redirected file until you ran a program that pointed 'stdout' back to the display.

MRAAuthor Commented:

Some improvement today.
I used setvbug along with freopen and the junk is gone
but the code is dumping core !

I added these lines of code extra and here it is.

I declared userdefined BUF using,

char *BUF = (char*)malloc(sizeof(1025));

///just before doing freopen i added memset.
//Since I suspect junk is in the file buffer just before doing freopen

                if ((rascal_trace_file = freopen(rfname, io_type, stdout)) == NULL) {
                 //debug statements
//added setvbuf immediately after freopen

 setvbuf ( rascal_trace_file,BUF,_IOFBF , 1024 );

It is coring in an unrelated code while Database fetch.

Please let me know if I am going wrong anywhere.
Also I am not sure why using userdefined BUF eliminated the junk chars.


Kent OlsenData Warehouse Architect / DBACommented:

It looks like you've still got the same basic bug -- you're closing stdout and leaving a bad pointer in it.

In the last example that you sent, change:

rascal_trace_file = freopen (rfname, io_type, stdout)


rascal_trace_file = fopen (rtname, io_type)

and see what happens.

MRAAuthor Commented:

I finally fixed the problem with 1 line change by calling

setvbuf ( rascal_trace_file,NULL, _IOFBF , 1024*3 ); after
if ((rascal_trace_file = freopen(rfname, io_type, stdout)) == NULL) {
                //debug statements
While it was coring , I had declared a user defined buffer , here I am using system buffer but restricting the size to 2048.

But the strange thing is if I set the file buffer size to any value >= 1024*5 in setvbuf junk still appears!

What could be the reason???
Please let me know

Is this a good way to fix the problem???


Kent OlsenData Warehouse Architect / DBACommented:

Offhand, I don't think that you've actually fixed the problem, just "masked" it.  As your code changes (and the binary changes size) this might again rear its ugly head.

I still think that you should open rascal_trace_file with fopen(), not freopen(,,stdout).

But I'm glad that it's doing better!

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
No comment has been added lately, so it's time to clean up this TA.
I will leave a recommendation in the Cleanup topic area that this question is:

Answered by Kdo

Please leave any comments here within the next seven days.


EE Cleanup Volunteer
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
System Programming

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.