Junk characters while opening a file

Posted on 2003-02-24
Medium Priority
Last Modified: 2013-12-26

I have redirected stdout to a file in C.
This file is being opened/closed at several points of execution.
Only during one particular flow of execution, when I open the file junk characters are written before the first statement is written to the file.

I am wondering if file pointer is corrupted or file buffer has junk at the beginning.

I am using freopen function call.
Please let me know how to track this problem.

Is there a way to purge the file buffer without writing to the file?
I see fpurge in linux but HP-UX I am using doesnt seem to support this.

Question by:MRA
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 4
LVL 45

Expert Comment

by:Kent Olsen
ID: 8008627

Several "red flags" come up on your query.

1)  Don't open and close stdout "several times" in a program.  It probably won't "hurt", but there are better ways.

2)  freopen() closes the current stream even when opening the new file fails.  Since you're opening an output stream, most failures will be due to read/write conflicts, but they are still possible.  It's possible that opening the new file could have failed and stdout now contains an invalid pointer.  One possible scenario is that stdout is closed but the pointer still points to the old buffer making it LOOK like a stream.  Of course, stdout could also contain NULL.  I don't believe that the stream library checks for NULL so you could be putting random data in some pretty sensitive areas of the O/S.

3)  Check that you're using the correct open modes on the freopen().  Are you trying to create a new file ("w") or append to an existing one ("a")?



Author Comment

ID: 8009070
Hi Kdo,

Thanks a lot for the reply.
Answering your points,

1)The stdout is being closed/open at several times by already existing code.I will have to change it in that case.

2) Your guess is right!
I ran gdb and found that even though stdout is closed, the file pointer is still having some old values as you said could be old buffer.

3)I need to append to existing file hence using  'a' with freopen.

Also I found that
As soon as I do freopen junk is written to file,
So ,I concluded that junk is in file buffer and file pointer is valid?

Am I right in assuming so?

So I tried the following :

fflush(stdout); // so that junk gets flushed
if ((r_trace_file = freopen(rfname, io_type, stdout)//io_type =a

but dint help.

Please let me know your suggestions to fix this.

LVL 45

Expert Comment

by:Kent Olsen
ID: 8009330


My first thought was "are you sure that the 'junk' that you're seeing wasn't in the buffer when the fclose() was executed"?

Upon further review....

stdin, stdout, and stderr are "standard" streams that are manipulated by C's I/O libraries.  Your code closes stdout, and opens the stream r_trace_file.  You could do the same with:

fclose (stdout);
r_trace_file = fopen (rfname, io_type);

Which leads to the question, "Why are you closing stdout"?

Since you haven't changed the stdout pointer, it still points to the old buffer, but the stream is closed.  (A VERY dangerous thing to do!)  Other routines may see the stdout pointer and attempt output.

stdout NEEDS to point to a valid stream, even if the stream is the null device.  On unix systems it's called "/dev/null".  I like to think of it as "write only" memory.  ;)

Try one of these approaches:

stdout = freopen (rfname, io_type, stdout);

This will redirect stdout to "rfname".  Your stream writes will, of course, have to be directed to stdout.

If you've got a lot of code that already uses "r_trace_file" as the stream, a bit of ugliness can get you around this:

1)  r_trace_file = stdout;  /* write to stdout */

2)  r_trace_file = fopen (rtname, io_type);

Before changing the r_trace_file pointer you will need to do this:

  if (r_trace_file && r_trace_file != stdout)
    fclose (r_trace_file);
    r_trace_file = NULL;

If you want to toggle your output between stdout and another file, try this:

AlternateStdout = fopen (rfname, io_type, stdout);

Then just set r_trace_file to the correct stream.

Good Luck,
Will your db performance match your db growth?

In Percona’s white paper “Performance at Scale: Keeping Your Database on Its Toes,” we take a high-level approach to what you need to think about when planning for database scalability.


Author Comment

ID: 8016609
Hi Kdo,

Not much luck :-(  I tried all methods and here goes the report.

1)  Tried to use fopen and and then redirecting stdout
  if ((r_trace_file = fopen(rfname, io_type)) == NULL) {

                     //Debug statements
                 r_trace_file = stdout;
                fprintf(r_trace_file, "Begin R Tracing\n");

But the file in rfname remained 0 bytes.
GDB showed that r_tarce_file had all the print statements but it never got into file

2nd method,
I tried to open a temporary junk file so that junk gets written to that file then I can open the required file
But still junk got written to r_trace_file !
Temporary junk.txt remained 0 bytes.

  if ((r_trace_file = freopen("junk.txt", io_type, stdout
)) == NULL) {
                       ///Debug statements
//I expect junk.txt to be closed during second freopen
  if ((r_trace_file = freopen(rfname, io_type,stdout)) == NULL) {
                      ///Debug statements

Also closed r_trace_file in both cases before doing freopen/fopen,
if ((r_trace_file != NULL) && (r_trace_file != stdout
)) {

Please let me know your inputs

LVL 45

Expert Comment

by:Kent Olsen
ID: 8016819

I guess my description was lacking... sorry...

1)  'stdout' is not a stream.  It is a pointer to a stream.  When you execute 'r_trace_file = stdout;' you're setting the pointer 'r_trace_file' to ALSO point to the 'stdout' stream.  Since 'r_trace_file' and 'stdout' now point to the same place, the subsequent 'fprintf (r_trace_file,...)' should print on 'stdout'.

In this method, just omit the 'r_trace_file = stdout' and you should at least get something written to the file.

2)  This goes back to the earlier example.

  r_trace_file = freopen("junk.txt", io_type, stdout);

is equivalent to:

  fclose (stdout);
  r_trace_file = fopen ("junk.txt", io_type);

This CLOSES the 'stdout' stream.  It doesn't point it to 'r_trace_file'.  If any of the C library routines (or any of your modules) write to 'stdout' they are now using an invalid pointer and the results are unpredictable.

To have 'stdout' point to your file, you need:

  stdout = freopen ("junk.txt", io_type, stdout);

If you want two pointers to the same stream (file) then use:

  stdout = freopen ("junk.txt", io_type, stdout);
  r_trace_file = stdout;

And of course, you can always leave 'stdout' pointing to the display and write to your own file just by opening a new strean:

  r_trace_file = fopen ("junk.txt", io_type);

Thankfully, you don't have to reset 'stdout' to its original value.  The operating system resets 'stdout' and 'stderr' to the terminal device at each job step.  (Well, not exactly -- but that's a whole explanation by itself.)  If not, little things like the prompt would go to your redirected file until you ran a program that pointed 'stdout' back to the display.


Author Comment

ID: 8025202

Some improvement today.
I used setvbug along with freopen and the junk is gone
but the code is dumping core !

I added these lines of code extra and here it is.

I declared userdefined BUF using,

char *BUF = (char*)malloc(sizeof(1025));

///just before doing freopen i added memset.
//Since I suspect junk is in the file buffer just before doing freopen

                if ((rascal_trace_file = freopen(rfname, io_type, stdout)) == NULL) {
                 //debug statements
//added setvbuf immediately after freopen

 setvbuf ( rascal_trace_file,BUF,_IOFBF , 1024 );

It is coring in an unrelated code while Database fetch.

Please let me know if I am going wrong anywhere.
Also I am not sure why using userdefined BUF eliminated the junk chars.


LVL 45

Expert Comment

by:Kent Olsen
ID: 8025408

It looks like you've still got the same basic bug -- you're closing stdout and leaving a bad pointer in it.

In the last example that you sent, change:

rascal_trace_file = freopen (rfname, io_type, stdout)


rascal_trace_file = fopen (rtname, io_type)

and see what happens.


Author Comment

ID: 8041788

I finally fixed the problem with 1 line change by calling

setvbuf ( rascal_trace_file,NULL, _IOFBF , 1024*3 ); after
if ((rascal_trace_file = freopen(rfname, io_type, stdout)) == NULL) {
                //debug statements
While it was coring , I had declared a user defined buffer , here I am using system buffer but restricting the size to 2048.

But the strange thing is if I set the file buffer size to any value >= 1024*5 in setvbuf junk still appears!

What could be the reason???
Please let me know

Is this a good way to fix the problem???


LVL 45

Accepted Solution

Kent Olsen earned 80 total points
ID: 8041951

Offhand, I don't think that you've actually fixed the problem, just "masked" it.  As your code changes (and the binary changes size) this might again rear its ugly head.

I still think that you should open rascal_trace_file with fopen(), not freopen(,,stdout).

But I'm glad that it's doing better!
LVL 18

Expert Comment

ID: 9827038
No comment has been added lately, so it's time to clean up this TA.
I will leave a recommendation in the Cleanup topic area that this question is:

Answered by Kdo

Please leave any comments here within the next seven days.


EE Cleanup Volunteer

Featured Post


Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction: Finishing the grid – keyboard support for arrow keys to manoeuvre, entering the numbers.  The PreTranslateMessage function is to be used to intercept and respond to keyboard events. Continuing from the fourth article about sudoku. …
Introduction: Database storage, where is the exe actually on the disc? Playing a game selected randomly (how to generate random numbers).  Error trapping with try..catch to help the code run even if something goes wrong. Continuing from the seve…
This video will show you how to get GIT to work in Eclipse.   It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.
In this video, Percona Solution Engineer Dimitri Vanoverbeke discusses why you want to use at least three nodes in a database cluster. To discuss how Percona Consulting can help with your design and architecture needs for your database and infras…
Suggested Courses
Course of the Month14 days, 16 hours left to enroll

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question