loosing bytes when reading from a linux serial port ttyS0

Basically we have a read thread that does the following:

    for ( bool done = false ; !done ; ) {
        //
        // Under normal conditions this loop should have a read action
        // in progress at all times. The only time this won't be true
        // is when there is no room in the RX queue. We have a member
        // in class LinuxPort that defines whether or not a read is
        // presently active. This section of code just makes sure that
        // if no read is currently in progress, we do our best to get
        // one started.
        //
        int bytes_to_read = 0;
        unsigned char read_buffer[ 256 ];
        int iBytesRead;
        if ( !m_bInputThreadReading )
            {
            bytes_to_read = m_RxQueue.SpaceFree();
            if ( bytes_to_read > 256 )
                bytes_to_read = 256;
            //
            // If there is room to add new bytes to the RX queue, and
            // we currently aren't reading anything, we kick off the
            // read right here with a call to ReadFile(). There are two
            // possible things that can then happen. If there isn't any
            // data in the buffer, ReadFile() can return immediately
            // with the actual input in progress but not complete. If
            // there was enough data in the input stream already to
            // fulfill the read, it might return with data present.
            //
            if ( bytes_to_read > 0 )
                  {
//                        fcntl(m_hPort, F_SETFL, FNDELAY); // don't block serial read
                        iBytesRead = read( m_hPort, read_buffer, bytes_to_read );
                        if(iBytesRead < 0)
                        {
                              // The only acceptable error condition is the I/O
                              // pending error, which isn't really an error, it
                              // just means the read has been deferred and will
                              // be performed using overlapped I/O.
                              if(errno == EAGAIN)
                              {
                                    printf("SERIAL EAGAIN ERROR \n");
                              }
                              else
                              {
                                    printf("SERIAL read error %d %s\n",errno, strerror(errno));
                                    m_bInputThreadReading = true;
                              }
                        }
                        else
                        {    
                              // If we reach this point, ReadFile() returned
                              // immediately, presumably because it was able
                              // to fill the I/O request. I put all of the bytes
                              // just read into the RX queue, then call the
                              // notification routine that should alert the caller
                              // to the fact that some data has arrive.
                              if ( iBytesRead )
                              {
                                    m_RxQueue.Insert( (char*)read_buffer, iBytesRead );
                                    RxNotify( iBytesRead );
                              }
                        }
            }
        }
        //
        // We've completed the preliminary part of the loop and we are
        // now read to wait for something to happen. Note that it is
        // possible that either the call to ReadFile() or
        // WaitCommEvent() returned immediately, in which case we aren't
        // actively waiting for data of that event type. If that's true,
        // we have to go back through the loop and try to set up the
        // ReadFile() or WaitCommEvent() again. That's what this first
        // conditional statement is checking for. It would be a simpler
        // statement, but we have to take into account the possibility
        // that we aren't reading because there is no room in the
        // RX queue, in which case we can wait right away.
        if ( m_bInputThreadReading || bytes_to_read == 0 )
            {
                  AutoPtr<Notification> pNf(m_hKillInputThreadEvent.waitDequeueNotification(1));
                  if (pNf)
                  {
                        done = true;
                        break;
                  }
        }
    }

When I debug on Linux, basically there should be 10 byte record coming from the serial device.  First 8 bytes comes through.  When I look at bye 7, I expect to see a 4, instead I see another value.  After another read, I look at the first byte of that read, and it is fine.  What is causing the last value to be bad?

Any assistance would be appreciated.  I am new to linux, but have programmed in windows.
cenetadminAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

evilrixSenior Software Engineer (Avast)Commented:
Is m_bInputThreadReading  being used as a wait to ensure this bit of code is mutually exclusive between different threads that are being used to drain from the same port and wrie to the same queue? If so, is this type defined as volatile? If not any changes to its value are not guarenteed to be seen cross-thread (as the compiler can optimize this away) and therefore you'd have a potential race condition here.

I'm not saying that is your problem but I thought I'd mention it anyway.

-Rx.
0
evilrixSenior Software Engineer (Avast)Commented:
0
evilrixSenior Software Engineer (Avast)Commented:
Typo: "...being used as a wait to ensure..."
Should have said: "...being used as a way to ensure..."
0
Learn Ruby Fundamentals

This course will introduce you to Ruby, as well as teach you about classes, methods, variables, data structures, loops, enumerable methods, and finishing touches.

evilrixSenior Software Engineer (Avast)Commented:
>> I'm not saying that is your problem but I thought I'd mention it anyway
Oh, I'm not saying it isn't your problem either -- I'm suggesting you should check this as clearly, if there is a race condition, this could explain the behaviour you are seeing :)

-Rx.
0
cenetadminAuthor Commented:
The first time throgh my loop, I am getting EAGAIN.  What does this error mean?  Is this part of my problem?  Then the 2nd time through, on byte 7 of the read_buffer it is still coming back with a different value that first expected.

Do I need to do anything with setting the file to be binary instead of text?
0
evilrixSenior Software Engineer (Avast)Commented:
>> I am getting EAGAIN.  What does this error mean?
EAGAIN means you are performing an asynchronous read and there was no data to be read from the handle so try again. You need to keep retrying until you have all the data.
man 2 read : "EAGAIN Non-blocking I/O has been selected using O_NONBLOCK and no data was immediately available for reading."

>> Do I need to do anything with setting the file to be binary instead of text?
If you expect binary data, yes probably.

Did you understand my point about the potential race condition (assuming I@ve understood what m_bInputThreadReading is being used for)?
http://en.wikipedia.org/wiki/Race_condition


0
cenetadminAuthor Commented:
yes, I am sorry, I already changed the code to add the volitale statement.  I also agree, that a race condition probably exists, this is why I have only been looking in GDB at read_buffer.  I expect binary data, how do I set this up to recieve binary data?

Thanks

Eric
0
evilrixSenior Software Engineer (Avast)Commented:
>> a race condition probably exists
If the variable is volatile it should be ok; however, there is another issue if the data type (I'm guessing a bool?) can't be written or read in one CPU instruction. If it can't then this two can cause a potential race condition. As long as the type is a standard integer that is no greater than the standard word size of that platform (if it's a 32 bit platform the word size will be 32 bit) then this should be ok; however, I always use the type sig_atomic_t, which is a standard C++ type as it is guaranteed to be read and writable in one CPU instruction.

http://www.cplusplus.com/reference/clibrary/csignal/sig_atomic_t.html


>> I expect binary data, how do I set this up to recieve binary data?
You normally do this when you open the file or device. The method used will depend on how, exactly, you open the file or device. I suggest you look at how it's opened and then read the man page for that function. To read the man page for a function you type "man func_name", where func_name is the name of the function. Sometimes func_name may clash with another man page entry so you'll get the wrong one. Programming stuff is normally in section 2 (or sometimes 3), so you can specify the section explicitly thus, "man 2 func_name". For more information on how man pages work try "man man" to read the man page entry for man pages :)

-Rx.
0
evilrixSenior Software Engineer (Avast)Commented:
Can you clarify what m_bInputThreadReading is used for as there isn't enough context in the code posted to be sure. I'm not sure if it is significant to the problem you are having so I'd like to rule it out. If you can clarify what its purpose is, apart from terminating the read loop, that would be useful. Can it be set outside that loop by another thread whilst this thread is polling? Is it used to synchronize 2 or more threads polling (looking at it again I'm not sure it is) or maybe it's used to abort the polling from the main thread? Or maybe it's not being used x-thread at all and is only used as a control within this thread, in which case I think we can assume it's not part of this problem.
0
cenetadminAuthor Commented:
I removed the m_bInputThreadReading variable.  I then attempted to rerun the code and the same thing happened.
0
cenetadminAuthor Commented:
I am attempting to read binary data.  I created a test program that does not use threads. It exhibits the same behavior.  What do I need to add / modify to change this so that it will recieve all of the data?
int main(int argc, char *argv[])
{
 
	int m_hPort;
	m_hPort = open("/dev/ttyS0", O_RDWR | O_NOCTTY | O_NDELAY);
 
    if ( m_hPort > 0 )
	{
		for ( bool done = false ; !done ; )
		{
			int bytes_to_read = 256;
			unsigned char read_buffer[ 256 ];
			int iBytesRead = 256;
            if ( bytes_to_read > 0 )
			{
			    memset(read_buffer,'\0',256);
//				fcntl(m_hPort, F_SETFL, FNDELAY); // don't block serial read
				iBytesRead = read( m_hPort, read_buffer, bytes_to_read );
				if(iBytesRead < 0)
				{
					// The only acceptable error condition is the I/O
					// pending error, which isn't really an error, it
					// just means the read has been deferred and will
					// be performed using overlapped I/O.
					if(errno == EAGAIN)
					{
						printf("SERIAL EAGAIN ERROR \n");
					}
					else
					{
						printf("SERIAL read error %d %s\n",errno, strerror(errno));
						done = true;
					}
				}
				else
				{    
					// If we reach this point, ReadFile() returned 
					// immediately, presumably because it was able
					// to fill the I/O request. I put all of the bytes
					// just read into the RX queue, then call the 
					// notification routine that should alert the caller
					// to the fact that some data has arrive.
					if ( iBytesRead )
					{
//						m_RxQueue.Insert( (char*)read_buffer, iBytesRead );
						RxNotify( iBytesRead );
					}
				}
            }
		}
	}
 
	close(m_hPort);
    return 1;
}
 
void RxNotify(int i)
{
	return;
}

Open in new window

0
Duncan RoeSoftware DeveloperCommented:
If you want to read binary data from a serial port, you should first use tcsetattr() to put it in raw mode (dirst save the current settings using tcgetattr, to restore at exit (atexit())).
Do all serial input in one thread only. For a single port, this can hang on a blocked read, or you can use poll() or select() to wait for data. These calls would allow your thread to handle multiple ports and give you the option to wake up periodically even if there is no input. Non-blocked i/o is unnecessary (so no EAGAIN to cater for).
Handle EINTR (keep retrying the read until you get a different error or no error)

See "man termios" for the serial calls
0
cenetadminAuthor Commented:
I put in the raw mode call into my source file and compiled it.  Does linux have something that it adds to the end of a character string or is the last byte of buffer from a tty / RS232 port?  Any other suggestions?
0
Duncan RoeSoftware DeveloperCommented:
When reading data you get the number of bytes as in the return from read(). Nothing is added. If you want a NUL terminator, read a maximum of 1 less than the buffer capacity. Then you will always have room to add the terminator. But I would program differently - use the returned count in a memmove() for instance.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Linux OS Dev

From novice to tech pro — start learning today.