Question

Does C programming stdin standard input stream have a size limit of 512 characters and can it be increased?

Asked by: etjuseraskq

I have modified an existing C program to process data scanned in from a 2D barcode matrix. The device used to scan is a hand held RF device that runs Windows Mobile and Telnet Client systems. It uses these to connect into our dev system that runs under Linux (Suse) operating system.

I am using the getch command to get character input from stdin standard input stream. Everything works as expected for 2D barcodes that are less than about 520 but when I scan in a 2D barcode that is over about 500 or more my program will either hang or terminate.

It will definitely terminate if I scan in a 2D barcode that is over 900 characters. It appears that it starts hanging when scanning 2D barcodes between 520 and about 800 or so and then will alwalys terminate if the 2D barcode is over 800 characters. I have tried running the C program directly from my laptop thru the linux system and I can see that it puts out the error code 114 with the message attempting to access item beyond bounds of memory.

My feeling is that the stdin input stream used by C programs to get input data may have a limit of 512 characters and if over 512 characters then memory starts to get overwritten which causes the C program to hang or terminate.

Do you know if there is a size limit on the stdin standard input stream used by C. Since 2D barcodes have a maximum of 2000 characters that could be contained at one time with a 2D matrix. All of my 2D barcodes scan successfully that are 135, 212 and 486 in character length but I have one 2D barcode from a customer that is 954 in length and it does not scan in but terminates the program when it is scanned.

On our SUSE Linux system I have used the ulimit -a linux command to find out data about memory and pipe sizes. It indicates the pipe size is set at 512 and has an 8 to the right of it. I am told by the systems people that it indicates the Linux pipe size is 512 times 8 which is 4096 in size. If this is true then the pipe size appears large enough.

I am assuming the pipe size defined on a box is what stdin is associated with from the C programming view but don't know this for sure. I have also looked up on the Internet regarding increasing a pipe size and found something on the DECC$PIPE_BUFFER_SIZE command but don't know if this can be used to increase a pipe size.

Need to know if C's stdin has a limit of 512 characters that can be processed. If so is there a way to increase the stdin size to about 1024 and can this be done thru the C system calls. Since the existing program already uses the getch command to retrieve data I don't want to change the way it is doing it.

I am assuming this issue is happening because of the large amounts of data that can be read in via the 2D barcode.

Also, would like to know if there is a way to debug and step thru a C program that runs on the SUSE linux side. I have no problem debugging on the Windows side with Micro Focus or Visual C++ but have not found anything to debug with on our Linux side. i've been adding printf statements to help in the debug process in the meantime.

Your response to these two questions would be greatly appreciated.

Thank you,
Elvin Jordan

This Question has been solved and asker verified All Experts Exchange premium technology solutions are available to subscription members.

Subscribe now for full access to Experts Exchange and get

Instant Access to this Solution

  • Plus...
  • 30 Day FREE access, no risk, no obligation
  • Collaborate with the world's top tech experts
  • Unlimited access to our exclusive solution database
  • Never be left without tech help again

Subscribe Now

Trusted by hundreds of thousands everyday for fast, accurate and reliable tech support.

  • "The time we save is the biggest benefit of Experts Exchange to Warner Bros. What could take multiple guys 2 hours or more each to find is accessed in around 15 minutes on Experts Exchange." Mike Kapnisakis, Warner Bros.
  • "Our team likes having a resource that is more secure than just using Google and most experts using this service really know their stuff. It's nice to look here first versus using Google." Dayna Sellner, Lockheed Martin
  • "Anytime that I've been stumped with a problem, 9 out of 10 times Experts Exchange has either the accepted solution or an open discussion of the potential solution to the problem." Kenny Red, eBay Inc.

See what Experts Exchange can do for you.

Got a question?

We've got the answer.

Experts Exchange has been collecting answers to technology questions since 1996…3 million and counting! If you have a question, chances are we already have your answer.

Screenshot of Experts Exchange Knowledgebase

Need individual assistance?

Our experts are ready to help.

If you can't find the exact answer you're looking for, ask our exclusive community of 50,000 experts. You’ll get a personalized answer from a trusted professional.

Screenshot of Experts Exchange Knowledgebase

Want to learn from the best?

Read articles from industry experts.

Thousands of free tech tips, tricks, how-to’s and tutorials are available in our peer reviewed articles section. See for yourself how smart our experts are, no login required.

Screenshot of an Article

Working on a long term project?

Store your work and research.

Save solutions to your questions, answers you’ve discovered through searching plus helpful articles in your personal knowledgebase for easy future access.

Screenshot of Experts Exchange Knowledgebase

Access the answers to your technology questions today.

Subscribe Now

30-day free trial. Register in 60 seconds.

What Makes Experts Exchange Unique?

Members of the expert community talk about why the experience at Experts Exchange is different than what you will find anywhere else.

Trusted by the world's most respected brands.

image of each brand's logo

Faithfully serving IT professionals since 1996.

Experts Exchange Logo

Try it out and discover for yourself.

Subscribe Now

30-day free trial. Register in 60 seconds.

Related Solutions

  1. printf
    In my compiler package there is a directory called include where all the header files are included.. Isn't there supposed to be a source file for every header file where functions are defined? I can't find anywhere the source files... for example I would be interested in see...
  2. Printf and Scanf
    Can anybody tell me the difference between specific printf() and scanf() functions? For example, what are the difference between wsprintf() and sprintf()? I just need a list of detailed specifics on some of these functions. Thanks!
  3. fflush(stdin) does not work
    I am working on Linux, and this is my pgm: #include <stdio.h> void main( void ) { char string[81]; printf( "Enter a word: " ); scanf( "%5s", string ); printf( "%s\n", string ); fflush( stdin ); printf( "Enter the sa...
  4. redirect of stdout/stderr to stdin
    I need to execute a program and redirect its stdout/stderr to the stdin. Can someone please tell me how to do this or point me in the right direction.
  5. Recieve file from stdin
    Hi, I have a utility for which I would like to add the ability to redirect a file in. e.g. c:\myUtility <aBinFile.tif I did the following as my first stab. It takes the file in from standard in and writes it to a tmp file. tmpnam(buff); strcat(buff,"tif"); i...
  6. printf problem
    hi why this code is doing different....... main() { int a,b; printf("Enter two values a and b:"); scanf("%d%d",a,b); printf("%d+%d=%d",a,b,a+b); } although it is printing but printing different values e.g ur input is 2 and 3 then ...

Free Tech Articles

  1. WARNING: 5 Reasons why you should NEVER fix a computer for free.
    It is in our nature to love the puzzle. We are obsessed. The lot of us. We love puzzles. We love the challenge. We thrive on finding the answer. We hate disarray. It bothers us deep in our soul. W...
  2. SCCM OSD Basic troubleshooting
    SCCM 2007 OSD is a fantastic way to deploy operating systems, however, like most things SCCM issues can sometimes be difficult to resolve due to the sheer volume of logs to sift through and the dispe...
  3. Migrate Small Business Server 2003 to Exchange 2010 and Windows 2008 R2
    This guide is intended to provide step by step instructions on how to migrate from Small Business Server 2003 to Windows 2008 R2 with Exchange 2010. For this migration to work you will need the fo...
  4. Create a Win7 Gadget
    This article shows you how to create a simple "Gadget" -- a sort of mini-application supported by Windows 7 and Vista. Gadgets can be dropped anywhere on the desktop to provide instant information, ...
  5. Outlook continually prompting for username and password
    There have been a lot of questions recently regarding Outlook prompting for a username and password whilst using Exchange 2007. There are a few reasons why this would happen and I will try to cover t...
  6. Backup Exchange 2010 Information Store using Windows Backup
    There seems to be quite a lot of confusion around the ability to backup Exchange 2010 using the built in Windows Backup feature. This stems from the omission of this feature prior to Exchange 2007 s...

Cloud Class Webinars

  1. Avoiding Bugs in Microsoft Access
    Alison Balter takes and in-depth look at avoiding bugs in Access. In this webinar you will learn about using the immediate window to debug your applications, invoking the debugger, using breakpoints to troubleshoot, stepping through code, setting the next statement to execute, ...
  2. Top 10 Best New Features in Visio 2010
    Scott Helmers gives live demonstrations of the top 10 new features in Visio 2010. This webinar will teach you how to create compelling diagrams by adding shapes to the page with a single click, linking the shapes in a diagram to data in Excel (or SQL Server, or SharePoint), ...
  3. IT Consultant Business Secrets Revealed
    Michael Munger, Experts Exchange tech pro and IT consultant, pulls back the curtain on his very successful businesses and answers question on every IT consultant and business owner should know about. He shares secrets on what he did to solve the 5 most common problems in IT, ...
  4. Disaster Recovery and Business Continuity
    Quest CTO, Mike Billon, gives an overview of the steps involved in building a dunamic disaster recovery plan. Through case studies and an examination of software/hardware tooles for monitoring and testing, you'll gain a better understandin of where you are, where you want ...
  5. Organize Your Visio Diagrams with Containers and Lists
    Scott Helmers uses cross functional flowcharts, wireframe diagrams, data graphic legends and seating charts to teach you: how to ustilize all three new structured diagram components in Visio 2010, the best practices for organizeing shapes in previous version of Visio, how to organize ...
  6. How to Us Objects, Properties, Events and Methods in Microsoft Access
    Alison Dalter gives an in-depbth look at objects, properties, events and methods in Microsoft Access. In this webinar you will learn about using the object browser, referring to objects, working with properties and methods, working with object variables, understanding the ...

Join the Community

Give a Little. Get a Lot.

Join the community of experts here and help other tech pros by answering question in your area of expertise. You can earn FREE access to all Experts Exchange's premium features and resources.

Join the Community

Answers

 

by: etjuseraskqPosted on 2008-02-19 at 05:11:02ID: 20927899

Need to know if the standard input (stdin) stream size can be increased thru the C program. I suspect the limit is 512 characters max that can be read into at one time.

 

by: Infinity08Posted on 2008-02-19 at 09:03:54ID: 20930230

There shouldn't be a limit to the stream size (whether that's stdin or another). What you might want to check however, is where you store these characters ... Is there enough space to store them all ?

Try just code like this (without storing !) :

        int c = 0;
        do {
            c = getc(stdin);
            printf("%c", (char) c);
        } while (c != EOF);           /* <--- or another terminator value */

and see whether it has the same restriction.



Can you show the relevant code ?

 

by: evilrixPosted on 2008-02-19 at 09:53:18ID: 20930718

There is no limit to an input stream. There is; however, a limit to the size of the IO buffer, which can be drained quicker than it can be filled. Are you sure this isn't just an IO buffering/timing issue? Maybe you are draining the input buffer quicker than the scanner can feed it? Maybe you need to perform a retry or implement some handshaking mechanism?

 

by: etjuseraskqPosted on 2008-02-19 at 09:54:33ID: 20930735

When you ask where am I storing the characters....I'm doing simular to what you listed in your comment.
The below statements are what I'm using to retrieve from stdin:
int InputChar = ' ';
fflush(stdin);
InputChar = getch();
After I scan in a 2D barcode that is 954 length in characters the program terminates without ever returning to the next statement after the getch().
I get the error:
  error code: 114, pc=0, call=1, seg=0  Attempt to access item beyond bounds of memory (Signal 11).
It appears this occurs when I attempt to scan 2D barcodes that are over about 500 characters which is why my thinking is there may be a 512 maximum length on what stdin can handle at one time.
If stdin is associated with the pipe size of the operating system some googling on the internet returned a possible 512 maximum characters that a pipe could handle.

thanks,
etjuseraskq

 

by: evilrixPosted on 2008-02-19 at 09:55:36ID: 20930751

This article looks like it discusses issues of IO starvation.
http://www.pixelbeat.org/programming/stdio_buffering/

 

by: etjuseraskqPosted on 2008-02-19 at 10:00:07ID: 20930797

Regarding the response about the IO buffer being the problem.....I am using the getch() command to retrieve data from stdin and this command does not buffer the data but returns it immediately after the user presses each key.
int InputChar = ' ';
InputChar = getch();

The problem is occuring at scan time before control ever returns back from the getch().
Using a symbol hand held RF device (MC909x).

thanks,
etjuseraskq

 

by: evilrixPosted on 2008-02-19 at 10:09:44ID: 20930887

>> I am using the getch() command to retrieve data from stdin and this command does not buffer the data but returns it immediately after the user presses each key.
That's not what I mean. I am referring to the stdin stream buffer and the fact that it could be empty and if it is getch() can just return error depending upon the current mode of operation. Unfortunately, because getch() is not defined by the C standard it is not guaranteed to behave the same on all platforms; it bypasses C's IO system and talks direct to the OS.

http://opengroup.org/onlinepubs/007908775/xcurses/getch.html

getch()

RETURN VALUE
Upon successful completion getch(), mvgetch(), mvwgetch() and wgetch() return the single-byte character, KEY_ value, or ERR. When in the nodelay mode and no data is available, ERR is returned.

Do you see what I am getting at now?

BTW: have you tried running this with gdb or some other debugger to see what backtrace you get?

 

by: Infinity08Posted on 2008-02-19 at 11:21:00ID: 20931592

>> InputChar = getch();

Yes, but what do you do with InputChar after this line ?

 

by: etjuseraskqPosted on 2008-02-19 at 11:22:26ID: 20931607

So if I follow you correctly you are saying the possiblity exists that stdin could be empty when a getch command is performed? I am not currently checking the response for ERR after performing the
InputChar = getch(): The getch is contained within a while(1) loop and we breack out whenever the user hits PF3(exit) or someother special key. The loop checks the keypress returned against all PF keys,backspace, etc and takes an appropiate action. Not sure if I follow your reasoning as to why the stdin could be empty after scanning over 500+ characters as oppose to under 500 characters where there is no problem so don't know how the draining of the stdin is done. I currently do a fflush(stdin) before every getch within the While loop.
This logic has been in place and worked successfully at our company for 1D or linear barcodes so not sure if I want to use a different stdin retrieval command at this point since the problem appears to be related to only 2D barcode data over 500+ characters.
I'll try doing a test and checking for ERR during the getch call. Do not believe we are using nodelay mode since we don't want to force the user to press a key within any time limits so since we are not using nodelay...not sure if ERR is returned if stdin is empty as you stated.
Have not tried running the C program with gdb debugger since I'm fairly new to C and would have to find out how to get to it for the backtrace. The error message I stated in my question regarding going beyond bounds of memory....does that indicate anything to you?

thanks,
etjuseraskq

 

by: Infinity08Posted on 2008-02-19 at 11:26:20ID: 20931636

>> does that indicate anything to you?

That's why I would like to see what you actually do with the read characters.

 

by: etjuseraskqPosted on 2008-02-19 at 11:31:00ID: 20931672

In response to the question of what is done after the InputChar = getch() command.
The following code uses the Switch statement to examine the content of InputChar and perform the required action. There are cases for Enter, Up key, Down key, tab, backspace, etc.
If no PF key or other special keypress was done we take the character datra and put it into an array.
This array data is then passed back to the calling C program whick in turn call a cobol program based on the current form id that is being processed. When I have the problem I'm never getting passed the getch after the 2D data is scanned from the RF device...the program either hangs or terminates.

thanks,
etjuseraskq

 

by: evilrixPosted on 2008-02-19 at 11:31:18ID: 20931678

>> So if I follow you correctly you are saying the possiblity exists that stdin could be empty when a getch command is performed?
Yes it is possible, reads from streams can fail if the stream buffer is empty. Most of the C IO functions (all in fact as far as I recall) block if the buffer is empty but getch() may not.

>> reasoning as to why the stdin could be empty after scanning over 500+ characters as oppose to under 500 characters where there is no problem
Purely a timing issue (race condition) if you read quicker than the buffer fills you may eventually run out of data before scanner has finished sending it because it cannot send as quick as you read. I'm not saying thi is the problem, just suggesting it might be so you need to ensure you code to protect against this.

>> Have not tried running the C program with gdb debugger since I'm fairly new to C and would have to find out how to get to it for the backtrace.
This is likely to prove useful if we can't nail it down with conjecture :)

>> does that indicate anything to you?
Nothing specific to reading from a stream, no

 

by: Infinity08Posted on 2008-02-19 at 11:33:19ID: 20931698

>> If no PF key or other special keypress was done we take the character datra and put it into an array.

And are you sure that array is big enough to hold all data you put in it ?

 

by: Infinity08Posted on 2008-02-19 at 11:34:44ID: 20931709

>> >> If no PF key or other special keypress was done we take the character datra and put it into an array.
>>
>> And are you sure that array is big enough to hold all data you put in it ?

Or in other words ... Is it by any chance 500 characters (or 512 maybe) wide, while you're trying to put in 900 ?

 

by: etjuseraskqPosted on 2008-02-19 at 11:41:37ID: 20931789

The array I'm reading into is defined with a size of 3000 characters so over 900 should not be a problem.

thanks,
etjuseraskq

 

by: Infinity08Posted on 2008-02-19 at 11:45:34ID: 20931836

What happens if you put a printf inside the loop, and show each character as it's read ? At what point does it start showing the wrong value ?

 

by: etjuseraskqPosted on 2008-02-19 at 12:38:49ID: 20932342

Previously when I put in a printf within the loop it never makes it to the printf statement before the program hangs or terminates. I then tried putting the printf right after the getch statement and it never gets to the printf before the problem. This is why I originally believed the problem is happening at scan time after the console is paused from the getch. When the problem occurs it never makes it passed the getch statement.
Regarding checking for ERR...since I am not using the nodelay option I don't believe checking for ERR will work....after looking thru the internet it looks like the ERR check is done in conjuction with the nodelay option which I don't want to use for the program.
I am trying the ioctl command to check for input waiting before performing the getch. I was already doing this but only after it is determined in the program the data length read in is equal to or greater than the defined field length of the field defined on the form.
So there is currently the getch that normally retrieves data from stdin and then as part of the 2D barcode logic I added a special routine that contains a getch to get the rest of the data when a 2D barcode is scanned....I know when this occurs because i'm chekcing field length of characters retrieved from the 1st getch against what the defined field length is for the field currently being scanned on our RF form. This worked for getting the rest of the data when a 2D is scanned but I had to add the ioctl statement so the console (RF device) would not pause but only do the getch for 2D when there was input waiting.
Ex. int KeyIO;
     ioctl(0, FIONREAD, &KeyIO);
    if (KeyIO)
       then do a getch.....

i'm going to add this code prior to the original getch in the while loop and see what happens.

thanks,
etjuseraskq

 

by: Infinity08Posted on 2008-02-19 at 12:57:58ID: 20932505

I think the explanation can be found in the fact that getch is not a standard function, and there are some peculiarities with the non-line-based input on your system.

What if you make it line-based ? Ie. make use of standard functions like getc ?

 

by: evilrixPosted on 2008-02-19 at 13:12:41ID: 20932636

>> I think the explanation can be found in the fact that getch is not a standard function
Agreed, I eluded to this much earlier. The getch() function has no guaranteed way to behave as it is platform dependent

 

by: etjuseraskqPosted on 2008-02-19 at 13:25:47ID: 20932761

It is true that getch is not the standard but the current C program I've been modifying has used it without any issues over the last 10 years. At this point to change how it is doing the retrieval would change the way it has been working thus may introduce issues outside of the 2D modifications. What I don't want to do at this point is change the way the existing 1D and linear barcode data has been read in since that has worked o.k.
I'll try testing some different things or maybe system calls to see if there is a work around for now.

thanks,
etjuseraskq

 

by: PaulCaswellPosted on 2008-02-19 at 14:03:41ID: 20933103

Hi etjuseraskq,

Interesting. :)

My suggestions:

1. Get 'getch' out of yor face. Replace it with standard C 'fread ( &InputChar, 1, 1, stdin )'.

I agree with the others, getch is a potential killer. That should also keep I08 and ER happy. ;-)

2. Start investigating upstream of the data.

What drivers/software is being used to receive the scanner data and pipe it into your stdin? Could any of them be old and need updating?

It sounds to me like something further up the line cant handle more than 512 characters in a single bite. Get the latest drivers or libraries.

Less likely, but possible, could be your use of ioctl. By doing this you will call getch a specific length of time after your ioctl call returns success. This may produce a race condition.

As an afterthought, have you tried setvbuf (if it is available) to increase the buffer size of stdin? 2000 would be a convenient size. :)

Paul

 

by: etjuseraskqPosted on 2008-02-19 at 14:47:56ID: 20933391

Yes, I can see now that the getch is not a standard C stdin retrieval. However, at this point in our project we cannot afford to change the retrieval process for 2D without impacting the current 1D barcode retreivals that have used it successfully over the years. Maybe we will need a phase 2 of this project where we can try the other stdin retrieval commands.
The RF device we use is a MC909x symbol hand held device. It runs Windows Mobile and the Telnet client on it. The device is able to connect into our system where the C programs reside and run them from the Linux side. I will talk more with the vendor to determine if there are any drivers or settings that could be involved in this issue. The ioctl command is executed further down in the code after the getch and I'm sure it never even makes it to the firs ioctl before the program hangs or terminates. The scenario is the program is never returned to from the getch when we encounter this problem so there should not be any other commands that are affecting it.
Yes, I have looked at the setvbuf and will attempt to try some things with it to see if there is a difference.
Thanks for your input,
etjuseraskq

 

by: PaulCaswellPosted on 2008-02-19 at 15:01:02ID: 20933477

etjuseraskq,

>>... used it successfully over the years. Maybe we will need a phase 2 of this project where we can try the other stdin retrieval commands.

Perhaps the others can confirm but I believe

InputChar = getch();

should be EXACTLY the same as

fread ( &InputChar, 1, 1, stdin );

The only difference will be that if 'fread ( &InputChar, 1, 1, stdin )' doesnt work you can blame someone else. :-)

>> ... I will talk more with the vendor to determine if there are any drivers or settings that could be involved in this issue.

If you have had this system for years then it is VERY likely that the tools cannot handle 2000 byte barcodes. Good idea.

>>The ioctl command is executed further down in the code after the getch ...
I thought this was an unlikely one.

>>The scenario is the program is never returned to from the getch when we encounter this problem...
Which implies that the roblem is upstream. What appens if you DONT call getch and scan a long barcode?

>>... setvbuf and will attempt to try some things with it to see if there is a difference.
I hope it helps. It would be the best way to avoid upstream issues.

>>Thanks for your input,
Happy to help. :)

Paul

 

by: PaulCaswellPosted on 2008-02-19 at 15:07:17ID: 20933514

P.S.

The code that I08 suggested further up:

        int c = 0;
        do {
            c = getc(stdin);
            printf("%c", (char) c);
        } while (c != EOF);           /* <--- or another terminator value */

was a very good suggestion.

If it goes bang too then the problem is likely to be upstream. If it works, the problem is almost certainly downstream.

Paul

 

by: Infinity08Posted on 2008-02-20 at 01:43:50ID: 20936231

>> The code that I08 suggested further up:
>> was a very good suggestion.

I'd like to think so too ;)

Have you tried it, etjuseraskq ? Just to see what the result is ?

 

by: Computer101Posted on 2008-04-26 at 19:10:53ID: 21447768

Forced accept.

Computer101
EE Admin

20120131-EE-VQP-002

3 Ways to Join

30-Day Free Trial

The Experts

98% positive feedback on 31,087 answers since March 2000. angeliii is a Microsoft Most Valuable Professional for his work with MS SQL Server & Develoment.

He has also proven his knowledge of Visual Basic Programming, PHP Scripting and Oracle Databases.

The Experts

97% positive feedback on 10,752 answers since July 2000. lrmoore has more than 18 years experience in the networking industry.

The six-time Mircosoft MVPs specialties include firewalls, virtual private networking, and network management.

Testimonials

"...and excellent source for support... Kind of like having your very own IT dept." Electriciansnet

Testimonials

"I was apprehensive at signing up at first. However... it has already made my life as an IT administrator much easier." JaCrews

Testimonials

"WOW! You guys have great, active, and knowledgeable people on here." moore50

Business Clients

Business Clients

In the Press

"If you’ve got a question... Experts Exchange can supply an answer.”

In the Press

"...an invaluable aid for both IT professionals and those who require tech support."

In the Press

"where IT professionals provide quick answers on just about any topic"

Business Account Plans

Loading Advertisement...