?
Solved

Occasional memory read errors

Posted on 2011-05-09
22
Medium Priority
?
286 Views
Last Modified: 2012-05-11
We are getting an intermittent error when running our software that connects to our interbase database.  We have attempted to track the issue but cannot seem to find out a direct cause for the error.

It is occurring during a simple insert into a table, but is only occurring randomly roughly once every 75 or 100 inserts.  Sometimes it'll happen once and then again after another 5 or so inserts.  Sometimes it won't happen for a few 100 inserts.

We are running interbase server version WI-V4.1.0.194 and the clients are connecting via ODBC driver, INTERSOLV InterBase ODBC Driver version 3.11.01.00.  The clients are simply running an insert statement (INSERT INTO fooTBL (bar) VALUES ('test')

As best I can tell these errors are only occurring when inserting into this table.  Selects from this table appear to run fine as do inserts into other tables.

The error we are receiving is: Application popup: ibremote.exe - Application Error : The instruction at "0x100105ae" referenced memory at "0x00000010".  The memory could not be "read".

It appears that there are no specific clients that this is occurring to or any specific values on the insert that is causing it.  As best I can tell all connections are closed and disposed after being used.

Are there any additional ways to trap this error?  Can anything be done to either prevent it from occurring or avoid it?

If you need any additional information I can attempt to track that down
0
Comment
Question by:hatterson09
  • 11
  • 10
21 Comments
 
LVL 19

Expert Comment

by:NickUpson
ID: 35722769
good grief I would have bet long odds against any version 4.1 interbase installations still existing.

I would run, not walk, towards upgrading to either interbase 7.x or firebird (the free open-source spinoff) and upgrading your odbs as well.

but to deal with your problem, is there anything different about this one table, many columns or rows for example?

as to a workaround, write a stored proc that does the insert and call that instead of doing the insert, see if that avoids the issue
0
 

Author Comment

by:hatterson09
ID: 35722870
We're moving towards the upgrade now.  We'll actually be moving away from interbase altogether, but that is likely 6 months down the road and an upgrade to a new version of interbase isn't really an option.

These clients are .net clients but the rest of the stuff that ties to this interbase DB is built in Delphi 2 (probably didn't expect to hear that either haha) so we have to deal with the legacy stuff until we can fully replace it.

As best I can tell there is nothing truly unique about this table, It is an order table (stores all orders) it is about in the middle in terms of usage and size as far as the rest of the DB goes.  Both the transaction table and the line table (individual lines of an order) both get significantly more use, but we don't see issues on those.

I just noticed I didn't mention it in the original question, but this is only occurring at two of our sites, we have a couple dozen other sites running very similar versions of our client (as well as the exact server and ODBC versions) without issue.

I've actually just finished creating the stored proc now to see if this will at least trap the error for us.  The frustrating thing up until now has been that it does not return an error to the ODBC client, or even timeout (despite connectionTimeout being explicitly specified), it just sits there waiting forever.

I will deploy the stored proc change and see if that at least alleviates the issue.
0
 
LVL 19

Expert Comment

by:NickUpson
ID: 35723106
" The frustrating thing up until now has been that it does not return an error to the ODBC client, or even timeout (despite connectionTimeout being explicitly specified), it just sits there waiting forever."

aahh, this may be the smoking gun, check the transaction details of ALL clients at the 'bad' sites. I'd bet that there is one, or more, with transaction-wait
0
[Video] Create a Disruption-Free Workspace

Open offices have their challenges. And Sometimes, it's even hard to work at work. It's time to reclaim your office and create a disruption-free workspace. With the MB 660, you can:

-Increase Concentration
-Improve well-being
-Boost Productivity

 

Author Comment

by:hatterson09
ID: 35723143
It only seems to hang up that one client/connection though, after the message pops up, other clients can continue query and insert into the database.  I've even had a second client generate an identical command to the first that caused an error, which goes through without issue.

However, I will check next time the error comes up.  How do I pull this information from the DB/Server?
0
 
LVL 19

Expert Comment

by:NickUpson
ID: 35723731
its not in the server, its in the individual clients, odbc and/or tdatabase with your setup I'd imagine
0
 

Author Comment

by:hatterson09
ID: 35723777
Ahhh, I see where it is in the ODBC settings.  We can attempt to change that to see if it allows the clients to recover.  Although we'd still have the issue of the error popping up on the server, we could at least allow the clients to continue.
0
 
LVL 19

Expert Comment

by:NickUpson
ID: 35724062
if you set to no wait, you will get an immediate error when trying to change a record that someone else has changed but not committed (at the moment you get this infinite wait which should clear when the commit is done but there were bugs in that area). remember to look at the equivalent delphi settings
0
 

Author Comment

by:hatterson09
ID: 35756122
We moved the calls over to a stored proc.  The error did not go away

We also changed the ODBC setting to 0-NoWait however the error still occurred and the ODBC client was still hung and waiting for the error to be cleared before it would continue on in the code.

In addition we have now seen the error occurring during select statements as well.  This has been across several tables/columns and several clients.

Is it possible that something is corrupt with the interbase installation on the server?
0
 
LVL 19

Expert Comment

by:NickUpson
ID: 35757586
well it's certainly possible,

shutdown all clients, copy the database file and work with that copy

do a backup and restore (hint the command is gbak) and see if that gives any errors
0
 

Author Comment

by:hatterson09
ID: 35757640
Backup and restore has been done regularly on that database, without errors.  We actually removed a large amount of data from the database two evenings ago to see if database size had anything to do with it (DB was at about 550MB) and performed a backup and restore after that was done (after a file copy too actually)

Internal consistency checks return good.
0
 
LVL 19

Expert Comment

by:NickUpson
ID: 35759718
no errors from the backup I assume, does it work ok initially and then go wrong or is it wrong immediately

you can get this kind of problem, its rare, from faulty hardware - memory, NIC, etc

anything in the logfile (interbase home directory)
0
 

Accepted Solution

by:
hatterson09 earned 0 total points
ID: 35769095
It would go wrong anywhere from an hour after a backup and restore to half a day after a backup and restore before erroring.

Checked the interbase log and there was nothing at the time of the error however the log was continually (every second or two) hit with an entry
<SERVERNAME>  <DATETIME>
    WNET/wnet_error: ReadFile end-of-file errno = 109

On a better front, we re-reviewed the code area that the client would always get stuck around when this error occurred on the server and found that there was a time when two commands were run back to back on the database without the connection being closed in between.  We modified this to close the connection and immediately re-open the connection and have not received an error in the 2 days since then.  Last weekend during the same timeframe we received roughly 20 errors so it appears that it may be fixed.

However this still is puzzling me as this identical code has been running at a few other facilities for months without a single error.  In addition the code was simply something like this:

 
command.commandText = "insert foo into bar"
command.executeNonQuery()
command.commandText = "insert foo2 into bar2"
command.executeNonQuery()

Open in new window


I am unsure why this would be an issue, and even more unsure why it would only be at one of our several sites.
0
 
LVL 19

Expert Comment

by:NickUpson
ID: 35769655
how big is the database, there was a low (by modern stardards) size limitation,
0
 
LVL 19

Expert Comment

by:NickUpson
ID: 35769670
forget that I see you looked at size already

could the operating system be low on disk space in any partation
0
 

Author Comment

by:hatterson09
ID: 35769724
Server only has 1 partition, total size is 550ish GB and there's over 500GB free.

As a note, DB is now roughly 200MB (down from the 550MB mentioned earlier)
0
 
LVL 19

Expert Comment

by:NickUpson
ID: 35769802
there is this: http://edn.embarcadero.com/article/25327

which includes:

Solution:
WNET errors are NetBeui networking errors.  InterBase tried to perform some sort
of networking function using NetBeui and the underlying NetBeui stack returned
an error.  The error returned is in the form of an errno.
0
 

Author Comment

by:hatterson09
ID: 35770164
I looked that error up, it appears to be a broken pipe error.  I'm not sure what's causing it.

I also checked a couple other sites which are not receiving the popup errors and they are also receiving this WNET error.  So there's nothing unique in terms of those errors with this site.
0
 
LVL 19

Expert Comment

by:NickUpson
ID: 35807809
I'm forced to suggest an issue with hardware, most likely an NIC

can you do the same inserts and selects solely on the server in an attempt to nail it down
0
 

Author Comment

by:hatterson09
ID: 35809333
Unfortunately the way the setup is on the site it would be very tough to simulate behavior on the server.  In addition due to the fact that this is our only 32 bit server on site and Delphi 2.0 won't run 64 bit we can't flip it over to another server to test on there.

Since updating the code to include the extra connect and disconnect I do not believe we have received any of the errors.  I will confirm this on Monday morning, however it appears that we have been able to avoid the errors, which may be the best I can hope for at this point.
0
 

Author Comment

by:hatterson09
ID: 36207031
Sorry, completely forgot about this.  Closing it up now.  Thanks for the reminder
0
 

Author Closing Comment

by:hatterson09
ID: 36240607
Still unsure of *why* the errors occurred, but ensuring that a connection was opened and closed between every command did make them go away.

At this point, it's stable enough to ignore until our upgrade to real technology comes (within the year)
0

Featured Post

Become an Android App Developer

Ready to kick start your career in 2018? Learn how to build an Android app in January’s Course of the Month and open the door to new opportunities.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

How much do you know about the future of data centers? If you're like 50% of organizations, then it's probably not enough. Read on to get up to speed on this emerging field.
This is an update to some code that someone else posted on Experts Exchange. It is an alternate approach, I think a little easier to use, & makes sure that things like the Task Bar will update.
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…
With just a little bit of  SQL and VBA, many doors open to cool things like synchronize a list box to display data relevant to other information on a form.  If you have never written code or looked at an SQL statement before, no problem! ...  give i…
Suggested Courses

580 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question