Solved

PSQL v9.1 - after a certain load, low performance with shared Btrieve files

Posted on 2008-10-28
7
787 Views
Last Modified: 2013-12-25
Let me explain shortly what our application is currently doing.

Our software is a voice mail system. People call in on a number where they can manage their mailbox, listen to messages, changing settings.
All these settings and the call flow is stored in Btrieve BTV files stored locally on a telecomnode. This system can handle more than 60 simultaneous calls.

Remark: We only use one single pervasive client-id per Telecom node, since its our application that connects to the BTV files trough the Btrieve API.
We open the btrieve files locally (on the same machine).  Pervasive Server v8 is installed.

Our application is also using record locking and transactional updates.
Another application is also using ODBC access from a .Net Application (not applicable for the moment).

****All this is working great at the moment.****


What we would like to setup.

In a production environment we want to set up
"  2 of our applications on 2 different machines (telecom nodes) connecting to the same shared BTrieve files.
"  These BTrieve files should be located on a Cluster environment.
"  1 telecom node should handle 120 simultaneous calls.

Telecoms nodes:
"  Windows server 2003 standard edition (SP2)
"  3GB ram

Cluster environment:
"  Windows server 2003 enterprise edition (SP2)
"  3GB ram
**************************

We already set up 1 telecom node connecting to the shared BTrieve files.
"  On the server (location of the BTrieve files) we have installed Pervasive Server v9.1, according to the Clustering documentation from Pervasive.
"  On the Telecom node, we have installed the Pervasive Client (available on the Pervasive Server CD-ROM).

All this is working functionally OK, but we have performance problems when we reach more than +/- 35 simultaneous calls (on 1 telecom node).
"  Up to 35 callers --> performance ok
"  >35 callers --> performance very slow

We have read about MEFS (multi engine file sharing), which would allow us to use the same application without modification but which is substituted by other technologies like Pervasive Workgroup Engine and Pervasive Gateway, unfortunately we didnt find any documentation about how to install/configure this.  


Do you have any suggestion how to set up this platform without performance issues?
client/server or server/server ???

Thanks in advance.
Pervasive-with-shared-Btrieve-fi.pdf
0
Comment
Question by:Artilium
  • 4
  • 3
7 Comments
 
LVL 28

Expert Comment

by:Bill Bach
ID: 22822437
I would first recommend patching your PSQLv9.1 setup to PSQLv9.5 (Service Pack 2), and then applying the latest 9.52 update.  Both patches can be found at Pervasive's web site.  

Second, you should review your memory configuration to see if you have adequate L1 and L2 cache, and decide if you want to use System Cache or not.

The last thing is to review your use of the ClientID field and threading.  If you run a number of clients through the same connection, then they will be single-threaded at the networking layer.  This is probably the biggest detractor from performance in what you are doing, and is why you didn't see a problem on a standalone machine.  You may find better parallelism by implementing a new process for each call with its own database connection.

To answer your original question, MEFS is an implementation detail from Btrieve 6.15, which was eliminated in the PSQL7 engine due to its severe performance limitations in multi-user environments.  It was ONLY suitable for small workgroup setups.  The Pervasive Workgroup Engine uses Single-Engine File Sharing (like the server) and may be an option for you to reduce costs, depending on your application design and the Pervasive licensing agreement, which I suggest that you read in detail.  The WGE tops out at 5 concurrent users, but from your description, you may only have 2.
0
 

Author Comment

by:Artilium
ID: 22830193
Hi,
we have enabled a 2nd workstation (installed Pervasive CLIENT° 9.52) and launched 30 calls on this one... so on the server (also pervasive 9.52 SERVER) we currently have 4 communication threads and 2 licences used of the 6... and in total 60 calls without performance loss...

if we launch 50 calls on one of the clients.... we notice that calls on THAT client have a performance loss;
the 2nd client has no performance loss.

Now our programmers told us they have 120 "threads" towards the btrieve client ... but with netstat we only see 1 unc and 1 btrieve pipe towards the server (hence 2 communication threads per workstation on the server)....

Is there something they have to code to have really multithreading connectivity with the btrieve client?
or is there a hidden registry setting for this to be enabled?

We can install a 3rd box...4th box... but the idea is that 1 box could handle the load of the 2 if one fails...
and it should be 120 simultaneous calls per box... not 30

but currently 30 seems to be the limit for each "btrieve" thread...
0
 
LVL 28

Expert Comment

by:Bill Bach
ID: 22832516
This is from a conversation with the original developer of the code itself.  It's a bit dated, but you get the idea:

>>>
1) Is the PSQL2000i NSL still single-threading requests to the server?
Yes.

2) Any chance that this will change soon? PSQL8? PSQL9?
It will some time, maybe in PSQL8 but no guarantees at this point.  BTW, it is not just that the app needs to be multi-threaded; it also needs to use different Btrieve clientIDs because the MKDE itself only allows one request per clientID to be active at any one time.

3) Do the ODBC drivers have the same single-outstanding-request limitation?
There can only be one outstanding request per database connection.  If the app opens multiple database connections and make separate requests on each connection on different threads I think the requests would be handled in parallel.  

4) What about the new SP3 OLEDB components?
Ultimately, the OLEDB components all wind up going through the MIF and thus through NSL for remote requests; therefore remote requests would be single threaded through NSL.
<<<


The most telling side is the Engine Monitor, which is reporting one connection from each computer, right?  This essentially indicates that the program is single-threading Btrieve requests through the communications layer.

A network analyzer might give you some more insight into what is happening.  With it, you could see the response times between the various Btrieve calls and replies, and you'll see the ClientID buried in the data block so that you can see the various threads at play.  Look specifically for database calls that take an inordinate amount of time (anything over a millisecond or so), which could indicate waiting for disk activity on the server.  

You MIGHT be able to address this by adjusting cache, keeping file sizes small, and other such things, but you'll eventually run into the same issue as you continue to scale -- single-threading at the network layer.

There is a page in the on-line manuals called "Btrieve Clients" that you may wish to review.  I believe, however, that you must actually spawn child processes, not just child threads, for this to work the way that you want.

0
6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

 

Author Comment

by:Artilium
ID: 22841713
okay our developper has changed the usage of "BTRV" into "BTRVID" whereas an extra parameter "clientid" is now presented. We have successfully launched upto 32 different clientid's from one application to the server; but when we want to use them we get a btrieve error status 3 telling us the files are not "open"...
when we launch 120x 1 clientid the problem does not represent itself... but we still have a limit of max 30 calls on 1 clientid :(

another thread (http://cs.pervasive.com/forums/p/10231/10231.aspx) explains to me that changing clientid will not matter hence files are locked to a certain clientid... which explains why other clientid's can not open the btv we need :(
0
 
LVL 28

Accepted Solution

by:
Bill Bach earned 500 total points
ID: 22842004
Files are never "locked" to a client ID, unless you specifically apply a lock on them.  Instead, each client ID is just like an independent connection from an independent computer or process.  This means that when you create the session, you must also open the database files inside EACH connection.  Each CID also gets its own record locks, and of course must remember to close the files when it is finished.

When you do all of this, the "Active Users" screen WILL show a different connection for each user, as it should.  However, the telltale sign will be the number of communications sessions, which is reported in the Pervasive Monitor under Communications Statistics.  This MUST show the proper number.  If it still shows 2, then your threads are all still sharing the same communications session, and you still won't be able to get past your limits.  
0
 

Author Comment

by:Artilium
ID: 22903250
    microkernel  -> active files  2  app x 4 clientid, 240+120 voice    Active microkernel files:  24  
 
    Selected file's handles :   8  
 
    Microkernel -> Active Users  
 
 
    Active Microkernel Users:  8  
 
    Selected user's handles:  2159  
 
    microkernel -> resource  usage  
 
 
    Files:  24  
 
    handles:  13240/22004  
 
    clients:  8/13  
 
    worker threads (MAX  1024)  1/9  
 
    licences in use (MAX  6)  2/2  
 
    transactions  0/0  
 
    locks/peak  1/4  
 
    Microkernel ->  communications  
 
 
    Total Requests :  1392907516  
 
    spx requests:  0  
 
    tcp/ip requests:  1392907516  
 
    netbios requests:  0  
 
    connection timeouts:  0  
 
    connection recoveries  0  
 
    communication thread current/peak (MAX 16)  3/9  
 
    total remote sessions  current/peak  9/10  
 
    SPX remote sessions current/peak  0/0  
 
    TCP/IP remote sessions current/peak  9/10  
 
    Netbios remote sessions current/peak  0/0  
 
   
    port 139 and/or 445  2  
 
    Port 3351  8  
 
         
   
0
 

Author Closing Comment

by:Artilium
ID: 31510657
Thnx for the tip...BTRVID is indeed a better solution than BTRV
0

Featured Post

6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

Join & Write a Comment

Suggested Solutions

This article describes some very basic things about SQL Server filegroups.
Entering a date in Microsoft Access can be tricky. A typo can cause month and day to be shuffled, entering the day only causes an error, as does entering, say, day 31 in June. This article shows how an inputmask supported by code can help the user a…
Video by: Steve
Using examples as well as descriptions, step through each of the common simple join types, explaining differences in syntax, differences in expected outputs and showing how the queries run along with the actual outputs based upon a simple set of dem…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now