Solved

C# ODBC DB CPU

Posted on 2014-02-07
6
290 Views
Last Modified: 2014-02-26
Hi All,

We have a web application using WCF services hosted by a self-hosting Windows Service. For the moment we are using ODBC as our database connectivity and we have a question regarding CPU at the database level.

We are currently working on a large operation (importing hundreds of thousands of records) and we notice during this operation that the database (Oracle, SQL Server) begins using a large amount of CPU and Memory as well. We are less concerned about the memory growth but it is the CPU that we wonder if we are doing something incorrect to cause such a thrashing of the database. Let me explain the workflow:

- (Web Application) User selects data to import and passes this selection to our self-hosted WCF service. To increase download speeds we are sending data concurrently using a OneWay operation contract (chunks of 200 items).

- (Self-Hosted WCF service) Accepts the selected data from the Web Application, opens a database connection via ODBC, and begins processing the data for import.

During the import of the data we are keeping our transactions very small (only opening transactions at time of save and committing immediately).

Now, we are doing a lot of selects and there are a lot of SELECTS/UPDATES/INSERTS going on but how can we prevent such a thrashing of the database CPU?

I'm sure there are questions, please feel free to ask whatever you need to help us diagnose what the issue could be.
0
Comment
Question by:hmstechsupport
  • 3
  • 2
6 Comments
 
LVL 75

Expert Comment

by:käµfm³d 👽
ID: 39844100
I'm sure you knew this would come up, but I've got to ask:  Why use ODBC if there are better performing technologies available (e.g. ODAC, ADO.NET)?
0
 
LVL 142

Accepted Solution

by:
Guy Hengel [angelIII / a3] earned 500 total points
ID: 39845260
actually, I "disagree" with above question, as the "database level" is asked about, and not the "communication level" ...

so, if "high cpu" is the issue, you should check exactly what queries ARE using high CPU, using the sql profiler tool.
once we pinpoint to 1 (or a couple of) queries using high CPU (and maybe also high I/O ...), you shall attempt to see why they are doing so.
it could be a missing index, a query that could be rewritten, no parameter binding, parallelized queries for actually small queries etc.

this may result in the application code to be adjusted, as not everything can be "tuned" by just "doing dba magic" on the database level.

though, changing from ODBC to other communication level can also improve, but usually the effort is much too high for the result, migrating a whole (and usually old) application from "old ODBC" to new stuff.
0
 

Author Comment

by:hmstechsupport
ID: 39847726
Ok to be fair we are using ADO.NET (SqlConnection, OracleConnection, etc...) with commands (SqlCommand, OracleCommand, etc...).

We are using parameterized insert and update statements but I would imagine these are freed with the command (we are wrapping the command with a using):

using (IDbCommand dbCommand = GetCommand(ProviderType))
{
PrepareCommand(dbCommand, Connection, Transaction, commandType, commandText, Parameters);

int returnValue = dbCommand.ExecuteNonQuery();
                    
return returnValue;
}

Open in new window


I'm not sure what this means in terms of parameter binding but it seems we will have to dig in with profiler and see what we find out. Will update as we know more.
0
Optimizing Cloud Backup for Low Bandwidth

With cloud storage prices going down a growing number of SMBs start to use it for backup storage. Unfortunately, business data volume rarely fits the average Internet speed. This article provides an overview of main Internet speed challenges and reveals backup best practices.

 
LVL 142

Expert Comment

by:Guy Hengel [angelIII / a3]
ID: 39849410
your code looks just fine in those regards.
so, I would really like to see the "results" of checking with the sql profiler
0
 

Author Closing Comment

by:hmstechsupport
ID: 39886712
It was indeed the application doing too many SQL operations.  We cached a bunch of data to reduce this and have completed this with acceptable performance.
0
 

Author Comment

by:hmstechsupport
ID: 39888612
Just to extend on the last comment. We were definitely making way to many SELECT statements on the database and this was causing some major thrashing at the database level and severely reduced the performance of the operation. As mentioned, we are now selecting all the needed data at the beginning of the operation and using that cache rather than constantly going into the database.
0

Featured Post

Optimizing Cloud Backup for Low Bandwidth

With cloud storage prices going down a growing number of SMBs start to use it for backup storage. Unfortunately, business data volume rarely fits the average Internet speed. This article provides an overview of main Internet speed challenges and reveals backup best practices.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Why is this different from all of the other step by step guides?  Because I make a living as a DBA and not as a writer and I lived through this experience. Defining the name: When I talk to people they say different names on this subject stuff l…
Ever wondered why sometimes your SQL Server is slow or unresponsive with connections spiking up but by the time you go in, all is well? The following article will show you how to install and configure a SQL job that will send you email alerts includ…
Using examples as well as descriptions, and references to Books Online, show the different Recovery Models available in SQL Server and explain, as well as show how full, differential and transaction log backups are performed
Via a live example, show how to backup a database, simulate a failure backup the tail of the database transaction log and perform the restore.

816 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now