?
Solved

C# ODBC DB CPU

Posted on 2014-02-07
6
Medium Priority
?
308 Views
Last Modified: 2014-02-26
Hi All,

We have a web application using WCF services hosted by a self-hosting Windows Service. For the moment we are using ODBC as our database connectivity and we have a question regarding CPU at the database level.

We are currently working on a large operation (importing hundreds of thousands of records) and we notice during this operation that the database (Oracle, SQL Server) begins using a large amount of CPU and Memory as well. We are less concerned about the memory growth but it is the CPU that we wonder if we are doing something incorrect to cause such a thrashing of the database. Let me explain the workflow:

- (Web Application) User selects data to import and passes this selection to our self-hosted WCF service. To increase download speeds we are sending data concurrently using a OneWay operation contract (chunks of 200 items).

- (Self-Hosted WCF service) Accepts the selected data from the Web Application, opens a database connection via ODBC, and begins processing the data for import.

During the import of the data we are keeping our transactions very small (only opening transactions at time of save and committing immediately).

Now, we are doing a lot of selects and there are a lot of SELECTS/UPDATES/INSERTS going on but how can we prevent such a thrashing of the database CPU?

I'm sure there are questions, please feel free to ask whatever you need to help us diagnose what the issue could be.
0
Comment
Question by:hmstechsupport
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
6 Comments
 
LVL 75

Expert Comment

by:käµfm³d 👽
ID: 39844100
I'm sure you knew this would come up, but I've got to ask:  Why use ODBC if there are better performing technologies available (e.g. ODAC, ADO.NET)?
0
 
LVL 143

Accepted Solution

by:
Guy Hengel [angelIII / a3] earned 2000 total points
ID: 39845260
actually, I "disagree" with above question, as the "database level" is asked about, and not the "communication level" ...

so, if "high cpu" is the issue, you should check exactly what queries ARE using high CPU, using the sql profiler tool.
once we pinpoint to 1 (or a couple of) queries using high CPU (and maybe also high I/O ...), you shall attempt to see why they are doing so.
it could be a missing index, a query that could be rewritten, no parameter binding, parallelized queries for actually small queries etc.

this may result in the application code to be adjusted, as not everything can be "tuned" by just "doing dba magic" on the database level.

though, changing from ODBC to other communication level can also improve, but usually the effort is much too high for the result, migrating a whole (and usually old) application from "old ODBC" to new stuff.
0
 

Author Comment

by:hmstechsupport
ID: 39847726
Ok to be fair we are using ADO.NET (SqlConnection, OracleConnection, etc...) with commands (SqlCommand, OracleCommand, etc...).

We are using parameterized insert and update statements but I would imagine these are freed with the command (we are wrapping the command with a using):

using (IDbCommand dbCommand = GetCommand(ProviderType))
{
PrepareCommand(dbCommand, Connection, Transaction, commandType, commandText, Parameters);

int returnValue = dbCommand.ExecuteNonQuery();
                    
return returnValue;
}

Open in new window


I'm not sure what this means in terms of parameter binding but it seems we will have to dig in with profiler and see what we find out. Will update as we know more.
0
Optimize your web performance

What's in the eBook?
- Full list of reasons for poor performance
- Ultimate measures to speed things up
- Primary web monitoring types
- KPIs you should be monitoring in order to increase your ROI

 
LVL 143

Expert Comment

by:Guy Hengel [angelIII / a3]
ID: 39849410
your code looks just fine in those regards.
so, I would really like to see the "results" of checking with the sql profiler
0
 

Author Closing Comment

by:hmstechsupport
ID: 39886712
It was indeed the application doing too many SQL operations.  We cached a bunch of data to reduce this and have completed this with acceptable performance.
0
 

Author Comment

by:hmstechsupport
ID: 39888612
Just to extend on the last comment. We were definitely making way to many SELECT statements on the database and this was causing some major thrashing at the database level and severely reduced the performance of the operation. As mentioned, we are now selecting all the needed data at the beginning of the operation and using that cache rather than constantly going into the database.
0

Featured Post

Get 15 Days FREE Full-Featured Trial

Benefit from a mission critical IT monitoring with Monitis Premium or get it FREE for your entry level monitoring needs.
-Over 200,000 users
-More than 300,000 websites monitored
-Used in 197 countries
-Recommended by 98% of users

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I have a large data set and a SSIS package. How can I load this file in multi threading?
In part one, we reviewed the prerequisites required for installing SQL Server vNext. In this part we will explore how to install Microsoft's SQL Server on Ubuntu 16.04.
Via a live example, show how to set up a backup for SQL Server using a Maintenance Plan and how to schedule the job into SQL Server Agent.
Via a live example, show how to setup several different housekeeping processes for a SQL Server.
Suggested Courses

801 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question