Solved

C# ODBC DB CPU

Posted on 2014-02-07
6
293 Views
Last Modified: 2014-02-26
Hi All,

We have a web application using WCF services hosted by a self-hosting Windows Service. For the moment we are using ODBC as our database connectivity and we have a question regarding CPU at the database level.

We are currently working on a large operation (importing hundreds of thousands of records) and we notice during this operation that the database (Oracle, SQL Server) begins using a large amount of CPU and Memory as well. We are less concerned about the memory growth but it is the CPU that we wonder if we are doing something incorrect to cause such a thrashing of the database. Let me explain the workflow:

- (Web Application) User selects data to import and passes this selection to our self-hosted WCF service. To increase download speeds we are sending data concurrently using a OneWay operation contract (chunks of 200 items).

- (Self-Hosted WCF service) Accepts the selected data from the Web Application, opens a database connection via ODBC, and begins processing the data for import.

During the import of the data we are keeping our transactions very small (only opening transactions at time of save and committing immediately).

Now, we are doing a lot of selects and there are a lot of SELECTS/UPDATES/INSERTS going on but how can we prevent such a thrashing of the database CPU?

I'm sure there are questions, please feel free to ask whatever you need to help us diagnose what the issue could be.
0
Comment
Question by:hmstechsupport
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
6 Comments
 
LVL 75

Expert Comment

by:käµfm³d 👽
ID: 39844100
I'm sure you knew this would come up, but I've got to ask:  Why use ODBC if there are better performing technologies available (e.g. ODAC, ADO.NET)?
0
 
LVL 143

Accepted Solution

by:
Guy Hengel [angelIII / a3] earned 500 total points
ID: 39845260
actually, I "disagree" with above question, as the "database level" is asked about, and not the "communication level" ...

so, if "high cpu" is the issue, you should check exactly what queries ARE using high CPU, using the sql profiler tool.
once we pinpoint to 1 (or a couple of) queries using high CPU (and maybe also high I/O ...), you shall attempt to see why they are doing so.
it could be a missing index, a query that could be rewritten, no parameter binding, parallelized queries for actually small queries etc.

this may result in the application code to be adjusted, as not everything can be "tuned" by just "doing dba magic" on the database level.

though, changing from ODBC to other communication level can also improve, but usually the effort is much too high for the result, migrating a whole (and usually old) application from "old ODBC" to new stuff.
0
 

Author Comment

by:hmstechsupport
ID: 39847726
Ok to be fair we are using ADO.NET (SqlConnection, OracleConnection, etc...) with commands (SqlCommand, OracleCommand, etc...).

We are using parameterized insert and update statements but I would imagine these are freed with the command (we are wrapping the command with a using):

using (IDbCommand dbCommand = GetCommand(ProviderType))
{
PrepareCommand(dbCommand, Connection, Transaction, commandType, commandText, Parameters);

int returnValue = dbCommand.ExecuteNonQuery();
                    
return returnValue;
}

Open in new window


I'm not sure what this means in terms of parameter binding but it seems we will have to dig in with profiler and see what we find out. Will update as we know more.
0
Comprehensive Backup Solutions for Microsoft

Acronis protects the complete Microsoft technology stack: Windows Server, Windows PC, laptop and Surface data; Microsoft business applications; Microsoft Hyper-V; Azure VMs; Microsoft Windows Server 2016; Microsoft Exchange 2016 and SQL Server 2016.

 
LVL 143

Expert Comment

by:Guy Hengel [angelIII / a3]
ID: 39849410
your code looks just fine in those regards.
so, I would really like to see the "results" of checking with the sql profiler
0
 

Author Closing Comment

by:hmstechsupport
ID: 39886712
It was indeed the application doing too many SQL operations.  We cached a bunch of data to reduce this and have completed this with acceptable performance.
0
 

Author Comment

by:hmstechsupport
ID: 39888612
Just to extend on the last comment. We were definitely making way to many SELECT statements on the database and this was causing some major thrashing at the database level and severely reduced the performance of the operation. As mentioned, we are now selecting all the needed data at the beginning of the operation and using that cache rather than constantly going into the database.
0

Featured Post

What is SQL Server and how does it work?

The purpose of this paper is to provide you background on SQL Server. It’s your self-study guide for learning fundamentals. It includes both the history of SQL and its technical basics. Concepts and definitions will form the solid foundation of your future DBA expertise.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article shows how to deploy dynamic backgrounds to computers depending on the aspect ratio of display
The article shows the basic steps of integrating an HTML theme template into an ASP.NET MVC project
This videos aims to give the viewer a basic demonstration of how a user can query current session information by using the SYS_CONTEXT function
Viewers will learn how to use the SELECT statement in SQL and will be exposed to the many uses the SELECT statement has.

730 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question