Solved

C# ODBC DB CPU

Posted on 2014-02-07
6
286 Views
Last Modified: 2014-02-26
Hi All,

We have a web application using WCF services hosted by a self-hosting Windows Service. For the moment we are using ODBC as our database connectivity and we have a question regarding CPU at the database level.

We are currently working on a large operation (importing hundreds of thousands of records) and we notice during this operation that the database (Oracle, SQL Server) begins using a large amount of CPU and Memory as well. We are less concerned about the memory growth but it is the CPU that we wonder if we are doing something incorrect to cause such a thrashing of the database. Let me explain the workflow:

- (Web Application) User selects data to import and passes this selection to our self-hosted WCF service. To increase download speeds we are sending data concurrently using a OneWay operation contract (chunks of 200 items).

- (Self-Hosted WCF service) Accepts the selected data from the Web Application, opens a database connection via ODBC, and begins processing the data for import.

During the import of the data we are keeping our transactions very small (only opening transactions at time of save and committing immediately).

Now, we are doing a lot of selects and there are a lot of SELECTS/UPDATES/INSERTS going on but how can we prevent such a thrashing of the database CPU?

I'm sure there are questions, please feel free to ask whatever you need to help us diagnose what the issue could be.
0
Comment
Question by:hmstechsupport
  • 3
  • 2
6 Comments
 
LVL 74

Expert Comment

by:käµfm³d 👽
Comment Utility
I'm sure you knew this would come up, but I've got to ask:  Why use ODBC if there are better performing technologies available (e.g. ODAC, ADO.NET)?
0
 
LVL 142

Accepted Solution

by:
Guy Hengel [angelIII / a3] earned 500 total points
Comment Utility
actually, I "disagree" with above question, as the "database level" is asked about, and not the "communication level" ...

so, if "high cpu" is the issue, you should check exactly what queries ARE using high CPU, using the sql profiler tool.
once we pinpoint to 1 (or a couple of) queries using high CPU (and maybe also high I/O ...), you shall attempt to see why they are doing so.
it could be a missing index, a query that could be rewritten, no parameter binding, parallelized queries for actually small queries etc.

this may result in the application code to be adjusted, as not everything can be "tuned" by just "doing dba magic" on the database level.

though, changing from ODBC to other communication level can also improve, but usually the effort is much too high for the result, migrating a whole (and usually old) application from "old ODBC" to new stuff.
0
 

Author Comment

by:hmstechsupport
Comment Utility
Ok to be fair we are using ADO.NET (SqlConnection, OracleConnection, etc...) with commands (SqlCommand, OracleCommand, etc...).

We are using parameterized insert and update statements but I would imagine these are freed with the command (we are wrapping the command with a using):

using (IDbCommand dbCommand = GetCommand(ProviderType))
{
PrepareCommand(dbCommand, Connection, Transaction, commandType, commandText, Parameters);

int returnValue = dbCommand.ExecuteNonQuery();
                    
return returnValue;
}

Open in new window


I'm not sure what this means in terms of parameter binding but it seems we will have to dig in with profiler and see what we find out. Will update as we know more.
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 142

Expert Comment

by:Guy Hengel [angelIII / a3]
Comment Utility
your code looks just fine in those regards.
so, I would really like to see the "results" of checking with the sql profiler
0
 

Author Closing Comment

by:hmstechsupport
Comment Utility
It was indeed the application doing too many SQL operations.  We cached a bunch of data to reduce this and have completed this with acceptable performance.
0
 

Author Comment

by:hmstechsupport
Comment Utility
Just to extend on the last comment. We were definitely making way to many SELECT statements on the database and this was causing some major thrashing at the database level and severely reduced the performance of the operation. As mentioned, we are now selecting all the needed data at the beginning of the operation and using that cache rather than constantly going into the database.
0

Featured Post

Free Gift Card with Acronis Backup Purchase!

Backup any data in any location: local and remote systems, physical and virtual servers, private and public clouds, Macs and PCs, tablets and mobile devices, & more! For limited time only, buy any Acronis backup products and get a FREE Amazon/Best Buy gift card worth up to $200!

Join & Write a Comment

Everyone has problem when going to load data into Data warehouse (EDW). They all need to confirm that data quality is good but they don't no how to proceed. Microsoft has provided new task within SSIS 2008 called "Data Profiler Task". It solve th…
Ever wondered why sometimes your SQL Server is slow or unresponsive with connections spiking up but by the time you go in, all is well? The following article will show you how to install and configure a SQL job that will send you email alerts includ…
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.
Via a live example combined with referencing Books Online, show some of the information that can be extracted from the Catalog Views in SQL Server.

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now