Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

C# ODBC DB CPU

Posted on 2014-02-07
6
Medium Priority
?
317 Views
Last Modified: 2014-02-26
Hi All,

We have a web application using WCF services hosted by a self-hosting Windows Service. For the moment we are using ODBC as our database connectivity and we have a question regarding CPU at the database level.

We are currently working on a large operation (importing hundreds of thousands of records) and we notice during this operation that the database (Oracle, SQL Server) begins using a large amount of CPU and Memory as well. We are less concerned about the memory growth but it is the CPU that we wonder if we are doing something incorrect to cause such a thrashing of the database. Let me explain the workflow:

- (Web Application) User selects data to import and passes this selection to our self-hosted WCF service. To increase download speeds we are sending data concurrently using a OneWay operation contract (chunks of 200 items).

- (Self-Hosted WCF service) Accepts the selected data from the Web Application, opens a database connection via ODBC, and begins processing the data for import.

During the import of the data we are keeping our transactions very small (only opening transactions at time of save and committing immediately).

Now, we are doing a lot of selects and there are a lot of SELECTS/UPDATES/INSERTS going on but how can we prevent such a thrashing of the database CPU?

I'm sure there are questions, please feel free to ask whatever you need to help us diagnose what the issue could be.
0
Comment
Question by:hmstechsupport
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
6 Comments
 
LVL 75

Expert Comment

by:käµfm³d 👽
ID: 39844100
I'm sure you knew this would come up, but I've got to ask:  Why use ODBC if there are better performing technologies available (e.g. ODAC, ADO.NET)?
0
 
LVL 143

Accepted Solution

by:
Guy Hengel [angelIII / a3] earned 2000 total points
ID: 39845260
actually, I "disagree" with above question, as the "database level" is asked about, and not the "communication level" ...

so, if "high cpu" is the issue, you should check exactly what queries ARE using high CPU, using the sql profiler tool.
once we pinpoint to 1 (or a couple of) queries using high CPU (and maybe also high I/O ...), you shall attempt to see why they are doing so.
it could be a missing index, a query that could be rewritten, no parameter binding, parallelized queries for actually small queries etc.

this may result in the application code to be adjusted, as not everything can be "tuned" by just "doing dba magic" on the database level.

though, changing from ODBC to other communication level can also improve, but usually the effort is much too high for the result, migrating a whole (and usually old) application from "old ODBC" to new stuff.
0
 

Author Comment

by:hmstechsupport
ID: 39847726
Ok to be fair we are using ADO.NET (SqlConnection, OracleConnection, etc...) with commands (SqlCommand, OracleCommand, etc...).

We are using parameterized insert and update statements but I would imagine these are freed with the command (we are wrapping the command with a using):

using (IDbCommand dbCommand = GetCommand(ProviderType))
{
PrepareCommand(dbCommand, Connection, Transaction, commandType, commandText, Parameters);

int returnValue = dbCommand.ExecuteNonQuery();
                    
return returnValue;
}

Open in new window


I'm not sure what this means in terms of parameter binding but it seems we will have to dig in with profiler and see what we find out. Will update as we know more.
0
Get your Conversational Ransomware Defense e‑book

This e-book gives you an insight into the ransomware threat and reviews the fundamentals of top-notch ransomware preparedness and recovery. To help you protect yourself and your organization. The initial infection may be inevitable, so the best protection is to be fully prepared.

 
LVL 143

Expert Comment

by:Guy Hengel [angelIII / a3]
ID: 39849410
your code looks just fine in those regards.
so, I would really like to see the "results" of checking with the sql profiler
0
 

Author Closing Comment

by:hmstechsupport
ID: 39886712
It was indeed the application doing too many SQL operations.  We cached a bunch of data to reduce this and have completed this with acceptable performance.
0
 

Author Comment

by:hmstechsupport
ID: 39888612
Just to extend on the last comment. We were definitely making way to many SELECT statements on the database and this was causing some major thrashing at the database level and severely reduced the performance of the operation. As mentioned, we are now selecting all the needed data at the beginning of the operation and using that cache rather than constantly going into the database.
0

Featured Post

Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I have a large data set and a SSIS package. How can I load this file in multi threading?
In part one, we reviewed the prerequisites required for installing SQL Server vNext. In this part we will explore how to install Microsoft's SQL Server on Ubuntu 16.04.
Familiarize people with the process of retrieving data from SQL Server using an Access pass-thru query. Microsoft Access is a very powerful client/server development tool. One of the ways that you can retrieve data from a SQL Server is by using a pa…
Using examples as well as descriptions, and references to Books Online, show the documentation available for datatypes, explain the available data types and show how data can be passed into and out of variables.

610 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question