We help IT Professionals succeed at work.
Get Started

Performance Tuning - Job taking too much I/O

bhagatali asked
Last Modified: 2008-09-19
Hi All,

The batch job that I am working with right now, is responsible for deleting certain records from a very large table (The table has 81+ million records). Once we are through with the job, almost 75% of this table would have been cleared. We have a couple of approaches in which we can achieve the final results in less than 4 hours. But, all these approaches involve dropping the actual table or creating a new temp table and then renaming the temp table.

At this point, we want to go back to basics and see if we can achieve the results in 4-5 hours by using a simple DELETE command. The current approach in this approach is:

- Fetch employee id's which meet the criteria in a block of 1717 records
- For every record fetched, perform a DELETE from the table.
- Commit at the end of the first block i.e. after processing 1717 records and then continue with the next block (this loop continues till end of table).

The problem that I am seeing is that the job is utilizing only 5% CPU, which means, the job is very I/O bound. Can any one get me started on how to get about improving the I/O in this job? Any articles on improving the I/O will also suffice.

Thanks for your time.

Watch Question
Data Warehouse / Database Architect
This problem has been solved!
Unlock 1 Answer and 8 Comments.
See Answer
Why Experts Exchange?

Experts Exchange always has the answer, or at the least points me in the correct direction! It is like having another employee that is extremely experienced.

Jim Murphy
Programmer at Smart IT Solutions

When asked, what has been your best career decision?

Deciding to stick with EE.

Mohamed Asif
Technical Department Head

Being involved with EE helped me to grow personally and professionally.

Carl Webster
CTP, Sr Infrastructure Consultant
Ask ANY Question

Connect with Certified Experts to gain insight and support on specific technology challenges including:

  • Troubleshooting
  • Research
  • Professional Opinions
Did You Know?

We've partnered with two important charities to provide clean water and computer science education to those who need it most. READ MORE