Solved

Retrieving logs takes forever

Posted on 2013-01-03
4
270 Views
Last Modified: 2013-01-13
Takes forever to pull logs.

The logging is based on msg_id, msg_ts, msg fields

It takes forever to retrieve logs from this table.

Explain plan suggests it does a full table scan on mylog table which contain log entries since Jul-2012, having 500,000 entries.

There are no partitions or index available on the table.

It is not possible to add partitions now as all programs in my project will have to be modified so that the partition key is also sent by mypkg.write_log(v_msg,v_date) everwhere.

Index on msg filed does not seem to be a proper way since there is no key field in it just text.

How can i pull the logs faster, what are my alternatives.
0
Comment
Question by:gram77
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
4 Comments
 
LVL 37

Expert Comment

by:Geert Gruwez
ID: 38739843
add an index based on your search criteria

> i assume there is a date column ?
0
 
LVL 35

Accepted Solution

by:
Mark Geerlings earned 500 total points
ID: 38740888
Queries from a table with 500,000 entries (records) and no indexes and no partitions will be **SLOW**!  If you can't (or won't) add an index or some kind of partitioning scheme, your only option to speed it up is to get some kind of solid-state storage device and move this table from the disk partition where (I'm guessing) it is now onto some kind of SSD.  There are multiple hardware options these days for SSD storage.

Most likely a cheaper alternative would be to add an index and/or a partitioning scheme of some kind to this table.  If it doesn't include a date column at all now, this could also include adding a date column with a name something like "date_created" and a trigger to populate the new date column.
0
 

Author Comment

by:gram77
ID: 38744354
Including a date column may not help since log writing can overflow a day, ie my application can start writing logs on 2-jan-13 11:30 PM and continue till 3-jan-13 1:30 AM
0
 

Author Comment

by:gram77
ID: 38772186
We have partition on DayOfMonth (1-30) and index on timestamp. selecting with these in the where clause make retrieval much faster.
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article started out as an Experts-Exchange question, which then grew into a quick tip to go along with an IOUG presentation for the Collaborate confernce and then later grew again into a full blown article with expanded functionality and legacy…
Configuring and using Oracle Database Gateway for ODBC Introduction First, a brief summary of what a Database Gateway is.  A Gateway is a set of driver agents and configurations that allow an Oracle database to communicate with other platforms…
This video shows how to Export data from an Oracle database using the Original Export Utility.  The corresponding Import utility, which works the same way is referenced, but not demonstrated.
Via a live example, show how to restore a database from backup after a simulated disk failure using RMAN.

734 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question