Solved

Retrieving logs takes forever

Posted on 2013-01-03
4
271 Views
Last Modified: 2013-01-13
Takes forever to pull logs.

The logging is based on msg_id, msg_ts, msg fields

It takes forever to retrieve logs from this table.

Explain plan suggests it does a full table scan on mylog table which contain log entries since Jul-2012, having 500,000 entries.

There are no partitions or index available on the table.

It is not possible to add partitions now as all programs in my project will have to be modified so that the partition key is also sent by mypkg.write_log(v_msg,v_date) everwhere.

Index on msg filed does not seem to be a proper way since there is no key field in it just text.

How can i pull the logs faster, what are my alternatives.
0
Comment
Question by:gram77
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
4 Comments
 
LVL 37

Expert Comment

by:Geert Gruwez
ID: 38739843
add an index based on your search criteria

> i assume there is a date column ?
0
 
LVL 35

Accepted Solution

by:
Mark Geerlings earned 500 total points
ID: 38740888
Queries from a table with 500,000 entries (records) and no indexes and no partitions will be **SLOW**!  If you can't (or won't) add an index or some kind of partitioning scheme, your only option to speed it up is to get some kind of solid-state storage device and move this table from the disk partition where (I'm guessing) it is now onto some kind of SSD.  There are multiple hardware options these days for SSD storage.

Most likely a cheaper alternative would be to add an index and/or a partitioning scheme of some kind to this table.  If it doesn't include a date column at all now, this could also include adding a date column with a name something like "date_created" and a trigger to populate the new date column.
0
 

Author Comment

by:gram77
ID: 38744354
Including a date column may not help since log writing can overflow a day, ie my application can start writing logs on 2-jan-13 11:30 PM and continue till 3-jan-13 1:30 AM
0
 

Author Comment

by:gram77
ID: 38772186
We have partition on DayOfMonth (1-30) and index on timestamp. selecting with these in the where clause make retrieval much faster.
0

Featured Post

Salesforce Has Never Been Easier

Improve and reinforce salesforce training & adoption using WalkMe's digital adoption platform. Start saving on costly employee training by creating fast intuitive Walk-Thrus for Salesforce. Claim your Free Account Now

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

How to Unravel a Tricky Query Introduction If you browse through the Oracle zones or any of the other database-related zones you'll come across some complicated solutions and sometimes you'll just have to wonder how anyone came up with them.  …
When it comes to protecting Oracle Database servers and systems, there are a ton of myths out there. Here are the most common.
This video explains at a high level about the four available data types in Oracle and how dates can be manipulated by the user to get data into and out of the database.
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.

728 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question