Solved

Retrieving logs takes forever

Posted on 2013-01-03
4
272 Views
Last Modified: 2013-01-13
Takes forever to pull logs.

The logging is based on msg_id, msg_ts, msg fields

It takes forever to retrieve logs from this table.

Explain plan suggests it does a full table scan on mylog table which contain log entries since Jul-2012, having 500,000 entries.

There are no partitions or index available on the table.

It is not possible to add partitions now as all programs in my project will have to be modified so that the partition key is also sent by mypkg.write_log(v_msg,v_date) everwhere.

Index on msg filed does not seem to be a proper way since there is no key field in it just text.

How can i pull the logs faster, what are my alternatives.
0
Comment
Question by:gram77
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
4 Comments
 
LVL 38

Expert Comment

by:Geert Gruwez
ID: 38739843
add an index based on your search criteria

> i assume there is a date column ?
0
 
LVL 35

Accepted Solution

by:
Mark Geerlings earned 500 total points
ID: 38740888
Queries from a table with 500,000 entries (records) and no indexes and no partitions will be **SLOW**!  If you can't (or won't) add an index or some kind of partitioning scheme, your only option to speed it up is to get some kind of solid-state storage device and move this table from the disk partition where (I'm guessing) it is now onto some kind of SSD.  There are multiple hardware options these days for SSD storage.

Most likely a cheaper alternative would be to add an index and/or a partitioning scheme of some kind to this table.  If it doesn't include a date column at all now, this could also include adding a date column with a name something like "date_created" and a trigger to populate the new date column.
0
 

Author Comment

by:gram77
ID: 38744354
Including a date column may not help since log writing can overflow a day, ie my application can start writing logs on 2-jan-13 11:30 PM and continue till 3-jan-13 1:30 AM
0
 

Author Comment

by:gram77
ID: 38772186
We have partition on DayOfMonth (1-30) and index on timestamp. selecting with these in the where clause make retrieval much faster.
0

Featured Post

[Live Webinar] The Cloud Skills Gap

As Cloud technologies come of age, business leaders grapple with the impact it has on their team's skills and the gap associated with the use of a cloud platform.

Join experts from 451 Research and Concerto Cloud Services on July 27th where we will examine fact and fiction.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article started out as an Experts-Exchange question, which then grew into a quick tip to go along with an IOUG presentation for the Collaborate confernce and then later grew again into a full blown article with expanded functionality and legacy…
Using SQL Scripts we can save all the SQL queries as files that we use very frequently on our database later point of time. This is one of the feature present under SQL Workshop in Oracle Application Express.
This video explains at a high level about the four available data types in Oracle and how dates can be manipulated by the user to get data into and out of the database.
This video shows information on the Oracle Data Dictionary, starting with the Oracle documentation, explaining the different types of Data Dictionary views available by group and permissions as well as giving examples on how to retrieve data from th…
Suggested Courses

632 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question