Solved

Retrieving logs takes forever

Posted on 2013-01-03
4
266 Views
Last Modified: 2013-01-13
Takes forever to pull logs.

The logging is based on msg_id, msg_ts, msg fields

It takes forever to retrieve logs from this table.

Explain plan suggests it does a full table scan on mylog table which contain log entries since Jul-2012, having 500,000 entries.

There are no partitions or index available on the table.

It is not possible to add partitions now as all programs in my project will have to be modified so that the partition key is also sent by mypkg.write_log(v_msg,v_date) everwhere.

Index on msg filed does not seem to be a proper way since there is no key field in it just text.

How can i pull the logs faster, what are my alternatives.
0
Comment
Question by:gram77
  • 2
4 Comments
 
LVL 37

Expert Comment

by:Geert Gruwez
ID: 38739843
add an index based on your search criteria

> i assume there is a date column ?
0
 
LVL 35

Accepted Solution

by:
Mark Geerlings earned 500 total points
ID: 38740888
Queries from a table with 500,000 entries (records) and no indexes and no partitions will be **SLOW**!  If you can't (or won't) add an index or some kind of partitioning scheme, your only option to speed it up is to get some kind of solid-state storage device and move this table from the disk partition where (I'm guessing) it is now onto some kind of SSD.  There are multiple hardware options these days for SSD storage.

Most likely a cheaper alternative would be to add an index and/or a partitioning scheme of some kind to this table.  If it doesn't include a date column at all now, this could also include adding a date column with a name something like "date_created" and a trigger to populate the new date column.
0
 

Author Comment

by:gram77
ID: 38744354
Including a date column may not help since log writing can overflow a day, ie my application can start writing logs on 2-jan-13 11:30 PM and continue till 3-jan-13 1:30 AM
0
 

Author Comment

by:gram77
ID: 38772186
We have partition on DayOfMonth (1-30) and index on timestamp. selecting with these in the where clause make retrieval much faster.
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Article by: Swadhin
From the Oracle SQL Reference (http://download.oracle.com/docs/cd/B19306_01/server.102/b14200/queries006.htm) we are told that a join is a query that combines rows from two or more tables, views, or materialized views. This article provides a glimps…
Working with Network Access Control Lists in Oracle 11g (part 1) Part 2: http://www.e-e.com/A_9074.html So, you upgraded to a shiny new 11g database and all of a sudden every program that used UTL_MAIL, UTL_SMTP, UTL_TCP, UTL_HTTP or any oth…
This video shows information on the Oracle Data Dictionary, starting with the Oracle documentation, explaining the different types of Data Dictionary views available by group and permissions as well as giving examples on how to retrieve data from th…
Via a live example, show how to restore a database from backup after a simulated disk failure using RMAN.

932 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

9 Experts available now in Live!

Get 1:1 Help Now