Solved

Retrieving logs takes forever

Posted on 2013-01-03
4
265 Views
Last Modified: 2013-01-13
Takes forever to pull logs.

The logging is based on msg_id, msg_ts, msg fields

It takes forever to retrieve logs from this table.

Explain plan suggests it does a full table scan on mylog table which contain log entries since Jul-2012, having 500,000 entries.

There are no partitions or index available on the table.

It is not possible to add partitions now as all programs in my project will have to be modified so that the partition key is also sent by mypkg.write_log(v_msg,v_date) everwhere.

Index on msg filed does not seem to be a proper way since there is no key field in it just text.

How can i pull the logs faster, what are my alternatives.
0
Comment
Question by:gram77
  • 2
4 Comments
 
LVL 36

Expert Comment

by:Geert Gruwez
Comment Utility
add an index based on your search criteria

> i assume there is a date column ?
0
 
LVL 34

Accepted Solution

by:
Mark Geerlings earned 500 total points
Comment Utility
Queries from a table with 500,000 entries (records) and no indexes and no partitions will be **SLOW**!  If you can't (or won't) add an index or some kind of partitioning scheme, your only option to speed it up is to get some kind of solid-state storage device and move this table from the disk partition where (I'm guessing) it is now onto some kind of SSD.  There are multiple hardware options these days for SSD storage.

Most likely a cheaper alternative would be to add an index and/or a partitioning scheme of some kind to this table.  If it doesn't include a date column at all now, this could also include adding a date column with a name something like "date_created" and a trigger to populate the new date column.
0
 

Author Comment

by:gram77
Comment Utility
Including a date column may not help since log writing can overflow a day, ie my application can start writing logs on 2-jan-13 11:30 PM and continue till 3-jan-13 1:30 AM
0
 

Author Comment

by:gram77
Comment Utility
We have partition on DayOfMonth (1-30) and index on timestamp. selecting with these in the where clause make retrieval much faster.
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
Sql to get orphans 7 50
oracle 10G 5 46
Remove Hyphens in Oracle SQL 5 44
Export BLOB data from Oracle 10g 4 23
Background In several of the companies I have worked for, I noticed that corporate reporting is off loaded from the production database and done mainly on a clone database which needs to be kept up to date daily by various means, be it a logical…
Configuring and using Oracle Database Gateway for ODBC Introduction First, a brief summary of what a Database Gateway is.  A Gateway is a set of driver agents and configurations that allow an Oracle database to communicate with other platforms…
This video shows setup options and the basic steps and syntax for duplicating (cloning) a database from one instance to another. Examples are given for duplicating to the same machine and to different machines
This video shows how to copy an entire tablespace from one database to another database using Transportable Tablespace functionality.

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now