Solved

Crystal Reports - Very slow SQL command

Posted on 2013-01-29
12
1,052 Views
Last Modified: 2013-08-16
I have a SQL query that I have created within a Crystal Report (2008) but it runs very slowly, I know this is because I have written it to do a search within a field.

Here is the query:
(select (min("TAUDIT"."ENTERED")) from "TAUDIT"
where SUBSTR("TAUDIT"."ACTION",1,22) = 'Arrived event (stop 1)'
and SUBSTR("TAUDIT"."RECORD_REF",7,6) = "TJOBS"."REFERENCE")

I have tested various methods of getting the correct value for "TAUDIT"."ACTION" =  'Arrived event (stop 1)', none of them really slow the system down.

The problem is when SUBSTR("TAUDIT"."RECORD_REF",7,6) = "TJOBS"."REFERENCE" is run, it takes a really long time to calculate.

What I really wanted to do was create a query that would search for complete string, not just part of the substring.
TAUDIT.RECORD_REF is a String(25) & the values I would be looking for would be a concatenation of "tJobs " & the TJOBS.REFERENCE (Number), so a value would look something like "tJobs 257407".
However, I am struggling to write a query that uses the CAST or CONVERT to join the 2 data types so that it can be compared with TAUDIT.RECORD_REF (String(25)).

Anyone have any ideas on this matter, or have had similar problems?

Thanks
0
Comment
Question by:Bizatanium
  • 3
  • 3
  • 2
  • +1
12 Comments
 
LVL 100

Expert Comment

by:mlmcc
ID: 38833697
How about trying

Val(SUBSTR("TAUDIT"."RECORD_REF",7,6)) = "TJOBS"."REFERENCE")

or perhaps it should be


Tonumber(SUBSTR("TAUDIT"."RECORD_REF",7,6)) = "TJOBS"."REFERENCE")


The problem is the system is converting the number to text to do the comparison not the substring part.

mlmcc
0
 
LVL 34

Expert Comment

by:James0628
ID: 38834601
I take it that this is a SQL Expression?  At first I assumed that it was a CR Command that was being used as a main datasource for a report.

 What db are you using?  CAST and CONVERT are MS SQL functions, but SUBSTR is not (unless it has been added in the more recent versions).

 FWIW, I doubt that converting the number to text is the problem, as mlmcc suggested.  He could be right.  It just doesn't seem to me like that would really be slowing things down.  If you want to try that, the functions he used - Val and ToNumber - are both CR functions, which may or may not exist in your db.  Since you've apparently got CAST and CONVERT, you'd probably have to use one of those.

 If you want to concatenate "tJobs" and TJOBS.REFERENCE, what kind of field is REFERENCE?  Int or float or something else?

 In MS SQL, if it's int, it could be as simple as something like 'tJobs ' + CAST (TJOBS.REFERENCE as varchar (7)).  For example,

select 'tJobs ' + CAST (257407 as varchar (7))

  produces

tJobs 257407


 If you are using MS SQL and were trying to use "&" to concatenate the strings, maybe that was your problem.  & is a logical AND.  + is used to concatenate strings.

 James
0
 
LVL 1

Author Comment

by:Bizatanium
ID: 38834877
Thanks for the thoughts.

I have tried the suggestions put forward by mlmcc, but as you said James0628 these are Crystal functions & don't work in a SQL query.

So if there was any misunderstanding here is the situation:
Using Crystal Reports (2008)
Accessing an Oracle Database
Building a SQL Expression Field within Crystal
TJOBS.REFERENCE is an Integer
TAUDIT.RECORD_REF is a String(25)
0
 
LVL 100

Expert Comment

by:mlmcc
ID: 38837925
Sorry, by the syntax mainly the ( ) around the expression should have realized this is a SQL expression.

A SQL expression gets added to the query as a subquery so it is being run and analyzed for each record you retrieve.  Depending on how it is added, it may be run for all records in the database since it may run before the filtering is applied.

mlmcc
0
 
LVL 34

Expert Comment

by:James0628
ID: 38838816
mlmcc,

 Wasn't there something about SQL Expressions not working for Oracle?

 James
0
Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

 
LVL 26

Expert Comment

by:Kurt Reinhardt
ID: 38840234
mlmcc,

 Wasn't there something about SQL Expressions not working for Oracle?

 James

I've never been able to get a correlated SQL Expression to work against an Oracle database.  Uncorrelated?  Yes. Correlated, No - generates syntax errors.  I've seen other people say they've gotten them to work, but I've never been able to prove it, nor do I now exactly how they were connecting to the database, what version of the database, etc...

In this particular case, the field that's causing a problem is the one that's correlated to the actual detail field.  It's interesting that it's working slow, as opposed to failing, but it's still not working as expected, which validates my experiences.
0
 
LVL 26

Expert Comment

by:Kurt Reinhardt
ID: 38842490
Try this:

(select (min(TAUDIT.ENTERED)) from TAUDIT
where SUBSTR(TAUDIT.ACTION,1,22) = 'Arrived event (stop 1)'
and SUBSTR(TAUDIT.RECORD_REF,7,6) = "TJOBS"."REFERENCE")

Open in new window


I've removed the double quotes from around all database objects except for the field that should be correlated from the database.  I would recommend you actually delete "TJOBS"."REFERENCE" and then drag it back in from the field tree into the same exact spot so that it's automatically formatted how your data connection needs it.

Why did I remove the double quotes from the other database objects?

1)  They're unnecessary for general SQL and probably unnecessary for the data connection. If you were writing this query in TOAD, for example, you wouldn't need them.

2)  To avoid "confusing" the database engine.  When you drag a field in from the field tree, it's formatted exactly as the connection needs it.  If that happens to be double quotes and you've manually entered in double quotes for all other fields, the SQL Expression might be trying to query the tables you already have in the report instead of an actual external query.

Please note, as mentioned above, I've never gotten a correlated SQL Expression to work with Oracle.  That same query I've listed above would work perfectly fine against SQL Server or DB2, however.  Regardless, please try and let us know if it works.
0
 
LVL 1

Author Comment

by:Bizatanium
ID: 38882330
Thanks again for your comments.
I've had this on the back burner for a while as I've just got used to it taking up to 45 minutes to churn through a days data.
I've tried all of the above suggestions with fingers crossed, all to no avail. I get the same results, a query that pulls a months worth of data still takes up to 13 hours to run....less than ideal.

I had hoped that the ultimately I would be able to resolve the issue that I see being the main cause of the problem. The query searches through all the records returned for the given period for a substring (SUBSTR(TAUDIT.RECORD_REF,7,6) = "TJOBS"."REFERENCE"). Rather than matching TAUDIT.RECORD against some sort of concatenation of "TJobs " & TJOBS.REFERENCE.
0
 
LVL 34

Expert Comment

by:James0628
ID: 38883667
I haven't used Oracle, but it seems like if it's possible to concatenate a string and a numeric column that was converted to a string, it should be fairly simple.

 However, there are apparently problems when trying to use a SQL Expression with an Oracle db, so even if you got the concatenation to work, the SQL Expression may not work, or may still be quite slow.

 The real solution may be a "manual" query that reads the main report data and incorporates the SQL Expression sub-query.  The report would use that as the datasource, instead of the tables.  That will probably give you the best possible performance.  Perhaps a CR Command, or something like a stored procedure or view in the db.

 FWIW, you could get a start on the manual query by going to Database > "Show SQL Query" to see the query that CR is using, and copying that.  Actually, even if you don't want to try a manual query at this point, looking at the query that CR is using may be informative.

 James
0
 
LVL 1

Accepted Solution

by:
Bizatanium earned 0 total points
ID: 38927888
I've tried a number of things based on all your suggestions. None of them really improved the performance of the report.

It did however make me realise that without the ability to concatenate the expression I was looking for, meant that the SQL query was being run for every record, against every record in a table of 5 million+entries.

To get it working for the Daily version of my report I was able to use:
(select (min("TAUDIT"."ENTERED")) from "TAUDIT"
where ("TAUDIT"."ENTERED")between (sysdate-2) and (sysdate)
and SUBSTR("TAUDIT"."ACTION",1,22) = 'Arrived event (stop 1)'
and SUBSTR("TAUDIT"."RECORD_REF",7,6) = "TJOBS"."REFERENCE")

This way it looked through a much smaller set of data & returned the data in seconds. Obviously as I look at Weekly & Monthly reports the run time increases greatly. I'll try and give an idea of the times when I increase the ranges.

Thanks
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Hot fix for .Net Crystal Reports 10.2.3600.0 to fix problems with sub reports running on 64 bit operating systems ISSUE: Reports which contain subreports fail with error "Missing Parameter Value" DEPLOYMENT SERVER OS: Windows 2008 with 64 bi…
Hello everyone, Hope you find this as helpful as we did. We have on the company I work for an application built in Delphi V with Crystal Reports 8. We all know that Crystal & Delphi can be temperamental sometimes and the worst thing is, nearly…
This Micro Tutorial will give you a basic overview how to record your screen with Microsoft Expression Encoder. This program is still free and open for the public to download. This will be demonstrated using Microsoft Expression Encoder 4.
In this video I am going to show you how to back up and restore Office 365 mailboxes using CodeTwo Backup for Office 365. Learn more about the tool used in this video here: http://www.codetwo.com/backup-for-office-365/ (http://www.codetwo.com/ba…

861 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

22 Experts available now in Live!

Get 1:1 Help Now