Solved

Crystal Reports - Very slow SQL command

Posted on 2013-01-29
12
1,016 Views
Last Modified: 2013-08-16
I have a SQL query that I have created within a Crystal Report (2008) but it runs very slowly, I know this is because I have written it to do a search within a field.

Here is the query:
(select (min("TAUDIT"."ENTERED")) from "TAUDIT"
where SUBSTR("TAUDIT"."ACTION",1,22) = 'Arrived event (stop 1)'
and SUBSTR("TAUDIT"."RECORD_REF",7,6) = "TJOBS"."REFERENCE")

I have tested various methods of getting the correct value for "TAUDIT"."ACTION" =  'Arrived event (stop 1)', none of them really slow the system down.

The problem is when SUBSTR("TAUDIT"."RECORD_REF",7,6) = "TJOBS"."REFERENCE" is run, it takes a really long time to calculate.

What I really wanted to do was create a query that would search for complete string, not just part of the substring.
TAUDIT.RECORD_REF is a String(25) & the values I would be looking for would be a concatenation of "tJobs " & the TJOBS.REFERENCE (Number), so a value would look something like "tJobs 257407".
However, I am struggling to write a query that uses the CAST or CONVERT to join the 2 data types so that it can be compared with TAUDIT.RECORD_REF (String(25)).

Anyone have any ideas on this matter, or have had similar problems?

Thanks
0
Comment
Question by:Bizatanium
  • 3
  • 3
  • 2
  • +1
12 Comments
 
LVL 100

Expert Comment

by:mlmcc
ID: 38833697
How about trying

Val(SUBSTR("TAUDIT"."RECORD_REF",7,6)) = "TJOBS"."REFERENCE")

or perhaps it should be


Tonumber(SUBSTR("TAUDIT"."RECORD_REF",7,6)) = "TJOBS"."REFERENCE")


The problem is the system is converting the number to text to do the comparison not the substring part.

mlmcc
0
 
LVL 34

Expert Comment

by:James0628
ID: 38834601
I take it that this is a SQL Expression?  At first I assumed that it was a CR Command that was being used as a main datasource for a report.

 What db are you using?  CAST and CONVERT are MS SQL functions, but SUBSTR is not (unless it has been added in the more recent versions).

 FWIW, I doubt that converting the number to text is the problem, as mlmcc suggested.  He could be right.  It just doesn't seem to me like that would really be slowing things down.  If you want to try that, the functions he used - Val and ToNumber - are both CR functions, which may or may not exist in your db.  Since you've apparently got CAST and CONVERT, you'd probably have to use one of those.

 If you want to concatenate "tJobs" and TJOBS.REFERENCE, what kind of field is REFERENCE?  Int or float or something else?

 In MS SQL, if it's int, it could be as simple as something like 'tJobs ' + CAST (TJOBS.REFERENCE as varchar (7)).  For example,

select 'tJobs ' + CAST (257407 as varchar (7))

  produces

tJobs 257407


 If you are using MS SQL and were trying to use "&" to concatenate the strings, maybe that was your problem.  & is a logical AND.  + is used to concatenate strings.

 James
0
 
LVL 1

Author Comment

by:Bizatanium
ID: 38834877
Thanks for the thoughts.

I have tried the suggestions put forward by mlmcc, but as you said James0628 these are Crystal functions & don't work in a SQL query.

So if there was any misunderstanding here is the situation:
Using Crystal Reports (2008)
Accessing an Oracle Database
Building a SQL Expression Field within Crystal
TJOBS.REFERENCE is an Integer
TAUDIT.RECORD_REF is a String(25)
0
 
LVL 100

Expert Comment

by:mlmcc
ID: 38837925
Sorry, by the syntax mainly the ( ) around the expression should have realized this is a SQL expression.

A SQL expression gets added to the query as a subquery so it is being run and analyzed for each record you retrieve.  Depending on how it is added, it may be run for all records in the database since it may run before the filtering is applied.

mlmcc
0
 
LVL 34

Expert Comment

by:James0628
ID: 38838816
mlmcc,

 Wasn't there something about SQL Expressions not working for Oracle?

 James
0
What Security Threats Are You Missing?

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 
LVL 26

Expert Comment

by:Kurt Reinhardt
ID: 38840234
mlmcc,

 Wasn't there something about SQL Expressions not working for Oracle?

 James

I've never been able to get a correlated SQL Expression to work against an Oracle database.  Uncorrelated?  Yes. Correlated, No - generates syntax errors.  I've seen other people say they've gotten them to work, but I've never been able to prove it, nor do I now exactly how they were connecting to the database, what version of the database, etc...

In this particular case, the field that's causing a problem is the one that's correlated to the actual detail field.  It's interesting that it's working slow, as opposed to failing, but it's still not working as expected, which validates my experiences.
0
 
LVL 26

Expert Comment

by:Kurt Reinhardt
ID: 38842490
Try this:

(select (min(TAUDIT.ENTERED)) from TAUDIT
where SUBSTR(TAUDIT.ACTION,1,22) = 'Arrived event (stop 1)'
and SUBSTR(TAUDIT.RECORD_REF,7,6) = "TJOBS"."REFERENCE")

Open in new window


I've removed the double quotes from around all database objects except for the field that should be correlated from the database.  I would recommend you actually delete "TJOBS"."REFERENCE" and then drag it back in from the field tree into the same exact spot so that it's automatically formatted how your data connection needs it.

Why did I remove the double quotes from the other database objects?

1)  They're unnecessary for general SQL and probably unnecessary for the data connection. If you were writing this query in TOAD, for example, you wouldn't need them.

2)  To avoid "confusing" the database engine.  When you drag a field in from the field tree, it's formatted exactly as the connection needs it.  If that happens to be double quotes and you've manually entered in double quotes for all other fields, the SQL Expression might be trying to query the tables you already have in the report instead of an actual external query.

Please note, as mentioned above, I've never gotten a correlated SQL Expression to work with Oracle.  That same query I've listed above would work perfectly fine against SQL Server or DB2, however.  Regardless, please try and let us know if it works.
0
 
LVL 1

Author Comment

by:Bizatanium
ID: 38882330
Thanks again for your comments.
I've had this on the back burner for a while as I've just got used to it taking up to 45 minutes to churn through a days data.
I've tried all of the above suggestions with fingers crossed, all to no avail. I get the same results, a query that pulls a months worth of data still takes up to 13 hours to run....less than ideal.

I had hoped that the ultimately I would be able to resolve the issue that I see being the main cause of the problem. The query searches through all the records returned for the given period for a substring (SUBSTR(TAUDIT.RECORD_REF,7,6) = "TJOBS"."REFERENCE"). Rather than matching TAUDIT.RECORD against some sort of concatenation of "TJobs " & TJOBS.REFERENCE.
0
 
LVL 34

Expert Comment

by:James0628
ID: 38883667
I haven't used Oracle, but it seems like if it's possible to concatenate a string and a numeric column that was converted to a string, it should be fairly simple.

 However, there are apparently problems when trying to use a SQL Expression with an Oracle db, so even if you got the concatenation to work, the SQL Expression may not work, or may still be quite slow.

 The real solution may be a "manual" query that reads the main report data and incorporates the SQL Expression sub-query.  The report would use that as the datasource, instead of the tables.  That will probably give you the best possible performance.  Perhaps a CR Command, or something like a stored procedure or view in the db.

 FWIW, you could get a start on the manual query by going to Database > "Show SQL Query" to see the query that CR is using, and copying that.  Actually, even if you don't want to try a manual query at this point, looking at the query that CR is using may be informative.

 James
0
 
LVL 1

Accepted Solution

by:
Bizatanium earned 0 total points
ID: 38927888
I've tried a number of things based on all your suggestions. None of them really improved the performance of the report.

It did however make me realise that without the ability to concatenate the expression I was looking for, meant that the SQL query was being run for every record, against every record in a table of 5 million+entries.

To get it working for the Daily version of my report I was able to use:
(select (min("TAUDIT"."ENTERED")) from "TAUDIT"
where ("TAUDIT"."ENTERED")between (sysdate-2) and (sysdate)
and SUBSTR("TAUDIT"."ACTION",1,22) = 'Arrived event (stop 1)'
and SUBSTR("TAUDIT"."RECORD_REF",7,6) = "TJOBS"."REFERENCE")

This way it looked through a much smaller set of data & returned the data in seconds. Obviously as I look at Weekly & Monthly reports the run time increases greatly. I'll try and give an idea of the times when I increase the ranges.

Thanks
0

Featured Post

Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

Join & Write a Comment

I hate sub reports and always consider them the last resort in any reporting solution.  The negative effect on performance and maintainability is just not worth the easy ride they give the report writer.  Nine times out of ten reporting requirements…
There have always been a lot of questions related to when Crystal Reports evaluates report components (such as formulas, summaries, cross-tabs, charts, to name a few examples). Crystal Reports uses a two-pass reporting process to provide greater …
Internet Business Fax to Email Made Easy - With eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, fr…
This video gives you a great overview about bandwidth monitoring with SNMP and WMI with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're looking for how to monitor bandwidth using netflow or packet s…

746 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now