Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Crystal Reports - Very slow SQL command

Posted on 2013-01-29
12
Medium Priority
?
1,183 Views
Last Modified: 2013-08-16
I have a SQL query that I have created within a Crystal Report (2008) but it runs very slowly, I know this is because I have written it to do a search within a field.

Here is the query:
(select (min("TAUDIT"."ENTERED")) from "TAUDIT"
where SUBSTR("TAUDIT"."ACTION",1,22) = 'Arrived event (stop 1)'
and SUBSTR("TAUDIT"."RECORD_REF",7,6) = "TJOBS"."REFERENCE")

I have tested various methods of getting the correct value for "TAUDIT"."ACTION" =  'Arrived event (stop 1)', none of them really slow the system down.

The problem is when SUBSTR("TAUDIT"."RECORD_REF",7,6) = "TJOBS"."REFERENCE" is run, it takes a really long time to calculate.

What I really wanted to do was create a query that would search for complete string, not just part of the substring.
TAUDIT.RECORD_REF is a String(25) & the values I would be looking for would be a concatenation of "tJobs " & the TJOBS.REFERENCE (Number), so a value would look something like "tJobs 257407".
However, I am struggling to write a query that uses the CAST or CONVERT to join the 2 data types so that it can be compared with TAUDIT.RECORD_REF (String(25)).

Anyone have any ideas on this matter, or have had similar problems?

Thanks
0
Comment
Question by:Bizatanium
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
  • 2
  • +1
12 Comments
 
LVL 101

Expert Comment

by:mlmcc
ID: 38833697
How about trying

Val(SUBSTR("TAUDIT"."RECORD_REF",7,6)) = "TJOBS"."REFERENCE")

or perhaps it should be


Tonumber(SUBSTR("TAUDIT"."RECORD_REF",7,6)) = "TJOBS"."REFERENCE")


The problem is the system is converting the number to text to do the comparison not the substring part.

mlmcc
0
 
LVL 35

Expert Comment

by:James0628
ID: 38834601
I take it that this is a SQL Expression?  At first I assumed that it was a CR Command that was being used as a main datasource for a report.

 What db are you using?  CAST and CONVERT are MS SQL functions, but SUBSTR is not (unless it has been added in the more recent versions).

 FWIW, I doubt that converting the number to text is the problem, as mlmcc suggested.  He could be right.  It just doesn't seem to me like that would really be slowing things down.  If you want to try that, the functions he used - Val and ToNumber - are both CR functions, which may or may not exist in your db.  Since you've apparently got CAST and CONVERT, you'd probably have to use one of those.

 If you want to concatenate "tJobs" and TJOBS.REFERENCE, what kind of field is REFERENCE?  Int or float or something else?

 In MS SQL, if it's int, it could be as simple as something like 'tJobs ' + CAST (TJOBS.REFERENCE as varchar (7)).  For example,

select 'tJobs ' + CAST (257407 as varchar (7))

  produces

tJobs 257407


 If you are using MS SQL and were trying to use "&" to concatenate the strings, maybe that was your problem.  & is a logical AND.  + is used to concatenate strings.

 James
0
 
LVL 1

Author Comment

by:Bizatanium
ID: 38834877
Thanks for the thoughts.

I have tried the suggestions put forward by mlmcc, but as you said James0628 these are Crystal functions & don't work in a SQL query.

So if there was any misunderstanding here is the situation:
Using Crystal Reports (2008)
Accessing an Oracle Database
Building a SQL Expression Field within Crystal
TJOBS.REFERENCE is an Integer
TAUDIT.RECORD_REF is a String(25)
0
On Demand Webinar: Networking for the Cloud Era

Ready to improve network connectivity? Watch this webinar to learn how SD-WANs and a one-click instant connect tool can boost provisions, deployment, and management of your cloud connection.

 
LVL 101

Expert Comment

by:mlmcc
ID: 38837925
Sorry, by the syntax mainly the ( ) around the expression should have realized this is a SQL expression.

A SQL expression gets added to the query as a subquery so it is being run and analyzed for each record you retrieve.  Depending on how it is added, it may be run for all records in the database since it may run before the filtering is applied.

mlmcc
0
 
LVL 35

Expert Comment

by:James0628
ID: 38838816
mlmcc,

 Wasn't there something about SQL Expressions not working for Oracle?

 James
0
 
LVL 26

Expert Comment

by:Kurt Reinhardt
ID: 38840234
mlmcc,

 Wasn't there something about SQL Expressions not working for Oracle?

 James

I've never been able to get a correlated SQL Expression to work against an Oracle database.  Uncorrelated?  Yes. Correlated, No - generates syntax errors.  I've seen other people say they've gotten them to work, but I've never been able to prove it, nor do I now exactly how they were connecting to the database, what version of the database, etc...

In this particular case, the field that's causing a problem is the one that's correlated to the actual detail field.  It's interesting that it's working slow, as opposed to failing, but it's still not working as expected, which validates my experiences.
0
 
LVL 26

Expert Comment

by:Kurt Reinhardt
ID: 38842490
Try this:

(select (min(TAUDIT.ENTERED)) from TAUDIT
where SUBSTR(TAUDIT.ACTION,1,22) = 'Arrived event (stop 1)'
and SUBSTR(TAUDIT.RECORD_REF,7,6) = "TJOBS"."REFERENCE")

Open in new window


I've removed the double quotes from around all database objects except for the field that should be correlated from the database.  I would recommend you actually delete "TJOBS"."REFERENCE" and then drag it back in from the field tree into the same exact spot so that it's automatically formatted how your data connection needs it.

Why did I remove the double quotes from the other database objects?

1)  They're unnecessary for general SQL and probably unnecessary for the data connection. If you were writing this query in TOAD, for example, you wouldn't need them.

2)  To avoid "confusing" the database engine.  When you drag a field in from the field tree, it's formatted exactly as the connection needs it.  If that happens to be double quotes and you've manually entered in double quotes for all other fields, the SQL Expression might be trying to query the tables you already have in the report instead of an actual external query.

Please note, as mentioned above, I've never gotten a correlated SQL Expression to work with Oracle.  That same query I've listed above would work perfectly fine against SQL Server or DB2, however.  Regardless, please try and let us know if it works.
0
 
LVL 1

Author Comment

by:Bizatanium
ID: 38882330
Thanks again for your comments.
I've had this on the back burner for a while as I've just got used to it taking up to 45 minutes to churn through a days data.
I've tried all of the above suggestions with fingers crossed, all to no avail. I get the same results, a query that pulls a months worth of data still takes up to 13 hours to run....less than ideal.

I had hoped that the ultimately I would be able to resolve the issue that I see being the main cause of the problem. The query searches through all the records returned for the given period for a substring (SUBSTR(TAUDIT.RECORD_REF,7,6) = "TJOBS"."REFERENCE"). Rather than matching TAUDIT.RECORD against some sort of concatenation of "TJobs " & TJOBS.REFERENCE.
0
 
LVL 35

Expert Comment

by:James0628
ID: 38883667
I haven't used Oracle, but it seems like if it's possible to concatenate a string and a numeric column that was converted to a string, it should be fairly simple.

 However, there are apparently problems when trying to use a SQL Expression with an Oracle db, so even if you got the concatenation to work, the SQL Expression may not work, or may still be quite slow.

 The real solution may be a "manual" query that reads the main report data and incorporates the SQL Expression sub-query.  The report would use that as the datasource, instead of the tables.  That will probably give you the best possible performance.  Perhaps a CR Command, or something like a stored procedure or view in the db.

 FWIW, you could get a start on the manual query by going to Database > "Show SQL Query" to see the query that CR is using, and copying that.  Actually, even if you don't want to try a manual query at this point, looking at the query that CR is using may be informative.

 James
0
 
LVL 1

Accepted Solution

by:
Bizatanium earned 0 total points
ID: 38927888
I've tried a number of things based on all your suggestions. None of them really improved the performance of the report.

It did however make me realise that without the ability to concatenate the expression I was looking for, meant that the SQL query was being run for every record, against every record in a table of 5 million+entries.

To get it working for the Daily version of my report I was able to use:
(select (min("TAUDIT"."ENTERED")) from "TAUDIT"
where ("TAUDIT"."ENTERED")between (sysdate-2) and (sysdate)
and SUBSTR("TAUDIT"."ACTION",1,22) = 'Arrived event (stop 1)'
and SUBSTR("TAUDIT"."RECORD_REF",7,6) = "TJOBS"."REFERENCE")

This way it looked through a much smaller set of data & returned the data in seconds. Obviously as I look at Weekly & Monthly reports the run time increases greatly. I'll try and give an idea of the times when I increase the ranges.

Thanks
0

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Crystal Reports: 5 Tests for Top Performance It is complete, your masterpiece report.  Not only does it meet your customer’s expectations, it blows them out the water, all they want is beautifully summarised and displayed in a myriad of ways. …
Hello everyone, Hope you find this as helpful as we did. We have on the company I work for an application built in Delphi V with Crystal Reports 8. We all know that Crystal & Delphi can be temperamental sometimes and the worst thing is, nearly…
Do you want to know how to make a graph with Microsoft Access? First, create a query with the data for the chart. Then make a blank form and add a chart control. This video also shows how to change what data is displayed on the graph as well as form…
This tutorial will teach you the special effect of super speed similar to the fictional character Wally West aka "The Flash" After Shake : http://www.videocopilot.net/presets/after_shake/ All lightning effects with instructions : http://www.mediaf…
Suggested Courses

688 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question