Rabbit80
asked on
Slow user function in MS Access
I have a database containing 10s of thousands of product details. In order to import these into our online store, I use some VBA functions to format the data for our site. The problem is that it is incredibly slow.
How can I speed things up?
How can I speed things up?
Public Function GenShortDesc(ftTableName As String, _
ftProductID As String, _
ftFeatureValue As String, _
ftSequence As String, _
ProdID As String) As Variant
On Error GoTo Err_Handler
Set dbs = CurrentDb
'ftTableName - Name of the features table
'ftProductID - Features table Product ID field name
'ftFeatureValue - Features table vale field name
'ftSequence - Features table order
Dim rs As DAO.Recordset 'Related records
Dim strSql As String 'SQL statement
Dim strOut As String 'Output string to concatenate to.
'Initialize to Null
GenShortDesc = Null
'Build SQL string, and get the records.
strSql = "SELECT * FROM " & ftTableName
strSql = strSql & " WHERE " & ftProductID & " = " & ProdID
strSql = strSql & " ORDER BY " & ftSequence
Set rs = dbs.OpenRecordset(strSql, dbOpenDynaset)
strOut = "<ul>"
'Loop through the matching records
Do While Not rs.EOF
strOut = strOut & "<li>" & rs(ftFeatureValue) & "</li>"
rs.MoveNext
Loop
rs.Close
strOut = strOut & "</ul>"
'Return the string
GenShortDesc = strOut
Exit_Handler:
'Clean up
Set rs = Nothing
Exit Function
Err_Handler:
MsgBox "Error " & Err.Number & ": " & Err.Description, vbExclamation, "GenShortDesc()"
Resume Exit_Handler
End Function
SELECT Products.ProductID AS ID, Products.*, genTable("CompAttributes","ProductID","Name","Value","Group","Sequence",[id]) AS LongDesc, GenShortDesc("Features","ProductID","Description","Sequence",[id]) AS ShortDesc
INTO [Excel 8.0;Database=I:\Products-txt.xls;].NewExcelTable FROM Products;
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
You have a 2nd function embedded in that query as well. What does the genTable( ) function look like?
Any time you embed a function in a query and pass it unique values, that function is going to fire for every record in your main query. If that function creates a separate recordset, then that process takes time.
One thing I would do is move your:
Set dbs = CurrentDb
line outside of that function. Declare dbs as a public variable to the form or module you are working in. Then set the value of that object to CurrentDb outside of the function call, so it doesn't have to be created for every record. There is a very miniscule amount of overhead associate with this process, but if you have 10's of thousands of records in your [Products] table, this will save you some time.
Any time you embed a function in a query and pass it unique values, that function is going to fire for every record in your main query. If that function creates a separate recordset, then that process takes time.
One thing I would do is move your:
Set dbs = CurrentDb
line outside of that function. Declare dbs as a public variable to the form or module you are working in. Then set the value of that object to CurrentDb outside of the function call, so it doesn't have to be created for every record. There is a very miniscule amount of overhead associate with this process, but if you have 10's of thousands of records in your [Products] table, this will save you some time.
ASKER
OK - thanks for the advice, I will take a further look this afternoon.
GenTable and GenShortDesc are very similar functions designed to build up a list of product features. They both operate on different tables within the database though (GenTable uses a complete list of attributes and builds a html specs sheet, GenShortDesc builds a list of highlights/features for the product)
I notice MacroShadow mentions the data being stored on a slow server - although this is not the case, I am using linked tables that are generated from text files. Would this have an impact on speed? These files are updated on a daily basis and I don't want for the end user (my boss) to have to update the tables manually.
Scott - for each ftProductID there may well be 5-10 records.
Fyed - will do as you suggest.
I'll update this question shortly as I change things around a bit!
GenTable and GenShortDesc are very similar functions designed to build up a list of product features. They both operate on different tables within the database though (GenTable uses a complete list of attributes and builds a html specs sheet, GenShortDesc builds a list of highlights/features for the product)
I notice MacroShadow mentions the data being stored on a slow server - although this is not the case, I am using linked tables that are generated from text files. Would this have an impact on speed? These files are updated on a daily basis and I don't want for the end user (my boss) to have to update the tables manually.
Scott - for each ftProductID there may well be 5-10 records.
Fyed - will do as you suggest.
I'll update this question shortly as I change things around a bit!
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
I just realised - my output is also being truncated to 255 characters :(
@Rabbit80
What version of Excel and Access are you using?
What version of Excel and Access are you using?
ASKER
@Aikimark 2010
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
It would seem the main issue was because I was using linked tables. MacroShadow picked up on the slow performance issue in the first response which led to the solution so I assigned him the highest points. I have taken on board other comments and have assigned points to others as well.
It would seem the main issue was because I was using linked tablesI don't see where the first comment mentioned linked tables at all. MacroShadow suggested you change the output to a String, and also that your environment might be at fault. Unless I missed something in earlier comments, my last comment suggested that you import the data instead of link to it.
So I'm curious as to exactly what the resolution was ...
ASKER
MacroShadow
Me
The resolution is not to use linked tables in this case. MacroShadows first response made me realise that the use of linked tables could be the 'problem with the environment'
I was already on to converting the tables when you posted your suggestion - I have given credit to you however as you were correct.
2. The code should not be slow unless the data is stored on a slow server. In other words it's more likely a problem with the environment than with the code.
Me
I notice MacroShadow mentions the data being stored on a slow server - although this is not the case, I am using linked tables that are generated from text files. Would this have an impact on speed? These files are updated on a daily basis and I don't want for the end user (my boss) to have to update the tables manually.
The resolution is not to use linked tables in this case. MacroShadows first response made me realise that the use of linked tables could be the 'problem with the environment'
I was already on to converting the tables when you posted your suggestion - I have given credit to you however as you were correct.
Thanks for clarifying that. It was somewhat confusing as to what exactly you accepted as the solution, since MacroShadow did not mention getting rid of linked tables, and you indicated that was the ultimate solution.
I was already on to converting the tables when you posted your suggestionJust an FYI: We have no way of knowing what you've tried (or are trying) unless you tell us this, so a correct answer posted after you began trying a different route is still a correct answer. That's sort of like taking your car to the mechanic when you can't diagnose a problem, and then refusing to pay your bill after the fact because you now know what the problem was!
As a general statement, you can sometimes find things to perform better if you use temporary tables. To do that, first create a table that would store the information returned from genShortDesc, and then Join the Products Table to that temporary table. Then, instead of re-running the function for each record, Access could intelligently pull records from the temporary table.
A simple SELECT INTO or INSERT INTO could do the tricK
Currentdb.Execute "SELECT ProdID, '<li>' & FeatureValue & '</li>' AS FeatureVal INTO tmpShortDes FROM [ftProductTable]"
Then just join tmpShortDesc to Production on ProductID
You seem to have multiple tables and/or fields that could be used to obtain the ShortDesc, so just add each of those table's data into tmpShortDesc.