Question

Speed up VB code to copy

Asked by: nelsonje

I have a VB code that is working to copy recordset not null to null fields, but the code is running very slow any ideas on how to make it faster, this code has to go though about 6000 records and takes about 30 mins to do so... slow?? If you could help I have no ideas on how to speed this up..

Public Function LA()
 
Dim rstThis As Recordset
Dim varField2 As Variant
 
Set rstThis = CurrentDb.OpenRecordset("Weekly Comments") ' or whatever it is called
 
With rstThis
.MoveLast
.MoveFirst
If .RecordCount Then
Do Until .EOF
If Len(Nz(!Bill, vbNullString)) Then
varField2 = !Bill
Else
.Edit
!Bill = varField2
.Update
End If
.MoveNext
Loop
End If
.Close
End With
Set rstThis = Nothing
 
End Function

                                  
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:

Select allOpen in new window

This Question has been solved and asker verified All Experts Exchange premium technology solutions are available to subscription members.

Subscribe now for full access to Experts Exchange and get

Instant Access to this Solution

  • Plus...
  • 30 Day FREE access, no risk, no obligation
  • Collaborate with the world's top tech experts
  • Unlimited access to our exclusive solution database
  • Never be left without tech help again

Subscribe Now

Asked On
2008-10-16 at 10:22:07ID23821283
Tags

Microsoft

,

access vb

Topics

Microsoft Access Database

,

Visual Basic v1.0.5.x

,

Visual Basic Programming

Participating Experts
4
Points
500
Comments
20

Trusted by hundreds of thousands everyday for fast, accurate and reliable tech support.

  • "The time we save is the biggest benefit of Experts Exchange to Warner Bros. What could take multiple guys 2 hours or more each to find is accessed in around 15 minutes on Experts Exchange." Mike Kapnisakis, Warner Bros.
  • "Our team likes having a resource that is more secure than just using Google and most experts using this service really know their stuff. It's nice to look here first versus using Google." Dayna Sellner, Lockheed Martin
  • "Anytime that I've been stumped with a problem, 9 out of 10 times Experts Exchange has either the accepted solution or an open discussion of the potential solution to the problem." Kenny Red, eBay Inc.

See what Experts Exchange can do for you.

Got a question?

We've got the answer.

Experts Exchange has been collecting answers to technology questions since 1996…3 million and counting! If you have a question, chances are we already have your answer.

Screenshot of Experts Exchange Knowledgebase

Need individual assistance?

Our experts are ready to help.

If you can't find the exact answer you're looking for, ask our exclusive community of 50,000 experts. You’ll get a personalized answer from a trusted professional.

Screenshot of Experts Exchange Knowledgebase

Want to learn from the best?

Read articles from industry experts.

Thousands of free tech tips, tricks, how-to’s and tutorials are available in our peer reviewed articles section. See for yourself how smart our experts are, no login required.

Screenshot of an Article

Working on a long term project?

Store your work and research.

Save solutions to your questions, answers you’ve discovered through searching plus helpful articles in your personal knowledgebase for easy future access.

Screenshot of Experts Exchange Knowledgebase

Access the answers to your technology questions today.

Subscribe Now

30-day free trial. Register in 60 seconds.

What Makes Experts Exchange Unique?

Members of the expert community talk about why the experience at Experts Exchange is different than what you will find anywhere else.

Trusted by the world's most respected brands.

image of each brand's logo

Faithfully serving IT professionals since 1996.

Experts Exchange Logo

Try it out and discover for yourself.

Subscribe Now

30-day free trial. Register in 60 seconds.

Related Solutions

  1. VB / SQL Recordset Validation
    Purpose of code: See if an existing record is completed before ANOTHER record can be created for the same "Log Number". Here is what I am trying to do: 1) Open a recordset from a stored query. 2) If recordset finds a record, continue. If recordset is empty, display...
  2. VB and VC++
    Hi, What are the main differences between VB and VC++ ? Thanks.
  3. How can I update null value to ado recordset fields
    Dear all, I have a problem with assigning null value to an ado recordset field. Following code doesn't work. rs.field(1).value = Null Pls, give me some solution if you had. Thanks in advance, Zaw Zaw.
  4. Recordsets in VB
    How shall we move the records without using Recordsets or Recordsets property (movenext, move, etc.,)?
  5. Which is faster?
    I want to make sure a variable is not NULL. Which is faster? 1. inST = IIf(IsNull(rst!Status), "", rst!Status) or 2. inST = rst!Status & "" By how much (%) would be nice to know. Thanks
  6. using null in vb
    my oracle table has some null values. how do i check for null value in a filed in vb. i tried : if rs.fields(0).value is null if rs.fields(0).value is empty if rs.fields(0).value is "" but these does not work. pls. help.

Free Tech Articles

  1. WARNING: 5 Reasons why you should NEVER fix a computer for free.
    It is in our nature to love the puzzle. We are obsessed. The lot of us. We love puzzles. We love the challenge. We thrive on finding the answer. We hate disarray. It bothers us deep in our soul. W...
  2. SCCM OSD Basic troubleshooting
    SCCM 2007 OSD is a fantastic way to deploy operating systems, however, like most things SCCM issues can sometimes be difficult to resolve due to the sheer volume of logs to sift through and the dispe...
  3. Migrate Small Business Server 2003 to Exchange 2010 and Windows 2008 R2
    This guide is intended to provide step by step instructions on how to migrate from Small Business Server 2003 to Windows 2008 R2 with Exchange 2010. For this migration to work you will need the fo...
  4. Create a Win7 Gadget
    This article shows you how to create a simple "Gadget" -- a sort of mini-application supported by Windows 7 and Vista. Gadgets can be dropped anywhere on the desktop to provide instant information, ...
  5. Outlook continually prompting for username and password
    There have been a lot of questions recently regarding Outlook prompting for a username and password whilst using Exchange 2007. There are a few reasons why this would happen and I will try to cover t...
  6. Backup Exchange 2010 Information Store using Windows Backup
    There seems to be quite a lot of confusion around the ability to backup Exchange 2010 using the built in Windows Backup feature. This stems from the omission of this feature prior to Exchange 2007 s...

Cloud Class Webinars

  1. Avoiding Bugs in Microsoft Access
    Alison Balter takes and in-depth look at avoiding bugs in Access. In this webinar you will learn about using the immediate window to debug your applications, invoking the debugger, using breakpoints to troubleshoot, stepping through code, setting the next statement to execute, ...
  2. Top 10 Best New Features in Visio 2010
    Scott Helmers gives live demonstrations of the top 10 new features in Visio 2010. This webinar will teach you how to create compelling diagrams by adding shapes to the page with a single click, linking the shapes in a diagram to data in Excel (or SQL Server, or SharePoint), ...
  3. IT Consultant Business Secrets Revealed
    Michael Munger, Experts Exchange tech pro and IT consultant, pulls back the curtain on his very successful businesses and answers question on every IT consultant and business owner should know about. He shares secrets on what he did to solve the 5 most common problems in IT, ...
  4. Disaster Recovery and Business Continuity
    Quest CTO, Mike Billon, gives an overview of the steps involved in building a dunamic disaster recovery plan. Through case studies and an examination of software/hardware tooles for monitoring and testing, you'll gain a better understandin of where you are, where you want ...
  5. Organize Your Visio Diagrams with Containers and Lists
    Scott Helmers uses cross functional flowcharts, wireframe diagrams, data graphic legends and seating charts to teach you: how to ustilize all three new structured diagram components in Visio 2010, the best practices for organizeing shapes in previous version of Visio, how to organize ...
  6. How to Us Objects, Properties, Events and Methods in Microsoft Access
    Alison Dalter gives an in-depbth look at objects, properties, events and methods in Microsoft Access. In this webinar you will learn about using the object browser, referring to objects, working with properties and methods, working with object variables, understanding the ...

Join the Community

Give a Little. Get a Lot.

Join the community of experts here and help other tech pros by answering question in your area of expertise. You can earn FREE access to all Experts Exchange's premium features and resources.

Join the Community

Answers

 

by: patricka_0377Posted on 2008-10-16 at 10:33:16ID: 22733620

could try via a SQL UPDATE statement

from memory i think the command is CurrentDB.RunSQL (or something like that dont quote me)

then the SQL you want is.

UPDATE table_name
SET column1=value, column2=value2,...
WHERE some_column=some_value

 

by: ScottGutmanPosted on 2008-10-16 at 10:33:26ID: 22733622

Could we have some more information?

Does this sum what you are trying to do:
1. If there is data in "!bill" then leave the field alone"
2. If the field is empty, then fill it with a NULL??

Why not create a recordset with only fields that have len(!bill) = 0 and then Update the entire recordset with !bill = vbnullstring?

Also, we might help more if you can give us background on what you need

 

by: nelsonjePosted on 2008-10-16 at 10:39:15ID: 22733672

The code looks for Records that are not Null and copies that field to the next Null field, and resets when it comes to a Not null field again and loops.
Example:  BEFORE
K80524A  05-06-2005
                 05-06-2006
                 06-05-2008
K90564A 05-06-2008
                06-05-2007
                07-05-2006

Example: AFTER
K80524A  05-06-2005
K80524A  05-06-2006
K80524A 06-05-2008
K90564A 05-06-2008
K90564A 06-05-2007
K90564A 07-05-2006

 

by: ScottGutmanPosted on 2008-10-16 at 10:45:25ID: 22733747

Is the recordset the WHOLE table?

Try creating the recordset with just the Primary key, !bill.  This way you don't load too much into memory.

 

by: nelsonjePosted on 2008-10-16 at 11:04:11ID: 22733957

I dont have a primary key set up on this table and I dont suggest using one either..I find that it just complicates matters

 

by: ScottGutmanPosted on 2008-10-16 at 11:09:14ID: 22734013

then your data may have issues, how can you be certain that the records will always come up in the same order?

Anyway, just create the recordset with the on column, then run the script, and see if you get a performance boost.

 

by: ScottGutmanPosted on 2008-10-16 at 11:11:06ID: 22734032

oops typo on should be on

Anyway, just create the recordset with the --->>>one <<<--- column, then run the script, and see if you get a performance boost.

 

by: nelsonjePosted on 2008-10-16 at 11:14:05ID: 22734060

Run the script where in SQL? Is that right, and if so the VB script or the SQL script listed from PAT??

 

by: BigRatPosted on 2008-10-16 at 11:20:41ID: 22734137

What is the underlying database? And does it support batch update?

 

by: ScottGutmanPosted on 2008-10-16 at 11:28:04ID: 22734223

ok, I am not sure of your environment, but use this for your recordset.  you may get some errors, filelocking that we will have to tackle.

Set rstThis = CurrentDb.OpenRecordset ("Select Bill from <TableName>")

 

by: nelsonjePosted on 2008-10-17 at 07:12:23ID: 22740964

What is this code going to do??

 

by: ScottGutmanPosted on 2008-10-17 at 07:18:15ID: 22741035

instead of having a recordset with lots of fields and data, there will only be 1 column of data.

 

by: aikimarkPosted on 2008-10-18 at 07:27:36ID: 22748255

1. Explicitly set the type of recordset.  Note that table access is faster than dynaset.

Example:
Set rstThis = CurrentDb.OpenRecordset("Weekly Comments", dbOpenTable)
Set rstThis = CurrentDb.OpenRecordset("Weekly Comments", dbOpenDynaset)

2. Do your updates within one or more transactions.
Example:

    DBEngine.Workspaces(0).BeginTrans
    With rstThis

...

        End If
        DBEngine.Workspaces(0).CommitTrans
        .Close
    End With

3. RecordCount is not reliable and is unnecessary overhead in your code.

4. You do not need to .MoveLast and .MoveFirst

5. Your Null test is very inefficient.

See snippet for faster example.

6. Is there a primary key (hopefully and autonumber column) in [Weekly Comments]?  If so, you can solve this problem with a single SQL statement, although it won't be a simple statement as suggested in an earlier comment.
Such a SQL-only solution should be much quicker than a VBA solution, which is interpreted.

Public Function LA()
    Dim rstThis As Recordset
    Dim varField2 As Variant
    
    Set rstThis = db.OpenRecordset("Weekly Comments", dbOpenTable)  'or dbOpenDynaset if a query
 
    DBEngine.Workspaces(0).BeginTrans
    With rstThis
        Do Until .EOF
            If IsNull(!Bill) Then
                .Edit
                    !Bill = varField2
                .Update
            Else
                varField2 = !Bill
            End If
            .MoveNext
        Loop
        DBEngine.Workspaces(0).CommitTrans
        .Close
    End With
    Set rstThis = Nothing
End Function

                                              
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:

Select allOpen in new window

 

by: aikimarkPosted on 2008-10-18 at 07:28:43ID: 22748264

oops.  Should be:

    Set rstThis = Currentdb.OpenRecordset("Weekly Comments", dbOpenTable)  'or dbOpenDynaset if a query
 

 

by: aikimarkPosted on 2008-10-19 at 10:21:53ID: 22752935

6. *update* I've attempted to get a good performing query without success.  I expanded the configuration to use several queries and a temporary table.  It appears that the Jet engine will not join two tables (or sets of data) efficiently on a BETWEEN condition. :-(

 

by: aikimarkPosted on 2008-10-21 at 05:32:58ID: 22766607

Here's a pertinent question...
7. Is the [Weekly Comments] table bound to any object(s) when you are trying to do this update?

 

by: aikimarkPosted on 2008-10-25 at 06:01:05ID: 22802909

6. *UPDATE 2* Although not performing as fast as a single recordset-updating pass, I have been able to solve the problem reasonably fast using UPDATE query, avoiding the Between operator in the Where clause.  

Testbed Profile:
  Number of groups: 6002 (the two supplied in this question and 6000 randomly generated)
  Number of Null Bill rows/group: 3-10 (randomly generated)
  Number of Null Bill rows (total): 43k-45k (approx observed)
  Performance difference:
    Recordset iteration: 2.4 seconds
    Query iteration within transaction: 9.1 seconds
    Query iteration without transaction: 20-25 seconds
  PC speed: 800 MHz
  PC memory: 512 MB

NOTES:
* This assumes that there is an autonumber ID column on the [Weekly Comments] table.

* The overview of this approach to the updating is that one Null Bill row within each 'group' will be updated in each iteration.  Since the randomly generated groups had between 3 and 10 items, you can see the decreasing number of rows updated with each iteration.

* There were two sets of queries that gave similar results.  One using a query that returned those rows with Null Bill values, and the other just using the [Weekly Comments] table (self joined).

* I tried a version of the LA_Update_Query_Loop function that used dynamic SQL instead of the query definition.  It performed only slightly slower than the querydef sets.

* The big difference = TRANSACTIONS!!! (duh)

[NullBills] Query
_________________
SELECT [Weekly Comments].ID, [Weekly Comments].Bill
FROM [Weekly Comments]
WHERE ((([Weekly Comments].Bill) Is Null));
 
[UpdateBill_NullBills] Query - requires the [NullBills] query
_____________________________________________________________
UPDATE [Weekly Comments], NullBills SET NullBills.Bill = [Weekly Comments].[bill]
WHERE ((([Weekly Comments].Bill) Is Not Null) AND ((NullBills.ID)=[Weekly Comments].[id]+1));
 
[UpdateBill_WC] Query
_____________________
UPDATE [Weekly Comments], [Weekly Comments] AS WC_Null SET WC_Null.Bill = [Weekly Comments].[bill]
WHERE (((WC_Null.Bill) Is Null) AND (([Weekly Comments].Bill) Is Not Null) AND ((WC_Null.ID)=[Weekly Comments].[id]+1));
 
================================================
Public Sub LA_Update_Query_Loop(UpdQryname As String, boolTrans As Boolean)
    Dim qd As QueryDef
    Dim db As Database
    Dim sngStart As Single
    Dim sngQStart As Single
    Dim strTransStatus As String
    Set db = CurrentDb
    Set qd = db.QueryDefs(UpdQryname)
    If boolTrans Then DBEngine.Workspaces(0).BeginTrans
    sngStart = Timer
    Do
        sngQStart = Timer
        qd.Execute
        Debug.Print qd.RecordsAffected, Timer - sngQStart
    Loop Until qd.RecordsAffected = 0
    If boolTrans Then DBEngine.Workspaces(0).CommitTrans
    If boolTrans Then
        strTransStatus = "within"
    Else
        strTransStatus = "without"
    End If
    Debug.Print "_________________________"; vbCrLf; , Timer - sngStart & "  seconds -- Finished query loop updating (" & UpdQryname & ") " & strTransStatus & " trans "
End Sub
 
=======================================
Testbed Results:
LA_Update_Query_Loop "UpdateBill_NullBills",True
 6002          0.8203125 
 6002          0.6796875 
 6000          0.7109375 
 5238          0.7910156 
 4508          0.8125 
 3794          0.8515625 
 3054          0.8613281 
 2297          0.890625 
 1526          0.8925781 
 754           0.9003906 
 0             0.4804688 
_________________________
               9.083984  seconds -- Finished query loop updating (UpdateBill_NullBills) within trans 
 
LA_Update_Query_Loop "UpdateBill_NullBills",false
 6002          0.8222656 
 6002          3.134766 
 6000          3.244141 
 5266          3.103516 
 4519          2.84375 
 3796          2.625 
 3057          2.423828 
 2293          2.171875 
 1538          1.914063 
 767           1.611328 
 0             0.7109375 
_________________________
              24.61523  seconds -- Finished query loop updating (UpdateBill_NullBills) without trans 
 
LA_Update_Query_Loop "UpdateBill_WC",True
 6002          0.8105469 
 6002          0.6621094 
 6000          0.7304688 
 5216          0.7714844 
 4483          0.8300781 
 3739          0.8398438 
 2979          0.8925781 
 2210          0.9003906 
 1455          0.9003906 
 689           0.8828125 
 0             0.5 
_________________________
               9.052734  seconds -- Finished query loop updating (UpdateBill_WC) within trans 
 
LA_Update_Query_Loop "UpdateBill_WC",false
 6002          0.8222656 
 6002          3.365234 
 6000          3.34375 
 5195          3.105469 
 4443          2.923828 
 3741          2.693359 
 2994          2.455078 
 2224          2.181641 
 1436          1.851563 
 736           1.613281 
 0             0.4785156 
_________________________
              24.875  seconds -- Finished query loop updating (UpdateBill_WC) without trans
                                              
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:
28:
29:
30:
31:
32:
33:
34:
35:
36:
37:
38:
39:
40:
41:
42:
43:
44:
45:
46:
47:
48:
49:
50:
51:
52:
53:
54:
55:
56:
57:
58:
59:
60:
61:
62:
63:
64:
65:
66:
67:
68:
69:
70:
71:
72:
73:
74:
75:
76:
77:
78:
79:
80:
81:
82:
83:
84:
85:
86:
87:
88:
89:
90:
91:
92:
93:
94:
95:
96:
97:
98:
99:
100:
101:
102:

Select allOpen in new window

 

by: aikimarkPosted on 2008-10-25 at 06:14:53ID: 22802978

@nelsonje

We haven't heard from you in a week.  Are you still participating in this question?  Have we already solved this question for you and just don't know it yet?

We've shown you tweaks to your posted code that run in 2.4 seconds on my PC.  This should be enough improvement from the 30 minutes of your code to warrant some response from you, such as "OMG" or "OMG!" or "Wow! You guys rock." or "Awesome" or "Thanks" or "I'll try that and get back to you" or "It still performs poorly on my PC"

If you are working on this, please let us know.  If you are still having performance problems, some of our questions will give us enough information to help you beyond the tweak.

 

by: ScottGutmanPosted on 2008-10-26 at 21:49:27ID: 22810189

@ aikimark

I definately think you rock!!! OMG, WOW, Awesome, And Thanks...  I learned a few things.  Too bad i can't give you points.

 

by: aikimarkPosted on 2008-10-29 at 10:08:09ID: 22833346

@nelsonje

Thanks for the points.  Glad I could help.  It was an interesting problem that revealed a slow aspect of the Jet database engine.

Out of curiosity, did you use the tweaked recordset updating code or the iterative updating query?  I assume both are several orders faster than your (current) 30 minutes.

20120131-EE-VQP-002

3 Ways to Join

30-Day Free Trial

The Experts

98% positive feedback on 31,087 answers since March 2000. angeliii is a Microsoft Most Valuable Professional for his work with MS SQL Server & Develoment.

He has also proven his knowledge of Visual Basic Programming, PHP Scripting and Oracle Databases.

The Experts

97% positive feedback on 10,752 answers since July 2000. lrmoore has more than 18 years experience in the networking industry.

The six-time Mircosoft MVPs specialties include firewalls, virtual private networking, and network management.

Testimonials

"...and excellent source for support... Kind of like having your very own IT dept." Electriciansnet

Testimonials

"I was apprehensive at signing up at first. However... it has already made my life as an IT administrator much easier." JaCrews

Testimonials

"WOW! You guys have great, active, and knowledgeable people on here." moore50

Business Clients

Business Clients

In the Press

"If you’ve got a question... Experts Exchange can supply an answer.”

In the Press

"...an invaluable aid for both IT professionals and those who require tech support."

In the Press

"where IT professionals provide quick answers on just about any topic"

Business Account Plans

Loading Advertisement...