Solved

Quick save to ADO

Posted on 2004-08-05
4
485 Views
Last Modified: 2013-11-23
We have an application that does a lot of data manipulation in memory using dynamic arrays. We have recently tried using ADO as the data storage mechanism, replacing 20 files with a single Access database with 20 tables.

It's great, except saving using APPEND/EDIT,{code}, then  POST is so SLOW when you want to do a bulk save (all records) to a single table. This is not a standard data-aware application that saves as you go. Is there a quick way to do a bulk save (savetotable?) using ADO.

   
0
Comment
Question by:Dennis_Treacy
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
4 Comments
 
LVL 3

Accepted Solution

by:
huferry earned 50 total points
ID: 11725137
Try using TADOCommand and inserting the records using an SQL Command:

INSERT INTO MyTable (Field1,Field2..) VALUES (Value1,Value2...)

It might help, it saves a few statements. If you use the Post method, you
probably populate the fields one by one:

   FieldValues['Field1'] := Value1;
   etc...

regards,
huferry
0
 
LVL 17

Assisted Solution

by:Wim ten Brink
Wim ten Brink earned 100 total points
ID: 11727765
I have to disappoint you... Any database action will be slower than just file manipulation actions. Databases have a lot of overhead for handling all the records. If you really need speed, don't use databases...
I had to write an application once that had to read half a million records from a textfile to send them to either an Oracle or SQL Server database. The whole reading, parsing and checking of the records in the textfile was done within two minutes. Sending the records to the database did add another half hour to an hour to the whole process. And there wasn't much that could be done about it. If I would just write it to another textfile, the whole application would run within 3 to 4 minutes...

Databases are powerful but there's always a cost. And this cost is the speed...

But okay, a quick option might be to build a large SQL query. For every record you need to add, add an "Insert into xxx (...) values (...);" to a string. This way, build a string with as many insert statements as you have records. Then execute that single query... This might or might not speed things up a bit.
0
 
LVL 4

Assisted Solution

by:Greg Rowland
Greg Rowland earned 100 total points
ID: 11730375
You could try using ClientDataSets and replacing your Arrays with DBGrids “whether or not they are visible”.
Then Apply updates periodically at appropriate moments during processing.
The nice thing ClientDatasets.ApplyUpdates will marshal the changed data for you. Although ClientDataSets can be a little tricky at first, surely the crew here at EE can help there.

Typically I would start with and ADOConnection component, connected to an ADOQuery, or ADOTable then a DataSetProvider connected to a ClientDataSet, finally a DataSource and DBGrid. The grids data can be index/enumerated just like an array; multiple grids can be layered to provided multi-dimensionality. Because ClientDataSets are in memory the performance should be pretty good. StringGrids also work real well if you what to view the processing results, and don’t mind doing type conversions on the fly.

If you do try this, pay careful attention the individual fields provider flags. While very useful limit the number of fields in the “where clause”.
“Date” data types also have some rather interesting anomalies when used with ClientDataSet.

Just an Idea.
 


0
 

Author Comment

by:Dennis_Treacy
ID: 11732566
Thanks for your comments.

huferry - the extra time taken is with the Append/Edit funtion, not Post. I expect what you have suggested will take the same time.

Alex - you have confirmed what I thought was the case. I was hoping I was wrong.

Surfer Joe - thanks. Worth a try.

Split points ok?
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Creating an auto free TStringList The TStringList is a basic and frequently used object in Delphi. On many occasions, you may want to create a temporary list, process some items in the list and be done with the list. In such cases, you have to…
Introduction Raise your hands if you were as upset with FireMonkey as I was when I discovered that there was no TListview.  I use TListView in almost all of my applications I've written, and I was not going to compromise by resorting to TStringGrid…
This video Micro Tutorial shows how to password-protect PDF files with free software. Many software products can do this, such as Adobe Acrobat (but not Adobe Reader), Nuance PaperPort, and Nuance Power PDF, but they are not free products. This vide…
In this video, viewers will be given step by step instructions on adjusting mouse, pointer and cursor visibility in Microsoft Windows 10. The video seeks to educate those who are struggling with the new Windows 10 Graphical User Interface. Change Cu…

728 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question