Link to home
Start Free TrialLog in
Avatar of RanjeetRain
RanjeetRain

asked on

Getting latest data

I need to generate a report containing latest data. Data is put into an Excel spreadsheet.

Database is medium-sized, approx 700 MB currently. About 50 forms and 100 views. It is for web but maintainance people do use Notes client.

I create a view that contains all the data I need (simple view, based on on form and a simple selection criteria). The view selects 7500 documents let us say. What will be the best approach?

Do I scroll thru using GetFirstDocument/GetNextDocument? Or should I use NotesViewEntry.Document? Its a flat view, no response documents, no totals, nothing. Please bear in mind that concurrency of data is important. Equally important is performance. This will be done many times a day, so more optimized the code the better it will be.

What is your opinion?
SOLUTION
Avatar of HemanthaKumar
HemanthaKumar

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Avatar of Sjef Bosman
Sjef Bosman
Flag of France image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of RanjeetRain
RanjeetRain

ASKER

>> Scrolling thru the view doesn't preserve the order .. But using NotesViewNavigator does !

THat is interesting Hemantha! I am however not thinking about ordering here per se. My concern is - a view with 1000s of rows of info it it, how to export it the fastest way. I'd also like to reduce the disk I/O overhead on the server.

Sjef, this shouldn't be done manually. It would be a schduled agent that will send the report to designated people on predetermined frequency. Also, that method is guarnteed to run out of steam, as the number of rows to export will easily overwhelm the size limit of WK? file format.

Kalios, thanks for offer to help, but I am not looking for code to export a view. I have several variants of it and some are faster than others. Just wondering which is best.

My concern is optimizations since I am looking forward to export several thousand records several times a day. I wouldn't want server to respond slowly when report generation agent was running.

I was of opinion (and it has not changed so far) that NotesViewEntryCollection would give the fastest access. But there is an aspect I am not satisfied with. If I use NotesViewEntryCollection and NotesViewEntry, the data may or may not be concurrent, right? Can I gurantee that?

Please correct me if I am wrong. I am just toying with various ideas to select the best method.
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
If you cannot use the "Copy Selected as Table" button, I think it's faster if you access data displayed in the view through the array notesDocuments.ColumnValues or the entry navigation (ss HemanthaKumar said). But it is a supposition, I have no experience.
ASKER CERTIFIED SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Ranjeet,

ViewEntry does not support show multiple values as seperate entries. This is a bug !

http://www-1.ibm.com/support/docview.wss?rs=899&uid=swg21089632

If any of your view columns have this option checked then it wont work.

Also I've this in one of my db's and it works great....It exports approximately 30000 records at timely intervals and i feel it pretty fast.

Sub Initialize
     Dim s As New notesSession    
     Dim view As NotesView
     Set view = s.currentDatabase.getView("MValues")
     Set doc = view.GetFIrstDocument    
     Open "C:\bh.txt" For Output As #1  
     While Not doc Is Nothing          
          Forall x In doc.columnValues
               If Isarray(x) Then
                    Forall colmember In x
                         Print #1 , colmember;
                    End Forall
               Else
                    Print #1 , x ;
               End If
          End Forall
          Print #1 ,          
          Set doc = view.GetNextDOcument(doc)
     Wend
     Close #1
     Dim temp As Variant
     temp = Shell("C:\bhini.bat")
End Sub

Kalios
Ranjeet,

Manually? There is a @Command([FileExport]; filetype; filename). Notes will export the current view. I just tested with the Designer Help database, it contains some 8000 documents: an export to a .wk4 file is no problem, nor is the import in Excel.

As easy as 1-2-3!
Just found that there is a maximum of 8192 documents... So please keep the # selected documents below 8192, I'd say :)

See
    http://www-912.ibm.com/s_dir/slkbase.NSF/0/fb0ffc353e7c9e218625664f00589bad?OpenDocument
Thanks for all the responses. But there are a few limitations.

1. Any method that puts a limit on the number of rows is out of discussion. Tomorrow it will fail even if it works today. So they are out.
2. FileExport command requires a current view. Not usable from scheducled agents.
3. Methods that rely on manual selection are out of question. They will not be useful in an scheduled agent.

I guess my best bet would be scrolling thru the NotesViewEntryCollection. This works well and i do feel its faster. But I was apprehensive about the concurrency of data. How useful is NotesView.Refresh? I read it just refreshes the view, doesn't rebuild the index. What exactly is that suposed to mean?
I heard that Notes R66 will contain addidional parameters for the [FileExport] command. Can you wait that long? ;)

Sorry, didn't see the sched agent requirement. Better next time :|
Partha,

I am looking at the code given in the linked post. Its looking like some clever trick. However I really have to check its performance. Do you know how it performs?

Kalios,

Your code is what is currently written and its very slow. Some agents run for over an hour. I took one of the simplest examples becasue the idea is to arrive at the best method. THere are agents that run forever during peak hours. I really need optimized code. At least it should not run slow because of poor code (we tried).

Sorry Sjef, find out something before that edition comes. Its a dull life with many agents running in the background :|

Also, what about the View refresh problem? If I have a view that is set to "Auto refresh", can I trust domino to keep the value in the view concurrent? (Please keep in mind that no one will explicitly refresh the view or rebuild the index).
I am still awaiting responses here. Anybody wants to throw some light on the concurrency aspect?
No cents left :(
Time to close this. Thanks for participating guys.