Question

"Too Many Paragraphs" in Lotus Notes DB

Asked by: irontodd

I've created a LotusScript that in general looks like this
myList = PickListMulti
forall item$ in myList
   existingdocgetdocumentbykey(key=item$)
   newdoc.body_x = existingdoc.body          '0 <= x <= 100, body is RTField
   newdoc.sectiontitle_x = item$
   x++
end forall
newdoc.form=newform
open newdoc for editing

' body is an existing field in each "item" in myList
' body_x (0<=x<=100) are all fields on the form "Newform"
' sectiontitle_x (0<=x<=100) are all fields on the form "Newform"

my question is this:
most times, this works just great, because in most cases, x is small (<10)
approximately 4 times a year, i will have to use my scripting where x will be large (>50)
today I ran in to a case (x happened to be 54) that once the document was ready to be opened, i received this error:
"Document has too many paragraphs - it must be split into several dcouments."
any ideas?
by the way, most times the resulting "newdoc" is < 100K, sometimes reaches 400K, and this time (i was testing a worse (not worst) case scenario, and the resulting newdoc is 1.4M
*trying to cover all the pertinent info*
each body and body_x may contain text, tables, images and/or doclinks

This Question has been solved and asker verified All Experts Exchange premium technology solutions are available to subscription members.

Subscribe now for full access to Experts Exchange and get

Instant Access to this Solution

  • Plus...
  • 30 Day FREE access, no risk, no obligation
  • Collaborate with the world's top tech experts
  • Unlimited access to our exclusive solution database
  • Never be left without tech help again

Subscribe Now

Asked On
2004-03-03 at 13:52:57ID20906143
Tags

many

,

has

,

too

,

paragraphs

,

lotus

Topic

Lotus SmartSuite

Participating Experts
1
Points
500
Comments
16

Trusted by hundreds of thousands everyday for fast, accurate and reliable tech support.

  • "The time we save is the biggest benefit of Experts Exchange to Warner Bros. What could take multiple guys 2 hours or more each to find is accessed in around 15 minutes on Experts Exchange." Mike Kapnisakis, Warner Bros.
  • "Our team likes having a resource that is more secure than just using Google and most experts using this service really know their stuff. It's nice to look here first versus using Google." Dayna Sellner, Lockheed Martin
  • "Anytime that I've been stumped with a problem, 9 out of 10 times Experts Exchange has either the accepted solution or an open discussion of the potential solution to the problem." Kenny Red, eBay Inc.

See what Experts Exchange can do for you.

Got a question?

We've got the answer.

Experts Exchange has been collecting answers to technology questions since 1996…3 million and counting! If you have a question, chances are we already have your answer.

Screenshot of Experts Exchange Knowledgebase

Need individual assistance?

Our experts are ready to help.

If you can't find the exact answer you're looking for, ask our exclusive community of 50,000 experts. You’ll get a personalized answer from a trusted professional.

Screenshot of Experts Exchange Knowledgebase

Want to learn from the best?

Read articles from industry experts.

Thousands of free tech tips, tricks, how-to’s and tutorials are available in our peer reviewed articles section. See for yourself how smart our experts are, no login required.

Screenshot of an Article

Working on a long term project?

Store your work and research.

Save solutions to your questions, answers you’ve discovered through searching plus helpful articles in your personal knowledgebase for easy future access.

Screenshot of Experts Exchange Knowledgebase

Access the answers to your technology questions today.

Subscribe Now

30-day free trial. Register in 60 seconds.

What Makes Experts Exchange Unique?

Members of the expert community talk about why the experience at Experts Exchange is different than what you will find anywhere else.

Trusted by the world's most respected brands.

image of each brand's logo

Faithfully serving IT professionals since 1996.

Experts Exchange Logo

Try it out and discover for yourself.

Subscribe Now

30-day free trial. Register in 60 seconds.

Related Solutions

  1. Include Doclink method in LotusScript?
    What is the LotusScript to include a doclink in an email you are sending? I am absolutely positive I've seen it before or even done it but I can't find it anywhere. The help files only have the "Newsletter" method FormatMsgWithDoclinks, and all the examples I can ...
  2. DocLink
    When creating an Action, how can I include a doclink to a different database?
  3. Create a doclink....
    How do I create a doclink with @Function (or Commands) when I only have the text version of the doclink (created and with @Text(DocUniqueID) Situation: I'm using a dialoglist (using a viewdialog for choices) to extract some data. I want the documentID of the document that y...
  4. LotusScript ADO/ADSI
    Here is a code snippet from VB that works fine in VB6.0. When converting code to LotusScript im not able to make it operate (return no value). I have replaced VB For Each to LotusScript For All. And the VB: oComm.ActiveConnection = oConn Is as sample shows converted to : Set ...
  5. Attach a DocLink in a mail using LotusScript
    Hi, I have some LotusScript which creates a mail, and sends it to various people. I would like to append a DocLink into this mail. I'm using AppendDocLink but nothing appears to be altering in the created mail. I'm obviously not using it properly, and would like some help ...
  6. Doclink in button
    Have a tracer email which is sent froma DB automatically. For this DB I have been asked if instead of the standard doclink, is it possible to have the doclink be access via a button. They want to have more of a "Click Here" type deal so it is a little clearer what t...

Free Tech Articles

  1. WARNING: 5 Reasons why you should NEVER fix a computer for free.
    It is in our nature to love the puzzle. We are obsessed. The lot of us. We love puzzles. We love the challenge. We thrive on finding the answer. We hate disarray. It bothers us deep in our soul. W...
  2. SCCM OSD Basic troubleshooting
    SCCM 2007 OSD is a fantastic way to deploy operating systems, however, like most things SCCM issues can sometimes be difficult to resolve due to the sheer volume of logs to sift through and the dispe...
  3. Migrate Small Business Server 2003 to Exchange 2010 and Windows 2008 R2
    This guide is intended to provide step by step instructions on how to migrate from Small Business Server 2003 to Windows 2008 R2 with Exchange 2010. For this migration to work you will need the fo...
  4. Create a Win7 Gadget
    This article shows you how to create a simple "Gadget" -- a sort of mini-application supported by Windows 7 and Vista. Gadgets can be dropped anywhere on the desktop to provide instant information, ...
  5. Outlook continually prompting for username and password
    There have been a lot of questions recently regarding Outlook prompting for a username and password whilst using Exchange 2007. There are a few reasons why this would happen and I will try to cover t...
  6. Backup Exchange 2010 Information Store using Windows Backup
    There seems to be quite a lot of confusion around the ability to backup Exchange 2010 using the built in Windows Backup feature. This stems from the omission of this feature prior to Exchange 2007 s...

Cloud Class Webinars

  1. Avoiding Bugs in Microsoft Access
    Alison Balter takes and in-depth look at avoiding bugs in Access. In this webinar you will learn about using the immediate window to debug your applications, invoking the debugger, using breakpoints to troubleshoot, stepping through code, setting the next statement to execute, ...
  2. Top 10 Best New Features in Visio 2010
    Scott Helmers gives live demonstrations of the top 10 new features in Visio 2010. This webinar will teach you how to create compelling diagrams by adding shapes to the page with a single click, linking the shapes in a diagram to data in Excel (or SQL Server, or SharePoint), ...
  3. IT Consultant Business Secrets Revealed
    Michael Munger, Experts Exchange tech pro and IT consultant, pulls back the curtain on his very successful businesses and answers question on every IT consultant and business owner should know about. He shares secrets on what he did to solve the 5 most common problems in IT, ...
  4. Disaster Recovery and Business Continuity
    Quest CTO, Mike Billon, gives an overview of the steps involved in building a dunamic disaster recovery plan. Through case studies and an examination of software/hardware tooles for monitoring and testing, you'll gain a better understandin of where you are, where you want ...
  5. Organize Your Visio Diagrams with Containers and Lists
    Scott Helmers uses cross functional flowcharts, wireframe diagrams, data graphic legends and seating charts to teach you: how to ustilize all three new structured diagram components in Visio 2010, the best practices for organizeing shapes in previous version of Visio, how to organize ...
  6. How to Us Objects, Properties, Events and Methods in Microsoft Access
    Alison Dalter gives an in-depbth look at objects, properties, events and methods in Microsoft Access. In this webinar you will learn about using the object browser, referring to objects, working with properties and methods, working with object variables, understanding the ...

Join the Community

Give a Little. Get a Lot.

Join the community of experts here and help other tech pros by answering question in your area of expertise. You can earn FREE access to all Experts Exchange's premium features and resources.

Join the Community

Answers

 

by: CRAKPosted on 2004-03-04 at 01:29:16ID: 10512397

I've never come across that error. There is a paragraph limit of 64k, but exceeding it raises a different error. The only other error I know of in RT is the total size of a RT field.... limited by the amount of available disk space (incl. max. db size if any).

What notes version are you using?

It could be of help to post (fragments) of the actual code here so we can see how you append all those fields. Any document refreshes / computations while you're appending to the RT field?

 

by: irontoddPosted on 2004-03-05 at 07:07:48ID: 10523284

this is notes 5.0.6a
i guess i forgot to post n my original that once I click OK on the above error, another comes up that says "User-defined error"

my code:

Sub Click(Source As Button)
' DECLARATIONS GO HERE
      
   AddTestCase=1
   index=0
   MAX_TEST_CASES=100

   Set db = s.CurrentDatabase
   Set NewDoc = New NotesDocument ( db )
   TestCaseList = Evaluate("@DbColumn( '' : 'NoCache' ; '' : '' ; '(TestCaseAndDocID)' ; 1 )")
   NewDoc.ProjectName=CurrentDoc.ProjectName
   NewDoc.CIT_assign_1=CurrentDoc.CIT_assign
   NewDoc.Tester=CurrentDoc.Tester
   NewDoc.Form = "CTPlan"

   Title = w.Prompt(PROMPT_OKCANCELEDIT,"Name Test Plan","What is this Test Plan For? ")
   If Isempty (Title) Then
      Goto Cancelled
   Else
      Call NewDoc.ReplaceItemValue("Title",Title)
   End If

   SelTestCase = w.Prompt(PROMPT_OKCANCELLISTMULT, "Test Case Selection", "Select all test cases to add to this Plan,"", TestCaseList)

   If Isempty (SelTestCase) Then
      Goto Cancelled
   Else
      Forall STC In SelTestCase
         Call NewDoc.ReplaceItemValue("Test_Cases_Included_"+index, STC)
         Set TestCaseAndDocID = DB.GetView("(TestCaseAndDocID)")
         Set TestCaseToPullFrom = TestCaseAndDocID.GetDocumentByKey(STC, True)
         Set notesItemToPullFrom = TestCaseToPullFrom.GetFirstItem( "Body" )
         Call NewDoc.CopyItem( notesItemToPullFrom, "Body_"+index )
         If index=MAX_TEST_CASES Then
            Goto LimitReached
         End If

         index = index+1
      End Forall
   End If
Cancelled:
   If index=0 Then
      Goto Quitting
   Else
      PromptSave=w.Prompt(PROMPT_YESNO, "Save Document", "Do You Want To Save This New Test Plan Document?")
      If PromptSave=1Then
         Goto SaveDoc
      Else
         Goto Quitting
      End If
   End If
      
LimitReached:
   Messagebox "You Have Reached The Max Number of Test Cases. Saving Document."
      
SaveDoc:
   Call NewDoc.Save(False,True)
'this next line is what cuses the error
   Call w.EditDocument( True, NewDoc )

Quitting:
End Sub

 

by: irontoddPosted on 2004-03-08 at 09:47:13ID: 10542386

Additional Note: This script Opens without error in Internet Explorer.

 

by: CRAKPosted on 2004-03-09 at 09:41:56ID: 10552475

???
Lotusscript is executed on the server when triggerred from a browser. Are you sure that the same error does not occur in the servers log (log.nsf)?
If it runs well on the server, it sounds like a machine related thing, like e.g. too little memory.

I haven't had a whole lot of time to look into this (this one required a bit more effort). Sorry about that.
If you execute the script with the debugger on (client), in which line is the error raized? I'd expect one of these. Can you please confirm?

...
Call NewDoc.ReplaceItemValue("Test_Cases_Included_"+index, STC)
...
Call NewDoc.CopyItem( notesItemToPullFrom, "Body_"+index )
...

 

by: CRAKPosted on 2004-03-09 at 09:43:17ID: 10552491

I meant "...to look into this yet"!
I will...!

 

by: irontoddPosted on 2004-03-09 at 10:33:07ID: 10553006

CRAK,
I'm doing the development of this on my local machine, not on a server. I checked the log file on my machine, and there are no entries in the log with todays date. I shut down Notes & brought it back up then opened log.nsf and still nothing for todays date. although the modification time on the file is today. so i'm not sure how my notes is configured (wonders if some configuration would reflect this situation).
I did some furter testing, and here's what I have found.
the script i've shown you is perfectly OK. all lines execute, including the w.editdocument. I found that the newdoc.initialize and newdoc.queryopen events complete successfully as well, but the first line of newdoc.postopen event never executes.
It seems to me that i'm looking at some limitation of Notes itself. GRR! I guess my question now is not about my script, but more of a "How do i catch this error, or better yet, prevent this error from showing? do you know if  I could look at any document-specific parameters, i.e. number of fields etc. for example, in pseudocode:

select all the items i want to copy over
forall
    see how many fields named "Body" there are in the existing doc
    if number of fields in newdoc plus value from previous line > supposed notes limit then
        throw a custom error and go back to top (ask user to select fewer items)
next

do you know what the limit of #of fields/items in a document is in Notes 5.06a?
if "Body" is split into 10 fields, 100 fields, or only 1, how do i figure this out?

thanks in advance. this is a bigger problem than i thought, so i've raised the bar. 500 pts.

 

by: CRAKPosted on 2004-03-10 at 02:16:26ID: 10559372

If you search designer help for ' limit & "field" ' you'll find following limits (and a lot more):
Size of a text field: 15 KB (not relevant since you use "copyitem")
Size of a richtext field: Limited only by available disk space up to 1GB (not very likely)
Max. size of a single paragraph: 64 KB (exceeding would raize a different error)
And perhaps more interesting:
How many fields in a database: ~ 3000 (limited to ~ 64K total length for all field names). You can enable the database property "Allow more fields in database" to get up to 64K uniquely-named fields in the database

Concerning debugging:
Unless you use error trapping (on error...) you can press continue. If an error occurs it will return showing you where it occurred.
If you decide to step through the code (not a lot of fun going through 50 docs untill someting MAY go wrong), some events are not offered to step through. Often those are script libraries. You can force debugger to debug that specific event by selecting the right event in an early stage of debugging (or at least when/while you can) and double click the first debuggable line in that event (place a red bullet as stop marker). Bear in mind that lines like "DIM" are skipped by the debugger!

> I'm doing the development of this on my local machine, not on a server.
Just to make absolutely sure: you're working on a client, not on the server, but where does the db reside? Is it "<database name> on local" or "<database name> on <servername>".
If on local: it's not of much use to check the log. When on the server: do check the log on that server when after the script was triggered from a browser.

 

by: irontoddPosted on 2004-03-10 at 04:54:13ID: 10560172

the database resides in my C:\NOTES\Data directory. =>"<database name> on local" so I guess checking the log doesnt mean anything.
I've already set the "Allow more fields" option to true. so this isnt going to work. This is a pain in the arse.
Any other ideas are greatly appreciated
Thanks!

 

by: CRAKPosted on 2004-03-10 at 09:07:44ID: 10562552

And why are you creating all those additional body-fields? I.e. why not merge the whole bunch into one?
Since is't not of much use (or fun) to test what happens when I run code like this near the no. of fields limit, I build something similar, copying body fields (w/o attachments etc.) to a single RT field on a new document.
Source documents were mainly EE notifications in my mailbox (to give you an impression of their size/looks).
I worked on local, using R6 on a 98SE notebook with 128 MB (not a heavy duty PC!).

It crashed with Count = 5001 on the WS.EditDocument line with a user defined error, pretty much just like your code.
As the results got saved, I tried to open it manually. That raized the infamous "too many paragraphs" error.

My code (agent, R%:run once/R6:none):

Sub Initialize
      Dim WS As New NotesUIWorkspace
      Dim Session As New NotesSession
      Dim ThisDb As NotesDatabase
      Dim NewDoc As NotesDocument
      Dim NewItm As Variant
      Dim MailDb As NotesDatabase
      Dim MailView As NotesView
      Dim MailDoc As NotesDocument
      Dim MailItm As Variant
      Dim Count As Integer
      
      ' set count = 0: no body fields copied yet
      Count = 0
      
      ' access current database and create a new doc.
      ' fill out formname and create an initial body field
      Set ThisDb = Session.CurrentDatabase
      Set NewDoc = ThisDb.CreateDocument
      NewDoc.Form = "Q_20906143"
      Set NewItm = NewDoc.CreateRichTextItem("Body")
      
      ' access maildatabase
      ' myMailPath was defined as constant; not included here!
      Set MailDb = New NotesDatabase("", "")
      If Not MailDb.Open("", myMailPath) Then Goto ForceExit
      Set MailView = MailDb.GetView("($Inbox)")
      Set MailDoc = MailView.GetFirstDocument
      Do While (Not (MailDoc Is Nothing)) And (Count <=5000)
            If Not MailDoc.HasEmbedded Then
                  Set MailItm = MailDoc.GetFirstItem("Body")
                  If Not (MailItm Is Nothing) Then ' previous version: HasItem passed, still the type property failed
                        If MailItm.type = 1 Then ' RT
                              Count = Count + 1
                              Call NewItm.AppendRTItem(MailItm)
                              Call NewItm.AddNewLine(1)
                        End If
                  End If
            End If
            Set MailDoc = MailView.GetNextDocument(MailDoc)
      Loop
      Call NewDoc.Save(True, True)
      Goto ForceExit
      
ForceExit:
      Print Count & " bodyfield collected"
      Call WS.EditDocument(True, NewDoc)
End Sub


This MUST be some limitation.
Taking that for truth, I see two ways out:
- splitting the code into several smaller doc's (perhaps one main and several respnses to keep them together)
- writing the RT content to word (we may lose (important?) details here!)

 

by: irontoddPosted on 2004-03-10 at 10:53:01ID: 10563685

Actually, I had not thought about appending into one field. I had designed the form the way it is now, but then a process change had me change the way the new doc is created, but i did not change the form, i decided to keep it the way it looks, but not the way it is created. i'll try to use the append method, and let you know how that goes
thanks

 

by: CRAKPosted on 2004-03-11 at 03:02:09ID: 10569976

Feel free, but as I mentioned: the error still does occur.
Hopefully yours is due to the no. of fields, while mine is the result of too little RAM.
Would listing up doclinks (+ e.g. doc title) be a solution? Would save you lots of redundant data and require far less resources!

 

by: CRAKPosted on 2004-03-11 at 03:03:23ID: 10569986

Strange.... I feel as if I'm the one asking the questions here...   ;-))

 

by: irontoddPosted on 2004-03-17 at 09:36:27ID: 10617078

Well, it turns out i FINALLY found the limit for the paragraph issue. the magic number is 21,509 paragraphs in a document. It turns out my problem is a result of the nature of the data i'm tranferring from the old docs to the new doc. the body's are all TABLES. Given that each table cell is considered a paragraph, and there are at least 4 columns in each of my tables, and i am trying to incorporate up to 50 existing docs (i've reduced MAX_TEST_CASES) this limits the average number of table rows to somewhere around 100. This is assuming there are no other "Paragraphs" in the body fields. It adds up quick! I guess my goal now is to see if there is any way to parse the body fields to see how many paragraphs are in the RTField, and create a new document if TOTAL_NUM_PARAGRAPHS reaches this magic number. Do you know if this would be possible in R5?

 

by: CRAKPosted on 2004-04-14 at 13:45:19ID: 10827598

I've put it to rest for a few days to see if I'd come up with an idea how to handle this.
There's no such thing as a "best practice" to work around this limit I'm afraid, but this is what I'd do....

Collect all "meta data" in a main document.
Collect a number of response document under it (use the MakeResponse-method) in which you store the collected paragraphs. Do watch TOTAL_NUM_PARAGRAPHS carefully: start a new response to the main doc when your next document can't be added. Provide a sequence number to each of the responses. You can keep track of your sequence by storing in in the main document.
Collect all responses in a non-hierarchical view. Categorise by @Text($Ref) (=maindoc's doc id.) in col 1, and sort on the sequence no. in col 2 (=hidden).
In the main document, you could embed this view as a single category view (formula: @Text(@DocumentUniqueID))

That'll keep things together. Next problem is proper disclosure....
May depend on personal preferences: click on a response in the embedded view to show it (either the same window, of in a different fram of a frameset?). From there, you could either rely on the embedded view for navigation, or add "next/previous" actions on the responses (using hide when etc.).

By choosing this approach, you could e.g. merge a number of doc's at some stage, and add a few more later on. Could turn the frustration of the problem into a nice (feasible!) challenge....

By seeing your existing code (and knowing my limitations regarding spare time) I trust that you don't really need it all chewed down to LS-code, but if you need a hand, you'll know where to find me!  ;-))

20120131-EE-VQP-002

3 Ways to Join

30-Day Free Trial

The Experts

98% positive feedback on 31,087 answers since March 2000. angeliii is a Microsoft Most Valuable Professional for his work with MS SQL Server & Develoment.

He has also proven his knowledge of Visual Basic Programming, PHP Scripting and Oracle Databases.

The Experts

97% positive feedback on 10,752 answers since July 2000. lrmoore has more than 18 years experience in the networking industry.

The six-time Mircosoft MVPs specialties include firewalls, virtual private networking, and network management.

Testimonials

"...and excellent source for support... Kind of like having your very own IT dept." Electriciansnet

Testimonials

"I was apprehensive at signing up at first. However... it has already made my life as an IT administrator much easier." JaCrews

Testimonials

"WOW! You guys have great, active, and knowledgeable people on here." moore50

Business Clients

Business Clients

In the Press

"If you’ve got a question... Experts Exchange can supply an answer.”

In the Press

"...an invaluable aid for both IT professionals and those who require tech support."

In the Press

"where IT professionals provide quick answers on just about any topic"

Business Account Plans

Loading Advertisement...