DIIOP memory leak when retrieving views using Database.getViews()

I have an application where I'm trying to retrieve all the views that exist in an NSF file using the Java/Corba api and talking to DIIOP. In the process I have found that even after recylcing the objects using the "recycle" call the DIIOP process does not release the memory after the operation is complete. The memory footprint is quite big depending upon the size of the NSF file.
I use the getViews() to basically get the names of all the views in an NSF.

My question is
1. Is this a known DIIOP behaviour. If so are there any things possible through the code that can be execute to force the DIIOP process to release, besides recyling the DIIOP process as that is not a solution.
2. Is there any API to just fetch the names of all the views that exist( including user defined dolders ) in an NSF.
deepankar_das1Asked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Sjef BosmanGroupware ConsultantCommented:
Do you recycle EVERY Notes-object you create in your application? Really every single one?

Here's an example from the Domino Designer Help database:

This agent gets the top-level entries in an outline last to first.
import lotus.domino.*;
public class JavaAgent extends AgentBase {
  public void NotesMain() {
    try {
      Session session = getSession();
      AgentContext agentContext = session.getAgentContext();
      // (Your code goes here) 
      Database db = agentContext.getCurrentDatabase();
      Outline outline = db.getOutline("DiscOutline");
      System.out.println("*** " + outline.getName() + " ***");
      System.out.println("TOP-LEVEL ENTRIES LAST TO FIRST");
      OutlineEntry tmpentry;
      OutlineEntry entry = outline.getLast();
      while (entry.getLevel() > 0)
        entry = outline.getParent(entry);
      while (entry != null) {
        System.out.println(entry.getLabel());
        tmpentry = outline.getPrevSibling(entry);
        entry.recycle();
        entry = tmpentry;
        }
    } catch(Exception e) {
      e.printStackTrace();
    }
  }
}

Open in new window

mbonaciCommented:
deepankar_das1Author Commented:
Here's the code that is used to retrieve the views. The database and session are recycled outside.

Here's my observation w.r.t the memory consumption on the DIIOP process.  
1. Right after I call the db.getViews the memory spikes up for e.g in my case by about 50MB.
2. Even after I complete the loop and recyle all the objects the memory does not seem to get released on the server.

try {
            Vector folders = null;
            folders = db.getViews();

            if( folders == null ) {
                logger.error("There are no views/folders in the database");
                return null;
            }
            else {
                logger.info("There are " + folders.size() + " views/folders in the database");
            }

            String origName = "";
            String retName = "";
            View view = null;
            //list out the names of the different views/folders in the database.
            for(int i = 0; i < folders.size(); i++)
            {
                view = (View)folders.get(i);
                if( toBeSkipped(view.getName()) )
                {
                    view.recycle();
                    continue;
                }
                if( view != null ) {
                    try {
                        //retrieve view properties.
                    }
                    catch(NotesException e) {
                        logger.error("Failed to fetch view : " + e.text + "[" + e.id + "]");
                    }
                }
            }

           db.recycle(folders);
        }
        catch(NotesException ne) {
            logger.error("Failed to fetchAllFolders : " + ne.text + "[" + ne.id + "]");
            retFolders = null;
        }
        return retFolders;
    }

Open in new window

JavaScript Best Practices

Save hours in development time and avoid common mistakes by learning the best practices to use for JavaScript.

Sjef BosmanGroupware ConsultantCommented:
Can you call that function several times (if it doesn't harm your application), to see if memory goes up 50Mb each invocation? Domino is sometimes very good at "finders keepers"...
mbonaciCommented:
Have you tried to request garbage collector to run after you recycled those reference vars?
AFAIK, recycle only declares an object to be eligible for garbage collector.

System.gc();
deepankar_das1Author Commented:
I did put a loop in the application adn tried calling db.getViews() several times. After the first call the memory still shoots up by 10MB for every call. Also I enabled log level to 4 on the server and see the log statements that the recycle/bulk recycle  is being executed but I do not see the corresponding impact on the memory taken up by the DIIOP process.

Is there a way to force the GC on the DIIOP process itself. I do not see any such API.
mbonaciCommented:
No, just the standard call to gc, on the JVM level.
Remember, you can only politely :) request garbage collector to run, but no one can guarantee you that it will actually run when you call System.gc().
mbonaciCommented:
Have you tried with NoteCollection?
How many views do you have in your db?
Sjef BosmanGroupware ConsultantCommented:
Do you get errors reported from the lines calling logger.error? If so, that might be the explanation: can you put the db.recycle and other recycle calls in a finally-clause, so that they're always executed?
deepankar_das1Author Commented:
Like I mentioned, I have already verified that the recycle calls are getting logged on the server log. Also the db and session are getting recycled.

I cannot use NoteCollection since I'm not trying to extract the view documents at this point.  I'm interested in fetching are the names of the views and user defined folders in the NSF. Once I have this list I would extract the documents in each of the views.

I'm very surprised that there is not API that is light weight call that would just return all the view names and/or folder names in an NSF. I did not find this even in the "C" api as well.
mbonaciCommented:
And what do you think you get by calling getViews?
You don't get a Vector of Strings, you get a collection of Views. View is a document (design element type doc, but nevertheless - document).

NoteCollection will provide the only other way to get the view names (select views > build collection > iterate through collection using getXxxNoteID > get view doc > get doc item value).

Example of iterating through a NoteCollection:
http://publib.boulder.ibm.com/infocenter/domhelp/v8r0/index.jsp?topic=/com.ibm.designer.domino.main.doc/H_EXAMPLES_REMOVE_METHOD_NC_JAVA.html

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
deepankar_das1Author Commented:
Since the solution did not really address the problem that was stated I'm giving the grade as "B". Ideally I want to leave this question open, but it is preventing me from posting other questions
mbonaciCommented:
Your goal was to get the view names (without memory leaks), right?

And you couldn't do it because you had memory leak, right?

So we would've probably solved your problem completely, if you've answered the questions I asked and followed my suggestions with NoteCollection.

We're not writing a research paper on memory leaks in DIIOP calls.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Lotus IBM

From novice to tech pro — start learning today.