Link to home
Start Free TrialLog in
Avatar of David Robitaille
David RobitailleFlag for Canada

asked on

need help to build a database.search formula

I'm writing a webservice with java on lotus domino designer / notes 7.

I want to get all contact that are in subset of "regions".

The problem is the region migh be defined in a "parent" field.
for that i build a unefective but working function "getFieldFormParents" i loop trough the colection to get only the documents that are in the selected regions.

I'm getting timed out...

Could i do this directly in the database.search using a formula or something faster than my loop thing?

private String getFieldFormParents(Document Doc, String FieldName, Database ContactDB){
		String ParentUNID;
		Document ParentDoc;
		String FieldValue;
		
		try{	
			//check for the field
			FieldValue=getField(Doc, FieldName);
			if (FieldValue.trim().length() > 0){
					//found!
					return FieldValue;
			}else{
				//check for a parent
				ParentUNID = Doc.getParentDocumentUNID();
				if (ParentUNID.trim().length() > 0){
					ParentDoc = ContactDB.getDocumentByUNID(ParentUNID);
					//recursive call
					return getFieldFormParents(ParentDoc, FieldName, ContactDB);					
				}else{
					//no more parent field not found
					return "";
				}
			}
		} catch(Exception e) {
			e.printStackTrace();
		}
		return null;
	}
private String getField(Document ContactDoc, String FieldName){
			 Item item;			
                 try{
					// Get item object. Global object nabdoc was set earlier in
					// getPersonDocument function.
					item=ContactDoc.getFirstItem(FieldName);
					//Sometimes item does not exist in the document, it's not an error so
					// we return an empty string.
					if (item==null){
						return "";
					}else{
					//Return the value that the item holds
						return item.getValueString();
					}
				} catch(Exception e) {
					e.printStackTrace();
				}
			return "";
		}
 
 
	private DocumentCollection getContactDocumentCollection(int intContactSelectionType,
												 String strFilter, 
												 boolean onlyFirstLetterFilter,
												 int[] RegionSelection,
												 Database ContactDB){			
			DocumentCollection ContactsSearch;
			DocumentCollection ContactsReturn;					
			Document ContactDoc;
			Document CompanyDoc;
			String strRegion;
			int intRegion;
			boolean blnFound;
			try{
				if (onlyFirstLetterFilter){
					//Search the database for documents where CompanyName field matches received parameter
					ContactsSearch = ContactDB.search("Form=\"Contact\" & @LowerCase(@Left("+ContactSelectionType.getSearchField(intContactSelectionType)+";1))=\""+strFilter.toLowerCase().charAt(0)+"\"", null, 0);
					//ContactsReturn = ContactDB.search("Form=\"Contact\" & @LowerCase(@Left("+ContactSelectionType.getSearchField(intContactSelectionType)+";1))=\""+strFilter.toLowerCase().charAt(0)+"\"", null, 0);
					
 
				}else{
					ContactsSearch = ContactDB.search("Form=\"Contact\" & \"A\"=\"X\"", null, 0);				
					//ContactsReturn = ContactDB.search("Form=\"Contact\" & "+ContactSelectionType.getSearchField(intContactSelectionType)+"=\""+strFilter+"\"", null, 0);				
				}
				ContactsReturn = ContactDB.search("Form=\"Contact\" & "+ContactSelectionType.getSearchField(intContactSelectionType)+"=\""+strFilter+"\"", null, 0);
				ContactDoc = ContactsSearch.getFirstDocument();
				while (ContactDoc != null) {
					strRegion=getFieldFormParents(ContactDoc, "Region", ContactDB);
					strRegion = strRegion.trim();
					if (strRegion.length() == 0){
						//region is from unlinked CIE
						CompanyDoc = getCompanyDocumentForContact(ContactDoc, ContactDB);
						strRegion=getField(CompanyDoc, "Region");
						strRegion = strRegion.trim();
					}
					if (isInteger(strRegion)){
						intRegion = Integer.parseInt(strRegion);
						blnFound = false;
						for (int i=0;  i<RegionSelection.length;  i++){
							if (intRegion == RegionSelection[i]){
								blnFound = true;
								break;
							}
						}
					}else{
						blnFound = false;
					}
					
					//remove doc from Collection
					if (blnFound) {
						ContactsReturn.addDocument(ContactDoc);
					}
					ContactDoc = ContactsSearch.getNextDocument();
				}
					
				
				return ContactsReturn;
			} catch(Exception e) {
				e.printStackTrace();
			}
			return null;
 		}

Open in new window

Avatar of SysExpert
SysExpert
Flag of Israel image

Not sure, but why not use a full text index and then use the search function if it exists in Java...

Avatar of David Robitaille

ASKER

Well, how does you do this in lotus script? I got a vague idea of what you are talking about, but I not a lotus expert and i hope to jump to another call when that webservice is done.
I will translate in java, I`m getting good a this....
Hi again,

Unfortunately, this is the way, if you want all the flexibility you can get. There are (at least) three ways of searching in Notes:
- Search, that's what you use, flexible but dead slow, because it does a linear search of the database without using any index
- FTSearch, very fast and less flexible, no formulas but a lot is still possible
- Views and GetDocumentByKey(), selection like Search, ultra-fast since indexed, but the selection is fixed; if you change the selection formula during runtime the speed is reduced to that of Search

GetDocumentByKey, or GetAllDocumentsByKey have a parameter that tells Notes to use an exact match or not; if not, then the key value (or key values) being looked for are matched at the start of the string in a specific column.

It seems to me that you could use one or more rather fixed views to search for your data. The GetDocumentByKey function's first parameter is an array of key values, each matching the corresponding column in a view.

Matching for a single starting letter seems no problem using the exact_match parameter set to false.

Now the parent problem. Two ways: your method if you don't want to copy data down the hierarchy all the time, or just that. It is rather common practice to copy data to the full hierarchy of documents if a view of only responses is required, or a sorted non-hierarchical view.

Does this help you a little?
The FT Search uses Select Statements, like you would in a View selection formula. It works quite well.
"Now the parent problem. Two ways: your method if you don't want to copy data down the hierarchy all the time, or just that. It is rather common practice to copy data to the full hierarchy of documents if a view of only responses is required, or a sorted non-hierarchical view."
Well, it s because the database in not made by someone in our company. It s a 3th party product and I`m trying to pull the data out of it as i discover it`s structure...
The data "should" be copied down the hierarchy, but it`s look like it does not be copied all the times...  There is a "Region" field within the "contact" document, but it look like they are not filled all the time, even if the contact is associated with the company in the lotus application.
I made the recursive getFieldFormParents search using ParentUNID, but it seem to not work verry well. I made it recursive because, theoricaly, company could have divisions. But  Debugging test just show me the field is populated from the getCompanyDocumentForContact(ContactDoc, ContactDB); it basically retrieve the "company document" using the "company name" field int the "Contact Document".
So, That company name field appear to be more reliable than those ParentUNID.

So here what i need
  • Contact Doc
    • UNID
    • FirstName
    • LastName
    • CompanyName
  • Company Doc
    • CompanyName
    • Region
When you are talking about "key" i always think about primary/foreign key type of thing. But i got a feeling it not what that. how could that help me? I started to work with lotus notes for about 2 week, and within one of theses weeks i was trying to make notesql work, so i m not that good at those search things.

 

private Document getCompanyDocumentForContact( Document ContactDoc, Database ContactDB){
		String CompanyName;
		DocumentCollection searchResults;
		try{	
			CompanyName = getField(ContactDoc, "CompanyName");
			searchResults = ContactDB.search("Form=\"Company\" & CompanyName=\""+CompanyName+"\"", null, 0);
			return searchResults.getFirstDocument();
		} catch(Exception e) {
			e.printStackTrace();
		}
		return null;
	}	

Open in new window

I think i put the finger on something.
i just saw a CompanyUnique field on the Contact Document. but for some reason:
that work (but it`s too slow)
CompanyUnique = getField(ContactDoc, "CompanyUnique");
searchResults = ContactDB.search("Form=\"Company\" & CompanyUnique=\""+CompanyUnique+"\"", null, 0);
return searchResults.getFirstDocument();

and that dont work (but it`s fast enough)
   CompanyUnique = getField(ContactDoc, "CompanyUnique");
   return ContactDB.getDocumentByUNID(CompanyUnique);
????
ok, CompanyUnique look like this : "BMIH-4RTS3C"
and UNID look like this : "98D8137851C21D2D852569A7006DAE04"
How could i use those CompanyUnique field????
 
Maybe there is a view that's ordered by the CompanyUnique field? So you can use a GetDocumentByKey there?

The unique-ID that's known as UNID (it actually means Universal ID) is used to uniquely identify documents within replicas of one database. That id is guaranteed to be unique across replicas. The hidden field $Ref contains the parent's UNID. If a document has a field $Ref, it is a response document, and then you should be able to follow the hierarchy up to the root.

Sadly, things can go wrong. What could have happened in your database? Someone deleted a parent document and forgot the responses. They are not automatically deleted, and they become orphan documents in the database. In a hierarchical view, you'll never see them again, unless something attaches them to a new parent. In a flat view though, that selects that particular type of documents, you can still see them.

Now, if they made a relationship between parent and child using the CompanyUnique field, and the field exists on both levels, you should be able to use that to find the company in a view. By the way, that unique key is generated by the formula function @Unique .

> ... but it`s too slow ...

That's what I said: Search is slow; it can be used instead of a view, e.g. in background/nightly agents that run only once per day; it is more efficient to do a Search than to maintain the asdditional view's indexes all day.

For your purpose, cannot you find a way to use several views for rapid searching?

@johnjardin:
> The FTSearch uses Select Statements
No, it does not. FTSearch has an entirely different syntax. But it IS very quick indeed.
"Now, if they made a relationship between parent and child using the CompanyUnique field, and the field exists on both levels, you should be able to use that to find the company in a view"
I will go in that trail, but i m losing myself in those views... the app have about 140 views...
  1. How could i find the correct view, what should i check?
  2. How will i use a view to get the document?
Or could i use FTSearch to fing the company faster ; the folowing code works well in a search.

CompanyUnique = getField(ContactDoc, "CompanyUnique");
searchResults = ContactDB.search("Form=\"Company\" & CompanyUnique=\""+CompanyUnique+"\"", null, 0);
return searchResults.getFirstDocument();
 
 
ASKER CERTIFIED SOLUTION
Avatar of Sjef Bosman
Sjef Bosman
Flag of France image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
i m with R7
I got that error installing DDSearch
IBM Lotus Notes 6 is installed as a MultiUser configuration on this system. Because a MultiUser configuration excludes IBM Lotus Domino Designer 6, you cannot install DDSearch for IBM Lotus Domino Designer 6 in this configuration.
but, How will i use a view to get the document?

ok, i think i got it, i found a view called "companybyname"
i found the way to extract the Document, seem fast enougn but need more test, giving new shortly...

private Document getCompanyDocumentForContact( Document ContactDoc, Database ContactDB){
		String CompanyName;
		View CompanyView; 
 
		try{	
			CompanyView = ContactDB.getView("companybyname");
			if (CompanyView == null){
				return null;
			}else{
				CompanyName = getField(ContactDoc, "CompanyName");	
				return CompanyView.getDocumentByKey(CompanyName, true);
			}
		} catch(Exception e) {
			e.printStackTrace();
		}
		return null;
	}	

Open in new window

Thanks a lot on that one. I got something usable!
That's the error indeed. Here's the solution:

http://www-10.lotus.com/ldd/sandbox.nsf/Threads/91DBF96DE7BAB7998525719500554A64?OpenDocument

Nice that the solution fits your problem, more or less. I hope you can you do without all the fancy search possibilities.

Thanks for the grade!

PS Installing DDSearch should really work after the the registry change. If you have trouble with "DXL exporter failed" or something like it, look on the same Sandbox page for more help.
Well, again, Thanks a lot, on that one and on the others questions on domino/lotus you helped me with.
I hope i will not need your help anymore, it not because i dont like you, but i dont especially like that kind of database. I think i got enough knowedge for the needs of that project.
I just whant your advice on one last thing, that could lead on a other question. just answer yes or no. i may open a new question based on your advice.
To speed up things, there is a easy way to build a view that fits my needs :
  • Contact Doc  
    • UNID  FirstName  
    • LastName  
    • CompanyName
  • Company Doc  
    • CompanyName (for linking)
    • Region

       

You should not "like" or "dislike" databases, they are "things" and not worth being liked or disliked. :D

No, but maybe.

Sorry, it needs a little explanation.
If you think of creating a view with two types of documents, that can be done, but the relationship cannot really be fixed. What can be done is a view based on two types of forms, ordered by CompanyName, and then by form-type. It all depends on the ordering. A potential problem is contacts without company: they'd show up in the view but there is no "parent" company document in the view.
Well, in fact, I not that i dislike them, but that I dislike working with them. I feels like I m hitting screws with a hammer. Not that note is the wrong tool, but that I`m the wrong tool. In that case, you where my screwdriver!
Your explanation on my last interrogation gave me some light. on my quest for a correct view, I found a view named "All By Company Hierarchy" that reflect what you are describing. they solved the "no company" problem with code like this :
@If(Form = "Division" | (Form = "Contact" & DivisionName != ""); @If(CompanyName = "" ; "-- No Company" ; @Trim(CompanyName)) + "\\" + @Implode(@Trim(parentDivisionNames:DivisionName); "\\") ;
form = "Company" | (Form = "Contact" & CompanyName != ""); @Trim(CompanyName) ;
Form = "Contact" & CompanyName = "" ; "-- Individual"; "" )

Anyway, i was not able to use it to get what i want and I think it will be better, for now, to leave it the way it is.
So finally, I cannot say it enough, thanks a lot. you really helped me with those "things".
 
LOL

You're welcome!  A+