Need to automate activity log export.


I have searched the channel for a similar question but did not find one.  I have a requirement to send Notes activity data to an external repository. Not being a Notes person, I have not been able to find much on how to do this.  Would the best solution be to write an agent that is somehow fired by the log and have the script then read and send the last entry of the log.nsf?  Or should I write vb or script using COM to interface with the log.nsf file.  Either way, I am not sure how to do that (or if it even can be done) but I guess I'll figure it out.  I can not just export the log.nsf manually.  This has to be automated.  Any suggestions? Can anyone steer me in the proper direction?

Who is Participating?
Replication>>Once a minute might kill the server<< nah, not for a couple of applications.  We had a couple that we replicated every five minutes because otherwise we'd end up with replication conflicts.   Never did every minute :)  It probably wouldn't finish replication.  We had to extend the time to allow replication to complete, too.

The issues with the logs in log.nsf is that they are text embedded events on a single form.  Being this, the field: BODY is what you have to mine, and in order to get any relevant answers from either the misc or mail events.  If you're not interested in grabbing specific mail or events it's easy enough to mail/export all the logs for a particular day via an agent.

So, I'm not sure what you're trying to mine here.

If not for that, using DECS to automatically grab and send to SQL via odbc would be the easiest, but then you'd need SQL/MYSQL/DB2 server to import as BLOB, and you won't have any meaningful results, unless you use a view for the ODBC collection.

There are these tools to search for specific items in the logs:,log,log

Either one can be modified to email the results.

Ok choices - pushing out to a text file is possible, and I think an INI or configuration call. But hard on system.

I like replicate to a local machine and mine from there.  When I had to mine the log, I mined from a LOCAL machine, this way I didn't use server resources.   GSX does this, and creates it's own reports so you can filter yourself using the product.

All will leave a footprint.  For instance, if I mined the logs to see how many times Joe Smith accessed the server, my "reads" will show up in the log.

the only way you're going to get real time results is if you mine via ODBC to another database, or maybe push the logs out to a text file.
Sjef BosmanGroupware ConsultantCommented:
More questions for you:
- we are talking Domino, aren't we, not Notes?
- what Domino release?
- what are the "activity data" you need?
- when do you need it, I mean every hour or day, or on command?

You're correct probably with the agent, but first we have to investigate the whole problem.
kmcclintonAuthor Commented:
<<<< - we are talking Domino, aren't we, not Notes? >> 

Yes of course, Domino, but you knew that didnt you.....

<< what Domino release?>>
Not to sure at the moment...I have to go back to the lab to see. I believe it's 6.5 (that may be the client install). I will get back to you.

< what are the "activity data" you need?>

Just about everything. User activity, smtp, etc. I don't need server startup and normal op. activity but if they are in same log, I will just filter.
<<- when do you need it, I mean every hour or day, or on command?>>
Realtime or close to it. which is why I thought if I could fire code and send it.....

Thanks for you help.

Free Tool: IP Lookup

Get more info about an IP address or domain name, such as organization, abuse contacts and geolocation.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Sjef BosmanGroupware ConsultantCommented:
Realtime? Nothing in Domino is realtime. So you are going to put a considerable load on a system when you start polling the database every minute or so?

What is so important that you want to see it before Domino can even warn you through its Events?

Could SNMP or an SNMP agent be of some help??
kmcclintonAuthor Commented:

This is a highly secured network.  SNMP will not be running on ANYTHING on this network. Not firewall,  router, server..nothing.  I am looking for a way to get Domino activity for various reasons but SNMP is not a solution that will be approved.


kmcclintonAuthor Commented:
When I say accepted, I mean accepted by the "powers that be". Personally, SNMP would be fine but I know I could not get that accepted.

Sjef BosmanGroupware ConsultantCommented:
Yes, I think I know how ro pull this one off. But, the cost of it can be very high. That's up to you and your "powers" to decide.

The general idea you can find here:

It would be to immediate answer to you problem, if it were not for the fact that "Trigger Happy never responds to events in the Trigger Happy Config database or the Log.nsf database. This is necessary for normal operation of Trigger Happy." And "Log.nsf events aren't logged either, because they can get into an infinite loop, where a message is logged, which triggers an event, which logss a message, which triggers an event. ad nauseum."

This means that you'd have to create the DLL's yourself. A lot of tricky work.

Maybe some external tool can do the job for you. If the keywords are highly, secure, Domino, I can smell the bucks underneath ;)

Solutions I found are best described and compared here:
kmcclintonAuthor Commented:
Thanks sjef for all your help. Unfortunately, there is too much code in this solution.  I am able to use a small bit of code to integrate but using too much will send off whistle to the government.  If I used this, it will take another 6 months of evals to get the project through (each and every time it is used). I have to come up with a cleaner way of doing this. I can use COTS solutions but if i have to compile anything, that would hold everything up while every line of code is inspected.  Am I asking for something that is not readily available or done often?

Sjef BosmanGroupware ConsultantCommented:
You have some pretty tough constraints there! Phew!

Did you have a look at GSX solution? It may be everything you need. And if you don't say what the external repository is... Maybe GSX have a good solution, off the shelf. Call them?
kmcclintonAuthor Commented:
Thanks sjef

Yes, the project is not a run-of-the-mill Domino integration. The requirements are VERY tight.  It's looking more like I have to write something to do this.  Thought  I'd give the Experts a try to see if something less painful could be done (that i could use of course). Considering my Notes App knowledge goes as far as just knowing how to CREATE a database (not designing an app) or replicate my mailbox. This is going to be a challenge that i actually am looking forward to. I wouldnt be me if I shyed from this sort of stuff.  Anyway, if you or anyone else, have a leads feel free to add it here. I won't close this as of yet. Unless you don't think I'll get a hits on it.

Sjef BosmanGroupware ConsultantCommented:
I think you won't get many hits, but one never knows.

Judging from your "specifications", you need the following:
- create a Domino Extension, using C and the C-API
- a spare Domino server to develop and test it on
- a lot of time (3 months??)

At least, that's how I would do it. And that's how GSX did it, most likely.
kmcclintonAuthor Commented:
I think I will use Java and a API.  Do you think I won't get many hits because of the nature of the question, how it was phrased, or just not enough who would be interested in it?
Sjef BosmanGroupware ConsultantCommented:
There are less than 10 experts who post regularly, less than 25 who post accasionally, and the rest have all gone. The die-hards will read your question. I hope they have different views! It's not the question itself, although the complexity is making it very difficult.

Java? And COM therefore, or CORBA? Which means you will be polling the server for information. How often are you planning to do that?? Bound to fail, I'm afraid.

IMHO, the only feasible solution is a Domino Extension, a DLL that is called by the server itself before and/or after modifications are made to a database. When the DLL is started, several events can be activated by means of callback functions. Realer time you can't get.

I'll ask an EE-colleague to look into this :)
I can not think of any good solution , but here are some ideas.
1) Set up a separate Domino server and replicate the log file to it. Use that server to do all your log processing, rather than your main server. It will not be realtime, but it will probably not cause your main server to grind to a halt either.

2) Look into the Unix/Linux Domino servers. maybe there is a way to get the log.nsf into one of the Unix system logs. there are plenty of tools that can do what you want on Unix.

3) Use C-API and do it yourself.
The bottom line is how fast do you really need the log info ?
The faster you want it, the more it will affect server performance - probably - unless you have serious Hardware overkill in place.

Uou should check the log.nsf  file, it can create thousands of entries per minute, so you have to decide what you really want.

I hope this helps !
kmcclintonAuthor Commented:
<< Java? And COM therefore, or CORBA? Which means you will be polling the server for information. How often are you planning to do that?? Bound to fail, I'm afraid>>>>

wow.... be more optimistic why don't you

Gents and if not both Gents .. I apologize.

Have you ever heard of domingo? I just found this. It's suppose to be a Javascript-like Java API to Notes/Domino.  I see there are some interesting methods in the Javadoc that I will play with. This might be the way to go.  I hate C++ (bad memories) and will try to avoid it.   I think whatever I come up with I will do what you both have said.  I will write the code to extract the data from the database, comparing this with what is in whatever external repository I may use.  I think the replication idea is great (hadn't thought of that) and I think I will incorporate that into the test environment when (notice I said when) I get this firing. Hardware will not be an issue - we will run what we have to. But, I can not miss events so it may be best to replicate on a seprate server (something else I don't know how to do yet) and have the code poll maybe once a minute. I have to think about failure responses but.. getting ahead here...

 I am on vacation next week and am leaving in about 45 mins to get a jump start. So I won't touch much till next week. Boy I sure like not having to worry about work when I leave here. I couldn't work from home even if I wanted to. Nothing is allowed to leave here. So...(click) the brain switch was just turned off. Thanks for all you help and I will take this up again next week.

Sjef BosmanGroupware ConsultantCommented:
> It's suppose to be a Javascript-like Java API to Notes/Domino
Read again: LOTUSscript-like

> replication idea is great
Will be unreal-time: replication is usually done every 60 minutes on a normal server, but it can be set faster. How ofter replication is actually done depends on the server's load. Once a minute might kill the server.

The only way not to follow a train is to be on it.
Read the accepted answer on this page: http:Q_10152910.html "How to avoid logon password?"

Here's a long thread about the Extension Manager's capabilities:

Have a nice holiday! Eh, I forgot what you weren't supposed to think about...
kmcclintonAuthor Commented:
My apologies for not closing and awarding points.  I have spent most of my last few days in the lab working with the Java API and NotesSQL and have not been too my desk very often.   I appreciate the comments made but feel marilyng's were most informative. Regards,,
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.