Need architectural information on User Tracking information system

This is generally the sort of thing that needs to be included in the design specification before any code is created, since it affects every page entry algorithm in the system.  If I were in the position of having a web application that is already built I would consider two things.  The minimalist solution would be Google Analytics.  It will not give you all of the information you might want, but it is useful and is an industry standard.  The better solution might be to engage one of the advertising networks like DoubleClick.  They know as much as anyone about tracking online client behavior.

Articles of interest:
http://dotnetslackers.com/articles/aspnet/Tracking-User-Activity.aspx
http://en.wikipedia.org/wiki/Web_analytics

Best of luck with it (it may be a big project, but potentially very rewarding), ~Ray

alternatively, I recommend to use the web server logs.
A web server can log all of the resource requests (pages, css, images etc) and save all of the relevant information. The web site logs can then be imported to a web analysis software like AWStats, or to a data warehouse for slicing and dicing.

The architecture and concepts in this book http://www.amazon.com/The-Data-Webhouse-Toolkit-Web-Enabled/dp/0471376809 can be the basis for a BI system based on user experience.

The best solution for you boils down to what you really want out of the user tracking system.

I m not sure how extensively  you want to log the activities but there are some good options.

here are few options

1. You could write a Http Module which is called on each and every request to your application
2. you could use AOP (aspect oriented programming). Spring AOP framework or Policy Injection Block (MS Enterprise Library)
3. Writing a custom attribute

Hi Folks,

I think I have to redefine my question. PLease see the redefined question below.

We have a web application mainly used by Customer Case representatives during the call with customer.  Using this application Rep can search the customer details and make some communication between the users like creating service notes and assigning it to some user groups.  On top of this application we need a user tracking system. Probably we need to capture below information.

Potential data points to capture: (one entry for each page or postback?)
Session ID
Page name
URL (like page name, but includes all query string, etc)
User ID
Postback sender(?), what button/link was clicked
Timestamp
Notes (a free field where perhaps we can customize certain pages to log additional information if desired)
SearchType
SearchText
Server

One obvious concern is the size that this table could grow to if we are logging a row for essentially every user action.  It quite likely would be over a million entries a week (searches alone might be around 70,000 per week, which we already are capturing).  However, we want the data accessible in a way so that we could reconstruct a user’s series of actions if desired, while also being able to query by page name and build a list that way.

Note: we would probably want to write it out using the Message Queue.

Thanks,
Mateti

Hi Shalomc,

Thanks for your answer and time. Pease see my redefined question. I have to gather as much as possible. I did some R & D on IIS logging. It has advantages and disadvntages. 1 disadvantage I foiund is it we can not have custom logs in it. Here see my queries regarding IIS logging.

1) Can I configure IIS logging capabilities? Can i add or remove parameters to IIS logging list? If yes how?
2) If I can not add parameters to IIS logging list- I will do logging in 2 steps. 1) I will use IIS logging and I will do my own logging methodology for custom log information. But here the challenge is how to sync both of these logs? If you have any idea let me know.


If I can sync both cutom logging and IIS logging it would give a best feasible solution.

Thanks in advance,
Mateti

Hi SriVaddadi,

Thanks for information could you please provide some more information on below queries and at the same time please see my revised query.

1) If I use HTTP modules how much data I can push in to my log store? what are the parameters I can log here. please provide the process of logging when I use http modules.

2) Could you please provide some more information on solution 2. because I am very new to all these terms.

3) Please throw some more light on solution 3 also.

Thanks,
Mateti

IIS up to version 6 has limited logging, as compared to apache. For example, you can't log the cookies.
This is an explanation of IIS logging in IIS 4, 5 and 6.
support.microsoft.com/kb/313437

And this is IIS 7
http://learn.iis.net/page.aspx/579/advanced-logging-for-iis---custom-logging/


To have a combined custom and IIS log you must have something to correlate them.
The best thing is to correlate by a page-unique cookie and the time of request.

If you are really interested to invest some time and effort in order to deliver the best solution, take a look at the methodology described in this book

http://www.amazon.com/The-Data-Webhouse-Toolkit-Web-Enabled/dp/0471376809

Hi dprasad,

Appreciate that you accepted my solution but i wondered why you graded as "C".