User Tracking system in asp.net

dprasad
dprasad used Ask the Experts™
on
Hi Experts,

We have a web application mainly used by Customer Case representatives during the call with customer.  Using this application Rep can search the customer details and make some communication between the users like creating service notes and assigning it to some user groups.  On top of this application we need a user tracking system. Probably we need to capture below information.

Potential data points to capture: (one entry for each page or postback?)
Session ID
Page name
URL (like page name, but includes all query string, etc)
User ID
Postback sender(?), what button/link was clicked
Timestamp
Notes (a free field where perhaps we can customize certain pages to log additional information if desired)
SearchType
SearchText
Server

One obvious concern is the size that this table could grow to if we are logging a row for essentially every user action.  It quite likely would be over a million entries a week (searches alone might be around 70,000 per week, which we already are capturing).  However, we want the data accessible in a way so that we could reconstruct a user’s series of actions if desired, while also being able to query by page name and build a list that way.

Note: we would probably want to write it out using the Message Queue.

Please provide some suggestions and views.

Thanks,
Mateti
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Purely from an architectural point of view:

The data storage can be a NoSQL database, like DynamoDB or Cassandra.
This has the benefit of easier scalability and better management for very large data sets.

Your idea of using a message queue as a buffer between the event collectors and the data storage is a good one. You could have used any queue implementation , even a log file on a file system, but a message queue scales much better and has additional advantages.

Author

Commented:
I did some R & D and found following options for user tracking
1)      IIS logs
2)      HTTP modules – begin and end request
3)      Sessions
4)      Membership
Each approach have some advantage and disadvantage. My requirement is I should be able to log various information, even custom log should be achievable. I am ok if I get the solution by combining more than one above approaches. Could you please suggest me which one is feasible and extendable?

Thanks in advance,
Mateti
The information you want is best collected from within the application itself.
If possible, add the tracking code to the web pages in a manner similar to google analytics: AJAX code that writes the tracking data to the tracking application collector.
The tracking application will then periodically process the data and make it available to your analysts.

Unfortunately IIS logs are quite limited and IMHO writing custom http processing modules either for request processing or for session capture is both risky and inefficient compared to the alternative.

Author

Commented:
Can you through some light on your answer please.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial