• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1197
  • Last Modified:

Transaction batching with message driven beans

Hi,

We have the following requirement on our current project:

We have an MQ setup that will receive around 17-18 millon txns/day. There will be multiple instances of Message driven beans configured to listen to the queue. The requirement is to break the number of incoming transactions into small chunks or unit of work.  The unit of work in this case could either be 'X' number of messages ( For e.g 100 messages) in a single batch or a predefined time interval (For e.g. 10 millisecond) whichever happens first. I need my MDB to be configured such that each instance processes a batch of messages as one transaction. Each batch is then stored in the Database for further processing.

Kindly let me know on how this custom batch framework can be implemented.

Websphere MQ 7.1.0/WAS 7.0/EJB 3


Thank you!
0
Ramy0413
Asked:
Ramy0413
  • 4
  • 3
1 Solution
 
Radek BaranowskiFull-stack Java DeveloperCommented:
I'm not sure how it exactly works, but when going to connection factory settings in WebSphereAppServer , one can see options like on the screenshot. the parameter as below:

Maximum batch size

The maximum number of messages that can be removed from a queue before at least one must be delivered to an asynchronous consumer.
Data type       Integer
Default       10
Range       A value greater than zero.


should be responsible for what you want to achieve, provided you set up you MDB application properly.


hope this helps,
2012-08-14-screen-001.gif
0
 
Ramy0413Author Commented:
Thank you for the response.  Please let me know if my understanding is correct.

1. If for e.g. I set my maximum batch size as 10, does that mean the onmessage() method in my MDB is called only once after all the 10 messages have been put in the queue. How does that change my processing and handling in my onmessage() method of the bean?

2. Can I also configure the batch size at an individual queue level?

3. Does the polling interval mean that the MDB will wait for 'x' milliseconds or maximum batch size before it can process the message?

It would be really helpful if you could provide more information on the above queries.


Thanks!
0
 
Radek BaranowskiFull-stack Java DeveloperCommented:
for 1. I can't really tell. I've never developed myself that sort of application. my assumption is made on my WAS knowledge.

for 2. no, I don't think so - I went through config options and did not see that possibility. that might have something to do with actual point of transaction and config object role - queue is an WAS object definition describing MQ object, whereas Connection Factory actually is responsible for creating runtime resource eg. mq connection to specified queue. hence, max batch size is not queue-specific but rather connection-specific and is governed by connection factory. I suppose you may simply setup multiple CFs and map them to different applications, using different queues and that way use different batch sizes

3. I'd say so. I understand it as follows: mdb listens to messages on several queues, say, we have q1 and q2. we start to listen on q1. but no messages have arrived within 5s (default), so we check q2. we see 15 messages there, so we receive 10 of them in first batch, then 5 and wait 5s for next to arrive, if they do appear they are added to second batch up to 10, if none arrive i guess second batch is "closed" after 5 seconds, processed by mdb but mdb moves back to q1 to check if any new messages arrived and so on and so on

hope this helps!
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
Ramy0413Author Commented:
Thanks again for the response. I have one last clarification. On my third point what happens if I have only 1 queue? This is actually my requirement. I will have multiple MDB instance listening to only 1 queue and I would like to batch my transactions based on number of messages or say at a fixed interval of time. Will the above logic be applicable?


Thank you!
0
 
Radek BaranowskiFull-stack Java DeveloperCommented:
well, in that case i'm not sure how the concurrency between multiple MDB instances would influence the whole picture. I wonder...perhaps when you have high load of 20mi transactions/day (that would account for ca. 250 txs PER SECOND. that is quite a number :D) your queue connections won't have any concurrency problem with correct weighing of polling interval and max batch...how many instances are you planning, and what will be their expected throughput ? say, instance1 gets batch1 of 50messages to process, how soon will it "return" to get another batch ?
0
 
Ramy0413Author Commented:
I am probably looking at around 20 instances of each MDB and I also have 4 servers. So based on your earlier comment so I would still be able to use the max batch and polling interval to achieve what I am looking for. Please let me know if this is correct.


Thank you!
0
 
Radek BaranowskiFull-stack Java DeveloperCommented:
yes, I think so. you will need to balance your backend throughput (how fast mdb processes it's batch) against the number of instances and batch size.
good luck.
0

Featured Post

VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

  • 4
  • 3
Tackle projects and never again get stuck behind a technical roadblock.
Join Now