Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

Implementing a java database work queue

Posted on 2007-11-16
3
Medium Priority
?
1,642 Views
Last Modified: 2012-05-30
We currently have two systems.  The first system is responsible for processing incoming data.  When it's processing is complete, the result is a record in a database table .  The second system is responsible for picking up the completed work and forwarding the data to it's recipient (via a number of different methods).  The record is then removed from the queue table and placed into a log.  Currently the systems communicate via a JMS queue.  When the first system is done with it's work, it sends a message to a JMS queue on the second server.  I'd like to remove this dependancy.  There are often times when the second system is down and thus never receives the JMS message or when we don't want the message processed immediately but at a later date and time.  We also experience some reliability problems when the volume of incoming data is exceptionally high.  I'd want to implement a process on the second system that reads the database table for records that are marked as ready to process.  There will be several hundred of these a minute.  I'm at a complete loss as to how to implement this.  Can anyone point me to a design pattern or example?  I obviously need to process a fairly large number of transactions in a timely manner.  The time to process each transaction varies greatly depending on who the data is being sent to and how they are receiving it.  Some transactions complete in microseconds, while others can take several minutes.  I also need to ensure that each database record is only processed once.

Also, we are using Oracle and BEA Weblogic.

Thanks.
0
Comment
Question by:trudyhlittle
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
3 Comments
 
LVL 13

Accepted Solution

by:
Bart Cremers earned 1000 total points
ID: 20303730
Using a JMS queue for triggering the DB reader is a common used why, which could be made quite reliable. Normally, as long as the JMS server is running it will be possible to make sure a message stays on the server until it is stopped.
Postponing the processing can be done through pausing the consumers on a queue, but it depends on the JMS system you use if this is possible.

The only alternative way of doing this without the dependency is writing your own thread checking database changes on a regular interval. But as you probably know, creating threads is prohibited in a J2EE environment.

But in J2EE there's a solution to this. You can creating custom worker threads using the AppServers WorkManager. Here are some articles describing this for a BEA Weblogic environment:

http://dev2dev.bea.com/pub/a/2005/05/parallel_tasks.html?page=1

http://dev2dev.bea.com/pub/a/2006/01/workload-management.html?page=1
0
 

Expert Comment

by:vvux
ID: 38026449
So basically, no explicit solution, just bla bla bla.
And the two links don't work !

The user asks:
"I'd want to implement a process on the second system that
reads the database table for records that are marked as ready to process.
Can anyone point me to a design pattern or example?"

Solution:
The only alternative way of doing this without the dependency
is writing your own thread checking database changes on a regular interval

This is the best solution I've ever seen. Great !
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

After being asked a question last year, I went into one of my moods where I did some research and code just for the fun and learning of it all.  Subsequently, from this journey, I put together this article on "Range Searching Using Visual Basic.NET …
In this post we will learn how to connect and configure Android Device (Smartphone etc.) with Android Studio. After that we will run a simple Hello World Program.
Viewers learn how to read error messages and identify possible mistakes that could cause hours of frustration. Coding is as much about debugging your code as it is about writing it. Define Error Message: Line Numbers: Type of Error: Break Down…
This tutorial covers a step-by-step guide to install VisualVM launcher in eclipse.
Suggested Courses

610 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question