Solved

Forcing Quartz Scheduler to maintain jobs in “job store” even in the case of failed jobs and having them re-run at the specified interval

Posted on 2012-03-25
8
2,078 Views
Last Modified: 2012-04-16
I'm using the following Quartz Scheduler Task object to run my jobs as follows. For some reason which I think lies in the catch clause, after a series of unsuccessful attempts, my jobs stop rescheduling and they would eventually get wiped out of the job store and at some point freeze the application. Does anyone know how to make this Quarts scheduler run the jobs only once per assigned intervals so that it does not either drop the job or attempts to refire? I used refire immediately since there was no other better handling available upon job failure. How can I assure my jobs remain in the jobs store even when they fail at times and not have to loose them. Could anyone with Quartz experience if I'm going wrong somewhere in this code?

Many Thanks

package IMP;

import IMP.resources.NewPortalConnection;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.util.ArrayList;
import java.util.Date;
import org.apache.http.NameValuePair;

import java.util.List;
import org.quartz.DisallowConcurrentExecution;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.quartz.Job;
import org.quartz.JobDataMap;
import org.quartz.JobExecutionContext;
import org.quartz.JobExecutionException;
import org.quartz.JobKey;
import org.quartz.PersistJobDataAfterExecution;

/**
 * 
 */
@PersistJobDataAfterExecution
@DisallowConcurrentExecution
public class Task implements Job {

    // parameter names specific to this job
    public static final String DATA_SOURCE_NAME = "data source name";
    public static final String DSN_USERNAME = "dsn username";
    public static final String DSN_PASSWORD = "dsn password";
    public static final String DMZ_USERNAME = "dmz username";
    public static final String DMZ_PASSWORD = "dmz password";
    public static final String DMZ_HOST = "dmz host";
    public static final String DMZ_PORT = "dmz port";
    public static final String QUERIES = "queries";
    public static final String IMPVIEW = "IMPview";

    String FTP_URI;
    String FTP_INITIAL_DIRECTORY;
    List<NameValuePair> ftpParams;
    private static Logger _log = LoggerFactory.getLogger(Task.class);

    /**
     * Empty constructor for job initialization
     */
    public Task() {
    }

    /**
     * <p>
     * Called by the <code>{@link org.quartz.Scheduler}</code> when a
     * <code>{@link org.quartz.Trigger}</code> fires that is associated with
     * the <code>Job</code>.
     * </p>
     * 
     * @throws JobExecutionException
     *             if there is an exception while executing the job.
     */
    public void execute(JobExecutionContext context)
            throws JobExecutionException {

        JobKey jobKey = context.getJobDetail().getKey();
        JobDataMap jobDataMap = context.getJobDetail().getJobDataMap();

        String dataSourceName = jobDataMap.getString(DATA_SOURCE_NAME);
        String dsnUserName = jobDataMap.getString(DSN_USERNAME);
        String dsnPassword = jobDataMap.getString(DSN_PASSWORD);
        String dmzUserName = jobDataMap.getString(DMZ_USERNAME);
        String dmzPassword = jobDataMap.getString(DMZ_PASSWORD);
        String dmzHost = jobDataMap.getString(DMZ_HOST);
        String dmzPort = jobDataMap.getString(DMZ_PORT);
        IMPView view = (IMPView) jobDataMap.get(IMPVIEW);

        view.statusMessageLabel.setText("Running Portal: " + dmzUserName);
        view.statusMessageLabel.repaint();


        try {

            EstablishDMZConnection dmz = new EstablishDMZConnection();

            dmz.userName = dmzUserName;
            dmz.password = dmzPassword.toCharArray();
            dmz.host = dmzHost;
            if (dmzPort != null) {
                dmz.port = dmzPort;
            }

            Connection conn = null;
            conn = NewPortalConnection.getConnection(dataSourceName, dsnUserName, dsnPassword);

            ArrayList<Query> queries = (ArrayList) jobDataMap.get(QUERIES);

            for (int i = 0; i < queries.size(); i++) {

                PreparedStatement pStmt = conn.prepareStatement(queries.get(i).query);
                ResultSet rs = pStmt.executeQuery();

                NewPortalConnection.generateXMLFile(rs, queries.get(i).query_save_name);
                ftpParams = dmz.fetchFTPSettings();

                for (int j = 0; j < ftpParams.size(); j++) {
                    if (ftpParams.get(j).getName().equals("FTPURI")) {
                        FTP_URI = ftpParams.get(j).getValue();
                    }
                    if (ftpParams.get(j).getName().equals("FTPInitialDirectory")) {
                        FTP_INITIAL_DIRECTORY = ftpParams.get(j).getValue() + "/" + dmzUserName;
                    }
                }
                UploadXML.ftp(FTP_URI, dmzUserName, dmzPassword, FTP_INITIAL_DIRECTORY, queries.get(i).query_save_name);
            }

            conn.close();

        } catch (Exception e) {
            _log.info("--- Error in job!");
            JobExecutionException e2 = new JobExecutionException(e);
            e2.refireImmediately();

        }
        _log.info("SimpleJob says: " + jobKey + " executing at " + new Date());
    }
}

Open in new window

0
Comment
Question by:snajalm
  • 5
  • 3
8 Comments
 
LVL 47

Expert Comment

by:for_yan
ID: 37763116
Sorry, I used to use Quartz some time ago, but I used to use Sheduler, SchedulerFarctory, triggers etc.
I see nothing of the kind in your code.
Is it some other  option in Quartz that you are using ?
0
 
LVL 47

Expert Comment

by:for_yan
ID: 37763128
I'm sure you raed this
http://stackoverflow.com/questions/4408858/quartz-retry-when-failure

and there is a comment down there that this can really freze some of the Quartz thread - see undeder the accepted solution

If you say that you don't care about retrying until next time why would you use this refireImmediately() method which is prone to issues
0
 

Author Comment

by:snajalm
ID: 37763149
Thanks for_yan,

There is no other form of handling under the JobExecutionException object that would leave the job in the job store for its following turn initially set for that job.  The is option ofr immediate execution or dropping the job.  I'm wondering if I leave the exception clause empty, would it leave the job for its next scheduled execution?  What do you think?
0
 
LVL 47

Expert Comment

by:for_yan
ID: 37763173
well, I frankly didn't encounter that i  my epsisodic interactions with Quartz,, but if I were writing Quartz, I would have lefty it for next scehdeuled execution - suppose you connect to some host, at some pont it does not respond, but next time it would, why to drop it without special command
0
What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

 
LVL 47

Expert Comment

by:for_yan
ID: 37763186
I'd have tried  just writing error message into the log in the catch brace and doing nothing elese.
0
 

Author Comment

by:snajalm
ID: 37763202
I'll give that a shot and let you know how it went.
0
 
LVL 47

Accepted Solution

by:
for_yan earned 500 total points
ID: 37763769
I ran some tests - they are based on example1 and example6 of the Quartz-2 distiribution
download -
quartz-2.1.0.tar\quartz-2.1.0\examples\src\main\java\org\quartz\examples\example1

and

quartz-2.1.0.tar\quartz-2.1.0\examples\src\main\java\org\quartz\examples\example6

So according to my experiments, I did not see any effect of refireImmediately()
method; so whether you call it or not it does not refore immediately but does
fire next time according to schedule

However if you call setUnscheduleAllTriggers(true);  (it is commented out in the first code below) it will not fire it next time on schedule

I also think you need to throw e2; in the end (didn't see this in your code)

package quarztest;
import java.util.Date;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.quartz.Job;
import org.quartz.JobExecutionContext;
import org.quartz.JobExecutionException;

/**
 * <p>
 * This is just a simple job that says "Hello" to the world.
 * </p>
 *
 * @author Bill Kratzer
 */
public class HelloJob implements Job {

    private static Logger _log = LoggerFactory.getLogger(HelloJob.class);
    String s = "abc";

    /**
     * <p>
     * Empty constructor for job initilization
     * </p>
     * <p>
     * Quartz requires a public empty constructor so that the
     * scheduler can instantiate the class whenever it needs.
     * </p>
     */
    public HelloJob() {
    }

    /**
     * <p>
     * Called by the <code>{@link org.quartz.Scheduler}</code> when a
     * <code>{@link org.quartz.Trigger}</code> fires that is associated with
     * the <code>Job</code>.
     * </p>
     *
     * @throws JobExecutionException
     *             if there is an exception while executing the job.
     */
    public void execute(JobExecutionContext context)
        throws JobExecutionException {

        // Say Hello to the World and display the date/time

             try{
        _log.info("Hello World! - " + new Date());
         System.out.println("Hello World! - " + new Date());
        System.out.println(Integer.parseInt(s));
             } catch(Exception e){
                 System.out.println("exception; string is not good " + s);
              //   s="5";
               JobExecutionException e2 = new JobExecutionException(e);
               // e2.setUnscheduleAllTriggers(true);
                 throw e2;
             }
    }

}

Open in new window


package quarztest;
import static org.quartz.JobBuilder.newJob;
import static org.quartz.TriggerBuilder.newTrigger;
import static org.quartz.DateBuilder.*;

import java.util.Date;

import org.quartz.*;
import org.quartz.impl.StdSchedulerFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
 * This Example will demonstrate how to start and shutdown the Quartz
 * scheduler and how to schedule a job to run in Quartz.
 *
 * @author Bill Kratzer
 */
public class HelloSchedule {


    public void run() throws Exception {
        Logger log = LoggerFactory.getLogger(HelloSchedule.class);

        log.info("------- Initializing ----------------------");
         System.out.println("------- Initializing ----------------------");

        // First we must get a reference to a scheduler
        SchedulerFactory sf = new StdSchedulerFactory();
        Scheduler sched = sf.getScheduler();

        log.info("------- Initialization Complete -----------");
        System.out.println("------- Initialization Complete -----------");

        // computer a time that is on the next round minute
        Date runTime = evenMinuteDate(new Date());

        log.info("------- Scheduling Job  -------------------");
          System.out.println("------- Scheduling Job  -------------------");

        // define the job and tie it to our HelloJob class
        JobDetail job = newJob(HelloJob.class)
            .withIdentity("job1", "group1")
            .build();

        // Trigger the job to run on the next round minute
        Trigger trigger = newTrigger()
            .withIdentity("trigger1", "group1")
             .withSchedule(SimpleScheduleBuilder.simpleSchedule()
                 .withIntervalInMinutes(1)
                 .repeatForever())
            .startAt(runTime)
            .build();

        // Tell quartz to schedule the job using our trigger
        sched.scheduleJob(job, trigger);
        log.info(job.getKey() + " will run at: " + runTime);

        // Start up the scheduler (nothing can actually run until the
        // scheduler has been started)
        sched.start();

        log.info("------- Started Scheduler -----------------");
           System.out.println("------- Started Scheduler -----------------");

        // wait long enough so that the scheduler as an opportunity to
        // run the job!
        log.info("------- Waiting 65 seconds... -------------");
            System.out.println("------- Waiting 200 seconds... -------------");
        try {
            // wait 65 seconds to show job
            Thread.sleep(200L * 1000L);
            // executing...
        } catch (Exception e) {
        }

        // shut down the scheduler
        log.info("------- Shutting Down ---------------------");
          System.out.println("------- Shutting Down ---------------------");
        sched.shutdown(true);
        log.info("------- Shutdown Complete -----------------");
          System.out.println("------- Shutdown Complete -----------------");
    }

    public static void main(String[] args) throws Exception {

        HelloSchedule example = new HelloSchedule();
        example.run();

    }

}

Open in new window


Output ( it still calleed it three times):

log4j:WARN No appenders could be found for logger (quarztest.HelloSchedule).
log4j:WARN Please initialize the log4j system properly.
------- Initializing ----------------------
------- Initialization Complete -----------
------- Scheduling Job  -------------------
------- Started Scheduler -----------------
------- Waiting 200 seconds... -------------
Hello World! - Sun Mar 25 16:31:00 PDT 2012
exception; string is not good abc
Hello World! - Sun Mar 25 16:32:00 PDT 2012
exception; string is not good abc
Hello World! - Sun Mar 25 16:33:00 PDT 2012
exception; string is not good abc
------- Shutting Down ---------------------
------- Shutdown Complete -----------------

Open in new window


No I uncommented e2.unscheduleAllTriggers(true);
like that:

package quarztest;
import java.util.Date;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.quartz.Job;
import org.quartz.JobExecutionContext;
import org.quartz.JobExecutionException;

/**
 * <p>
 * This is just a simple job that says "Hello" to the world.
 * </p>
 *
 * @author Bill Kratzer
 */
public class HelloJob implements Job {

    private static Logger _log = LoggerFactory.getLogger(HelloJob.class);
    String s = "abc";

    /**
     * <p>
     * Empty constructor for job initilization
     * </p>
     * <p>
     * Quartz requires a public empty constructor so that the
     * scheduler can instantiate the class whenever it needs.
     * </p>
     */
    public HelloJob() {
    }

    /**
     * <p>
     * Called by the <code>{@link org.quartz.Scheduler}</code> when a
     * <code>{@link org.quartz.Trigger}</code> fires that is associated with
     * the <code>Job</code>.
     * </p>
     *
     * @throws JobExecutionException
     *             if there is an exception while executing the job.
     */
    public void execute(JobExecutionContext context)
        throws JobExecutionException {

        // Say Hello to the World and display the date/time

             try{
        _log.info("Hello World! - " + new Date());
         System.out.println("Hello World! - " + new Date());
        System.out.println(Integer.parseInt(s));
             } catch(Exception e){
                 System.out.println("exception; string is not good " + s);
              //   s="5";
               JobExecutionException e2 = new JobExecutionException(e);
               e2.setUnscheduleAllTriggers(true);
                 throw e2;
             }
    }

}

Open in new window


and it executed it only once.
This is the outpiut:
log4j:WARN No appenders could be found for logger (quarztest.HelloSchedule).
log4j:WARN Please initialize the log4j system properly.
------- Initializing ----------------------
------- Initialization Complete -----------
------- Scheduling Job  -------------------
------- Started Scheduler -----------------
------- Waiting 200 seconds... -------------
Hello World! - Sun Mar 25 16:40:00 PDT 2012
exception; string is not good abc
------- Shutting Down ---------------------
------- Shutdown Complete -----------------

Open in new window


So, still don't know how to refire...,
but at least you should be safe to expect that if you don't unschedule
it will make a try next time also
0
 

Author Closing Comment

by:snajalm
ID: 37851918
Thanks for_yan :-)
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

For beginner Java programmers or at least those new to the Eclipse IDE, the following tutorial will show some (four) ways in which you can import your Java projects to your Eclipse workbench. Introduction While learning Java can be done with…
Java Flight Recorder and Java Mission Control together create a complete tool chain to continuously collect low level and detailed runtime information enabling after-the-fact incident analysis. Java Flight Recorder is a profiling and event collectio…
Viewers learn about the “for” loop and how it works in Java. By comparing it to the while loop learned before, viewers can make the transition easily. You will learn about the formatting of the for loop as we write a program that prints even numbers…
Viewers will learn about the regular for loop in Java and how to use it. Definition: Break the for loop down into 3 parts: Syntax when using for loops: Example using a for loop:

760 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now