JobServer Feature Summary
Feature Description Pro Standard
System Architecture
Architecture The main JobServer scheduling and processing engine runs in a dedicated Java VM(s). The supporting GUI applications are web based and integrate closely with the main JobServer engine. See JobServer architecture diagrams: Download diagrams of JobServer's architecture and deployment configuration options. Yes Yes
Database Transaction Reliability All critical operations, managed by JobServer, from the scheduling to the job processing are managed and organized in a relational database. All major functions are backed by database transaction level reliability to insure integrity and consistency. Yes Limited
Database Support JobServer Professional has support for Oracle, MYSQL, and PostgreSQL. JobServer Standard edition comes bundled with its own built-in HSQL database. JobServer Pro offers greater scalability in processing and management of jobs. Both Pro and Standard editions share many of the same set of features, but the Pro edition offers greater scalability, reliability and durability. Yes No
Mesos Cluster Processing JobServer integrates closely with Mesos clustering to allow jobs to be processed across a Mesos cluster of any size and capacity. Partitions/Queues in JobServer can be associated to a Mesos cluster for highly scalable distributed processing and leveraging the full power of Mesos. No JobServer software has to be installed on the Mesos slave nodes. Connect JobServer to Mesos and watch your jobs run with uber scalability! Yes Yes
Agent Clustering and Distributed Processing JobServer can be deployed on one or many servers. Job processing can be distributed across a cluster of servers allowing for highly scalable distributed processing and hot failover of job processing and job scheduling. You can easily scale JobServer to run on tens or hundreds of servers and spread your job processing load across a high capacity cluster. Yes No
Failover JobServer will detect failover on the primary scheduling engine and have the secondary server take over control and continue job scheduling and processing. Yes No
Web-based Interface All JobServer tools/applications are web based and also utilize the GWT (Google Web Toolkit) AJAX framework for SDK plugin extensibility and customization. Sample JobServer GUI screen shots. Yes Yes
Security and Authentication JobServer supports SSL/TLS for web based authentication. All user passwords and configuration passwords are encrypted. Yes Yes
LDAP and Active Directory JobServer has support for LDAP version 3 allowing integration with LDAP and Active Directory servers for centralized authentication management. Yes Yes
Job Scheduling
Advanced Scheduling Rules The JobServer Scheduling Engine leaves no scheduling rule out of your reach. Virtually, any repeating or one-off scheduling pattern that you can imagine can be represented using the JobServer Scheduling Engine.

As an example, here is a sample of scheduling rules that are supported:

  • Every 15 minutes starting on 1/1/2011 09:00AM and only between 10:00AM and 7:30PM and ending on 3/3/2012 5:00PM
  • Every other month starting on 1/1/2011 09:00AM and on the last, first and second Monday and last Friday of the Month
  • Every other Week on Monday and Wednesday starting on 7/7/2011 except for Wednesday of 7/30/2011
  • Many more...........

  • Yes Yes
    Relative & Absolute Time Scheduling With this scheduling rule, a job can be run using "absolute scheduling" or "relative scheduling". Absolute scheduling means the job will run relative to the original start time, so if the job has a start time of 10:00 AM and the interval is one hour then the job will run at the top of every hour. With this same example, if relative scheduling is used, the job will still run every hour but it may or may not run at the top of every hour as it will run one hour from the last time it ran last. Yes Yes
    Skip Ahead Scheduling Rule This features defines what a job does if it falls behind in its schedule. Often times this feature does not come into play. But for example, if a job is set to run at 6:00PM on a given day but JobServer is down for 2 days due to say a hardware problem. When JobServer is started, the job will get kicked off, since its next run date has long past. Now this is when the Skip-Ahead feature comes into play. If this option is set, then after the initial run, the job will "skip" ahead to the next valid run date that is in the future. If it is not set, then next scheduled date will be used even if it is not in the future. The job will keep running until it encounters a next run date that is in the future again. Yes Yes
    Schedule Jobs One-at-a-Time This scheduling feature permits only one instance of a job to be running at any one time. This is useful in scenarios where you want to prevent more than one instance of a job from being scheduled at any one time. For example, you may have a job that runs every two minutes and in some rare cases, when the job might take 3 minutes to complete its processing you do not want the same job to run again. This feature prevents any other instance of the job from running until the current instance has completed. This feature can also be used in scenarios where you want a job to be running for extended periods of time and you only want to have one instance of the job to ever be running at any moment. So as an example, you might setup a job to run every 2 seconds and this job might be polling indefinitely (until killed) looking for certain events like a file being dropped in an FTP directory. In this scenario the job runs forever looking for such events. If it ever dies, due to a failure for example, the scheduler will start another instance. Yes Yes
    System Management
    Controlled System Shutdown and Startup JobServer has a variety of controls available to administrators for managing and operating the JobServer scheduler and job processing environment. Users can startup JobServer in several modes, such as "Gradual Startup". This startup mode, for example, allows you to startup the JobServer engine but suspending all job processing and job scheduling. You can then go into the JobServer system controls and turn on job scheduling and/or processing for the entire system or for one Partition at a time - it is all under your control. There are similar controls for shutting down the system. You can shutdown the system "immediately" (jobs that are running will be killed) or "gracefully" (all currently running jobs will run to completion then JobServer will shut itself down once all jobs are done and during this time no new jobs will be allowed to be run or be scheduled). There are other features similar to these as well that administrators will find useful. Yes Yes
    Administer System Wide Scheduling and Queuing Administrators can control the scheduling, queuing and job processing of the JobServer environment at a system wide level or at a Partition by Partition level. This allows users to shutdown, suspend and throttle different groups of jobs in a variety of ways. Yes Yes
    Event Triggers & File Transfer Jobs can be triggered when certain events happen like a file put into a directory or a database record entered in a table. This feature allows you to create "monitoring jobs" that when they detect an event (like a file being placed in a directory) can then invoke other jobs to run, and also pass parameters to those jobs, based on the event that triggered them. Transferring files using FTP, SFTP and FTPS are provided using the Java APIs and through the open source SDK. Yes Yes
    Built-in Support for Batch/Shell Scripting JobServer comes with built-in support for executing Unix/Linux shell scripts and MS Windows batch scripts. This is provided by a Tasklet that comes pre-packaged with JobServer. The Tasklet allows job designers to configure arbitrary scripts to be run by JobServer. This scripting Tasklet uses JobServer's open source plugin SDK for building custom Tasklets and Jobs. Source code is provided for developers to customize if they choose. The scripting Tasklet provides many powerful features like the ability to store and save the script and output of the script for each job run and comes with a GUI interface. Yes Yes
    Variable Time Property (for ETL Jobs) This feature does not directly impact the scheduling rules of a job, but is instead passed to a job when it is run. The start and end offsets defines the amount of days that will be added/subtracted to the next run time and last run time, respectively. When a job is run, it has access to an effective date range (start time and end time values). The last run date is the end date and the next run date is the start date. Normally, this feature is used to make business decisions and for ETL types of jobs that need to know when they ran last because the next ETL run depends on the date/time the job ran last. For example, if you have a job that runs every day and it pulls data from a database, the start date and end date can be used to access data from the database over that given date range. This can be a very useful feature for ETL type processing applications. You also have the option of selecting a custom start and end dates for each run time. Yes Yes
    Job Management
    Point and Click Workflow Builder Jobs within JobServer are composed of multiple Tasklets (and an optional customizable GUI interface called a Weblet). A job can be composed of one or more Taskets that run as part of a workflow chain. Tasklets are modular pieces of code that execute and log their output to JobServer. When you assign a Tasklet to a job you choose the implementation of the Tasklet. You can use some of the built-in Tasklets or have a Java Developer write a custom one and add it dynamically to the JobServer environment. Tasklets within a job, can also share and pass data to each other when they run in a workflow chain. This allows for very advanced capabilities. The property passing between Tasklets can be done all through a point and click GUI interface. Yes Yes
    Job Dependency Scheduling Dependency rules are a very powerful feature that can allow one Job to affect the scheduling rules of other Jobs. The best way to describe it, is to use an example. Say you have Jobs C and B that run every day at 8:00am, but they must not run until Job A runs before them. And Job A normally runs at 2:00am. If for some reason Job A fails to run at its expected time, Jobs C and B must not run till Job A completes. This is just one example of how to use Dependency Rules. Yes Yes
    Job Property Time Versioning Versioning a Tasklet, allows you to specify changes to a Tasklet's input properties that will take effect at predefined dates/times in the future. A Job that is executed by JobServer, will run with a specific time version of a Tasklet's properties that are effective at the time the Job/Tasklet is run. The input properties of the Tasklet can change, with time, if you choose to define future versions to the properties of a Tasklet. This is a very powerful feature, as it allows you to make changes to a Tasklet's input properties ahead of time, and they only have them become "effective" at a set time in the future. For example, at the start of the new year, have the job (that runs weekly for example) start sending its summary report to "bob@aceme.com" instead of "mike@acme.com". You can do this by creating a future version of the Tasklet's properties that take effect at the start of the year, for example. Yes Yes
    Importing/Exporting Jobs Export a job's state to an XML file format and import back into another JobServer environment. This feature can be useful when needing to move a job from say a staging environment to a production system. You can export/import one or many jobs at a time. All the state of a job from scheduling to Tasklet properties are persisted in an XML file format and can be later imported back into another JobServer system. Import/Export jobs using simple point and click GUI app or script it using easy to use command line utilities. Yes Yes
    Job Processing
    Job Grouping Job Groups are a way to logically organize jobs together. You can restrict user access to groups of jobs and assign certain properties and behavior to groups of jobs (like error alerting for example). Yes Yes
    Throttle Job Processing JobServers allows you to organize jobs into Partitions. Each Partition can be configured with its own resource allocation like how many jobs are allowed to run concurrently within it. There are other attributes you can assign, but one of the main purposes of a Partition is to allow users to throttle job processing and job scheduling for a category or collection of jobs. Yes Yes
    Advanced Threading Design JobServer uses an advanced threading design throughout the system. This is used for such things as scheduling and queuing jobs as well as for managing job processing concurrency. JobServer is capable of taking advantage of high-end hardware with many CPUs and cores. For example, you can configure JobServer (via the GUI tools) to create multiple job scheduling threads so as to improve scheduler response times (for cases where you have hundreds or thousands of jobs running concurrently) by assigning more scan threads to the JobServer schedulers. JobServer makes many of these features available in the GUI for the administrator to configure if they need advanced tuning options. Yes Yes
    Kill Running Jobs JobServer allows users to kill running jobs. Jobs that run in their own isolated JVM and jobs that run in shared JobServer JVM pools can be killed. Yes Yes
    Delete and Edit Queued Jobs Jobs get put in a Queue when the maximum concurrent job threshold has been reached for a Partition. JobServer offers end users tools for deleting and changing the order and priority of jobs in the Queue. Yes Yes
    Isolated JVM Per Job When a job enters running state, it can be run in a dedicated thread within a shared JVM or the user has the choice to have the job run in its own processes isolated JVM. Running jobs in isolated JVM's can be useful in cases where process isolation is important and to reduce chances that a rogue or misbehaving job can bring down a partition or node. Yes Yes
    Queue Management Monitor and track jobs as they enter the Queue and the Job Processor. Jobs can be organized to use dedicated Queues and have varying priority levels. Yes Yes
    Running Jobs Manually Jobs can be launched on demand from the GUI, invoked from the web service interface, or launched from the command line (using the jsmonitor tool). When you need to run jobs outside their normal scheduling times, running jobs manually on demand can be very useful. Yes Yes
    Automatic Job Retry If there is an abrupt failure of a primary/secondary or remote job processing node (e.g. host computer crashes), JobServer will attempt to retry failed jobs. Jobs can be configured to allow this capability on a job per job basis. Jobs can be configured to retry from the start or they can be configured to retry from the last failed Tasklet. Yes Yes
    Reporting and Monitoring
    Detailed System Audit Trail Everything the main JobServer engine does is logged and can be monitored using JobServer's reporting and monitor tools. You can track every startup and shutdown of JobServer and every error that has occurred and even be alerted by email of problems. Yes Yes
    Detailed Monitoring of Job Processing JobServer offers powerful tools for tracking every step and transition a jobs goes thru from the time it is first scheduled to the last message it logs when the job finishes running. You can monitor and query for jobs based on a variety of criteria, allowing users and administrators great control and visibility over what is happening in the system as a whole and with individual jobs and groups of jobs. Yes Yes
    Detailed Logging and Reporting JobServer's Tasklet SDK allows jobs and Tasklets to log messages when they run. The SDK supports both the Java Logging API and Log4J. Every logging message made by a Tasklet is capture and stored and can be reported on. Yes Yes
    Future Schedule Reporting Calendar Would you like to know what jobs are about to run in the next hour or next month? JobServer offers reporting tools to allow you to query for such reports at an individual job level and/or for a collection of jobs. You can see what jobs are expected to run across any future time window - a very powerful tool. Yes Yes
    Performance Statistics Reports Curious how your job processing has been performing recently or in the past? Need to know how long it takes job or groups of jobs to run or how often do jobs sit in the Queue? Wondering if there is a pattern over the past few weeks that jobs are taking longer to run, wondering which jobs are the problem? Well, JobServer has reports you can run from the GUI to answer all these questions and to help you quickly diagnose potential problem jobs and discover performance changes. Yes Yes
    Security / Audit Trail / Alerting
    Job Processing Audit History With this feature, JobServer offers the ability for every job to keep an audit trail of every input and every output generated by the job. This is supported both by the SDK and by the JobServer applications. This capability can allow a job to store all data used during the job run and to then report on this information. This can be very critical for systems that need to keep strict accounting of what a job did and under what conditions it ran for accounting, legal and other troubleshooting purposes. For example, if you use the Batch Script Tasklet, you can have it store the exact script it used during every run and the exact output it generated during every run. All this information can then be reported on using JobServer's standard report GUIs. Yes Yes
    User Actions Audit Trail Keep track of every critical action users and administrators take. From who created a job, to who edited it, to when the system was shutdown and by whom. This kind of accountability is critical in many organizations. Yes Yes
    User Rights & Security Management Users can be granted access to applications/features and jobs using JobServer's detailed rights management. For example, you can grant a user the right to view and edit the schedules of one job group but only the rights view the schedules of another job group. You can grant users the rights to monitor and view job processing reports but not to edit job attributes or properties. JobServer offers fine grained feature and resource access. Yes Yes
    Cascading Email Alerts You can setup to receive alerts of system level errors/failures and/or individual job processing errors. Alerts can be setup to cascade to users/administrators based on events/errors/alerts that occur on groups of jobs, job partitions and for individual jobs. So you can sign up to receive errors if any job in a Partition fails or just sign up to receive email alerts if one particular job fails. Yes Yes
    APIs, Hadoop, and SOA
    Open Source Developer SDK The goal of the JobServer open source developer SDK, soafaces, is to allow developers to build their own jobs in a modular and reusable fashion as they so desire. Many of the features are optional, so you may choose the level of functionality and reusability that you wish to employ. To create a job that will run in JobServer, all it takes is implementing one interface and putting the developed Java software into an appropriate JAR file. The job can be one line of code or thousands of lines of code. JobServer also comes prepackaged with built-in components called Tasklets/Weblets that can, for example, execute unix bash, ksh, and perl scripts and others that can send email alerts, FTP files, make Web Services calls, or perform XML processing. Yes Yes
    Hadoop Support JobServer comes with integrated support for Hadoop out of the box. You can use JobServer as your central platform for building, launching and monitoring your Hadoop job processing activity. JobServer's Java developer API makes building custom extensions to Hadoop easy. Use JobServer to build, schedule and track a single Hadoop task or to compose multiple Hadoop tasks into more complex jobs and workflows. For example, a job can be composed of multiple Hadoop tasks where one task directs its output to the input of other Hadoop tasks in the job chain. JobServer makes it easy to centrally schedule, track and monitor all your Hadoop tasks and jobs from one place for real-time monitoring as well as for historical reporting and auditing. JobServer's workflow orchestration allows users to manage and manipulate files and content between your local storage system and your Hadoop HDFS. You can also build custom web GUIs (using GWT) for your Hadoop tasks to visualize all input and output content and show real-time status as the tasks are running or to report on final status and results after a Hadoop task has finished processing. Yes Yes
    Web Services API A web services API is available that allows control and managing various functions within JobServer from a web services API. Yes Yes
    SOA Integration JobServer comes with built-in support for SOA using the Mule platform. Both Tasklets and Weblets have the option of using Mule to access SOAP, JMS and other services using the Mule Java API. Yes Yes