Job Tracker

The Job Tracker tool allows you to view and monitor the run history and current running progress of jobs that have run in the past and jobs that are currently running. The tool allows you to view the runtime log history for jobs that the user is interested in monitoring. Perform the following steps to select and track the jobs you are interested in:

  1. Choose the Filter By Condition options you are interested in. This let's you search for jobs based on their status, such as jobs with errors or jobs that have thrown warnings for example.
  2. Choose the Filter By State options you are interested in. This let's you search for jobs based their current running mode. The modes are Finished, Queued, and Running or All.
  3. Search the job(s) of interest, using a pattern name search, including the Partition(s) and Group(s) of interest. You can enter a job name using "*" wild card and you can also specify one or more unique job ids (use comma to separate between multiple jobs to search for).
  4. Once the search results have returned, they will be listed in the drop down list. Using the drop down list, select All Listed Jobs or select only a specific job of interest from the drop down list.
  5. Specify a time range that you are interested in.
  6. Click the Search button to find all the job runs that match the above criteria.

You also have the option of monitoring "Running Jobs" or "Queued Jobs". This a nice feature if you are interested in tracking jobs that are still in progress. As described above you can filter for these kinds of jobs by setting the "Filter By Status" options.

If you are using JobServer Pro, you can also filter by the Agents that the jobs have run on. You can search for jobs running on Primary/Secondary hosts or on specific Agent servers.

How Tasklet Logging Works

JobServer captures all Log4J and Java Logging API messages made by the Tasklet and makes them available for viewing from the Job Tracker tool. Note that Log4J Logger must support additivity=true for logs to be captured and reported by Job Tracker.

Filter By Condition

This filter option allows you to filter for jobs that have certain runtime conditions. You can search for jobs that have one or more of the following conditions:

Filter By State

In addition to the search filters above, you can also specify to view jobs that are still "Running" or are in "Queued" state. This will return any job that is running or queued. You can also choose to only look for jobs that have "Finished" running and thus completed processing. "Finished" also includes jobs that may have been terminated or failed, but either way they are no longer running.

Search By Job Name/ID

Search for the jobs that you are interested in. You can search by job name or by job id. For job id, you also have the option of entering more than one job id using comma separated job ids. The pattern "*" is used as a wild card search. Using the character "*" before and/or after a search pattern functions as a wild card. If you enter "*" then all jobs within the Group and Partition specified will be returned. For example, "foo*" will return any job that starts with the name "foo". If you enter a number, then it will try to find a job with the exact job id. Once you execute the search (by clicking the "go" button or by hitting the "Enter" key), the results will be placed in the drop down list below it, showing all jobs that match the search criteria.

There is also a special option for this field that lets you find a specific job based on its unique run id. In the search text field, simply enter the unique run id prefixed by "++", for example "++1276". This will search for a job with run id of 1276 when you click the "go" button. If the run id being searched is part of retry chain, then all job runs of the original run id will be shown.

Refine Job Search

If the search pattern finds matching jobs, they will be in the drop down list. If you choose "All Listed Jobs", then all the jobs in the drop down list will be searched for, otherwise only one single job will be searched for.

Paging

The "Paging" field allows you to limit how many jobs are shown at once on the screen. You will be able to page backwards and forwards through the entire list as defined by the paging size. For example, if your search result returns 1000 matches and your paging size is 200, then you will need to page 5 times to get to the end of the list.

Specify Time Range

Now specify what time range you want to search within. This will search for jobs that started running between the start and end dates specified.

Search Results

If there are jobs that are running or have run in the past that match the criteria specified, they will be displayed. You can drill down and view more details by clicking on the RunID in the first column. This will allow you to view the job's logging information all the way down to each individual Tasklet. If the job had multiple Tasklets, each one will have an entry and you can drill down and see more detailed logging information per Tasklet.

Download Report

The download feature allows you to download the entire report to your browser in the form of a tab delimited file. Click the "Download" button to start the download. You can have the downloaded report contain "All" the columns/fields available to be reported on or just the the columns that where "Selected" and visible for display on the screen.

Edit Columns

By default you will see the following columns for each job run results:

You can select additional fields to display by using the "Set Display Fields" button. You can add the following fields:

Hints:
Note that any field that ends with a ".." will have a tooltip associated with it that provides additional information. If you hover the mouse over the label you will see the tooltip.

Delete Queued Action

Queued jobs can be deleted by selecting them and then clicking the "Delete Queued" button.

Kill Running Action

Running jobs can be killed by selecting them and then clicking the "Kill Running" button. Jobs that are running in their own JVM are faster and easier to kill and are usually terminated immediately upon the kill action. Jobs running in the shared JVM can only be effectively killed when they reach certain checkpoints such as transitioning between one Tasklet to another or when calling logging API or other certain API calls, so please given them time to be terminated.

Server Mode Indicator

This is a simple message showing what state the overall JobServer engine is in such as "Running" or "Idle". If in "Idle" mode, for example, it means that the job scheduler is not running and no jobs can be scheduled or are processing. However, if you are using remote Agents, jobs can still be running on remote Agents if they are still processing jobs.

Note, that Mesos related features are only available for JobServer server installations running on Linux.