Platform Analytics 7 Dataflow

Previous Topic Next Topic Index

Hourly HPC Job Information datamart

Main ETL XML file

jobmart_HPC_Hourly.xml

Output CSV files

jobmart_HPC_Hourly.csv

Record sample

JOB_ID;JOB_ARRAY_INDEX;HPC_ARRAY_INDEX;CLUSTERNAME;CPUTIME;EXECHOSTNAME;FINISHCOREHOUR; FINISHDATE; FINISHHOUR; FINISHWEEKDAY; FINISHCOREHOUR; FINISHSHIFT; SUBMITDATE; SUBMITHOUR; ?/span>SUBMITWEEKDAY; SUBMITCOREHOUR; STARTDATE; STARTHOUR; STARTWEEKDAY; STARTCOREHOUR; ?/span>CLUSTERNAME; USERNAME; PROJECTNAME; QUEUENAME; SUBHOSTNAME; EXECHOSTNAME; JOBTYPE; NUMEXECPROCS; JOBEXITSTATUS; NUMPROCESS; CPUTIME; MEMUSAGE; SWAPUSAGE; RUNTIME; ?/span>PENDINGTIME; PENDINGTIMECATEGORY; JOBEXITCODE; USERGROUPNAME; APPLICATION_TAG

101; 0; 1; "2005-10-31"; 14; 2; 1; "Shift 2"; "2005-10-21"; 13; 6; 1; "2005-10-31"; "13"; "2"; "1"; "dazhang"; "P1"; "priority"; "ib01b14.lsf.platform.com"; "ib01b14.lsf.platform.com"; "10"; 2.0; "DONE"; 2.0; 0.02; 2.78515625; 8.28515625; 50.05; 14447.7333; "One day or more"; 0.0; "Other"; ""

102; 0; -1; "2005-10-31"; 14; 2; 1; "Shift 2"; "2005-10-21"; 13; 6; 1; "2005-10-31"; "13"; "2"; "1"; "LSF62"; "dazhang"; "P1"; "priority"; "ib01b14.lsf.platform.com"; "ib04b12.lsf.platform.com"; "10"; 2.0; "DONE"; 2.0; 0.04; 3.1328125; 8.61328125; 50.8; 14445.3833; "One day or more"; 0.0; "Other"; "app1"

Column description

The Daily Job Information datamart gets its information from the wi_jobmart table.

The following table describes each column and its source:

Column name

Source and description

JOB_ARRAY_INDEX

This is the job_array_index in the wi_jobmart table, which is the array index of the job.

HPC_ARRAY_INDEX

This is the hpc_array_index in the wi_jobmart_hpc table, which is the index of the execution hosts.

FINISHDATE

This is the date portion of the localized finish_time in the wi_jobmart table, and the format is "YYYY-MM-DD".

FINISHHOUR

This is the hour portion of the localized finish_time in the wi_jobmart table.

FINISHWEEKDAY

This is the weekday number from the localized finish_time in the wi_jobmart table, where "1" is Sunday, "2" is Monday, etc.

FINISHCOREHOUR

The source is the localized finish_time in the wi_jobmart table. If the finish_time is within the range of 9am to 5pm on Monday to Friday, this field returns 1, otherwise, it returns 0.

A value of 1 means that the data in this record is aggregated from the core_hour portion of the day (9am to 5pm, Monday to Friday).

FINISHSHIFT

This is based on the localized finish_time in the wi_jobmart table. The shifts transformer maps the localized time_stamp based on shift mapping and returns the user-defined shift.

SUBMITDATE

This is the date portion of the localized queue_time in the wi_jobmart table, and the format is "YYYY-MM-DD".

SUBMITHOUR

This is the hour portion of the localized queue_time in the wi_jobmart table.

SUBMITWEEKDAY

This is the weekday number from the localized queue_time in the wi_jobmart table, where "1" is Sunday, "2" is Monday, etc.

SUBMITCOREHOUR

The source is the localized queue_time in the wi_jobmart table. If the queue_time is within the range of 9am to 5pm on Monday to Friday, this field returns 1, otherwise, it returns 0.

A value of 1 means that the data in this record is aggregated from the core_hour portion of the day (9am to 5pm, Monday to Friday).

STARTDATE

This is the date portion of the localized start_time in the wi_jobmart table, and the format is "YYYY-MM-DD".

 

STARTHOUR

This is the hour portion of the localized start_time in the wi_jobmart table.

STARTWEEKDAY

This is the weekday number from the localized start_time in the wi_jobmart table, where "1" is Sunday, "2" is Monday, etc.

STARTCOREHOUR

The source is the localized start_time in the wi_jobmart table. If the start_time is within the range of 9am to 5pm on Monday to Friday, this field returns 1, otherwise, it returns 0.

A value of 1 means that the data in this record is aggregated from the core_hour portion of the day (9am to 5pm, Monday to Friday).

CLUSTERNAME

This is the cluster_name in the wi_clustercode table, which is based on the cluster_code column in the wi_jobmart table.

USERNAME

This is the user_name in the wi_usercode table, which is based on the user_code column in the wi_jobmart table.

PROJECTNAME

The source is the project_name column in the wi_projectcode table, which is based on the project_code column from wi_jobmart table.

QUEUENAME

The source is the queue_name column in the wi_queuecode table, which is based on the queue_code column in the wi_jobmart table.

SUBHOSTNAME

This comes from the subhost_code in the wi_jobmart table. We use the subhost_code and lookup the host_name in the wi_hostcode table and put it in this column.

EXECHOSTNAME

This comes from the exechost_code in the wi_jobmart table. We use the exechost_code and lookup the host_name in the wi_exechostcode table and put it in this column.

JOBTYPE

The source is the job_type column in the wi_jobmart table.

NUMEXECPROCS

The source is the num_exec_procs column in the wi_jobmart table.

JOBEXITSTATUS

The source is the job_exit_status column in the wi_jobmart table.

NUMPROCESS

The source is the NUM_PROCESS column in the wi_jobmart table.

CPUTIME

This is the cputime in the wi_jobmart table.

MEMUSAGE

The source is the mem_usage column in the wi_jobmart table.

SWAPUSAGE

The source is the swap_usage column in the wi_jobmart table.

RUNTIME

This comes from the run_time column in the wi_jobmart table.

PENDINGTIME

The source is the pending_time column in the wi_jobmart table.

PENDINGTIMECATEGORY

This field categorizes the pending_time into the following seven predefined levels:

"Not available", "0 sec to 5sec", "5 sec to 30 sec", "30 sec to 5 minutes", "5 minutes to 1 hour", "1 hour to 1 day", and "more then one day".

JOBEXITCODE

The source is the job_exit_code column in the wi_jobmart table.

USERGROUPNAME

The source is the user_group_name in the wi_usergroupcode table, which is based on the user_group_code in the wi_jobmart table.

APPLICATION_TAG

This is the application_tag in wi_applicationcode table, which is based on application _code in wi_jobmart_hpc table.

Previous Topic Next Topic Index