Platform Analytics 7 Dataflow

Previous Topic Next Topic Index

Daily HPC Job Information datamart

Main ETL XML file

jobmart_HPC_Daily.xml

Output CSV files

jobmart_HPC_Daily.csv

Record sample

FINISHDATE; FINISHWEEKDAY; FINISHCOREHOUR; SUBMITDATE; SUBMITWEEKDAY; STARTDATE; STARTWEEKDAY; CLUSTERNAME; USERNAME; PROJECTNAME; QUEUENAME; SUBHOSTMAPNAME; EXECHOSTMAPNAME; EXECHOSTMODELNAME; EXECHOSTTYPENAME; JOBTYPE; NUMEXECPROCS; JOBEXITSTATUS; JOBEXITCODE; NUMPROCESS; CPUTIME; MEMUSAGE; SWAPUSAGE; RUNTIME; PENDINGTIME; NUMOFJOBS; PENDINGTIMECATEGORY; USERGROUPNAME;NPROCS;NCORES;NTHREADS;APPLICATION_TAG

"2005-10-31"; 2; 0.0; "2005-10-25"; 3; "2005-10-31"; "2"; "LSF62"; "dazhang"; "P2"; "priority"; "ib01b14.lsf.platform.com"; "ib06b14.lsf.platform.com"; "Intel_EM64T"; "LINUX86"; "00"; 1.0; "DONE"; "0"; ; 1.81; 144.91015625; 430.03515625; 1561.9487; 458864.2997; 52.0; "One day or more"; "Other";"UNKNOWN";"UNKNOWN";"UNKNOWN";""

"2005-10-31"; 2; 0.0; "2005-10-23"; 1; "2005-10-31"; "2"; "LSF62"; "dazhang"; "P2"; "normal"; "ib01b14.lsf.platform.com"; "ib01b14.lsf.platform.com"; "Opteron848"; "LINUX86"; "00"; 1.0; "DONE"; "0";;; 0.14; 11.14453125; 33.09375; 162.15; 50032.0667; 4.0; "One day or more"; "Other";2;1;3;"app1"

Column description

The Daily Job Information datamart gets its information from the wi_jobmartdaily table.

The following table describes each column and its source: :

Column name

Source and description

FINISHDATE

This is the date portion of the localized finish_time in the wi_jobmartdaily table, and the format is "YYYY-MM-DD".

FINISHWEEKDAY

This is the weekday number from the localized finish_time in the wi_jobmartdaily table, where "1" is Sunday, "2" is Monday, etc.

FINISHCOREHOUR

The source is the localized finish_time in the wi_jobmartdaily table. If the finish_time is within the range of 9am to 5pm on Monday to Friday, this field returns 1, otherwise, it returns 0.

A value of 1 means that the data in this record is aggregated from the core_hour portion of the day (9am to 5pm, Monday to Friday).

SUBMITDATE

This is the date portion of the localized queue_time in the wi_jobmartdaily table, and the format is "YYYY-MM-DD".

SUBMITWEEKDAY

This is the weekday number from the localized queue_time in the wi_jobmartdaily table, where "1" is Sunday, "2" is Monday, etc.

STARTDATE

This is the date portion of the localized start_time in the wi_jobmartdaily table, and the format is "YYYY-MM-DD".

STARTWEEKDAY

This is the weekday number from the localized start_time in the wi_jobmartdaily table, where "1" is Sunday, "2" is Monday, etc.

CLUSTERNAME

This is the cluster_name in the wi_clustercode table, which is based on the cluster_code column in the wi_jobmartdaily table.

USERNAME

The source is the user_name column in the wi_usercode table, which is based on user_code column in the wi_jobmartdaily table.

PROJECTNAME

The source is the project_name column in the wi_projectcode table, which is based on the project_code column from wi_jobmartdaily table.

QUEUENAME

The source is the queue_name column in the wi_queuecode table, which is based on the queue_code column in the wi_jobmartdaily table.

SUBHOSTMAPNAME

This field comes from the mapping of subhost_name to a user defined name. The subhost_name itself comes from the lookup of host_name in the wi_hostcode table by using the subhost_code in the wi_jobmartdaily table.

EXECHOSTMAPNAME

The source of this is the mapping of exechost_name to a user defined name. The exechost_name comes from the host_name in the wi_hostcode table, which is based on the exechost_code column in the wi_jobmartdaily table.

EXECHOSTMODELNAME

The source of this is the hostmodel_name in the wi_hostmodelcode table, which is based on the exechostmodel_code column in the wi_jobmartdaily table.

EXECHOSTTYPENAME

The source of this is the hosttype_name in the wi_hosttypecode table, which is based on the exechosttype_code column in the wi_jobmartdaily table.

JOBEXITCODE

The source is the job_exit_code column in the wi_jobmartdaily table.

JOBEXITSTATUS

The source is the job_exit_status column in the wi_jobmartdaily table.

JOBTYPE

The source is the job_type column in the wi_jobmartdaily table.

CPUTIME

This is the cputime in the wi_jobmartdaily table.

MEMUSAGE

The source is the mem_usage column in the wi_jobmartdaily table.

NUMEXECPROCS

The source is the num_exec_procs column in the wi_jobmartdaily table.

NUMOFJOBS

The source is the num_of_job column in the wi_jobmartdaily table.

NUMPROCESS

This field is not used.

PENDINGTIME

The source is the pending_time column in the wi_jobmartdaily table.

PENDINGTIMECATEGORY

This field categorizes the pending_time into the following seven predefined levels:

"Not available", "0 sec to 5sec", "5 sec to 30 sec", "30 sec to 5 minutes", "5 minutes to 1 hour", "1 hour to 1 day", and "more then one day".

RUNTIME

This comes from the run_time column in the wi_jobmartdaily table.

SWAPUSAGE

The source is the swap_usage column in the wi_jobmartdaily table.

USERGROUPNAME

The source is the user_group_name in the wi_usergroupcode table, which is based on the user_group_code in the wi_jobmartdaily table.

NPROCS

This is the number of physical processors of the host, which is got from nprocs in wi_hostproperties table.

NCORES

This is the number of cores per processors of the host, which is got from ncores in wi_hostproperties table.

NTHREADS

This is the number of threads per core of the host, which is got from nthreads in wi_hostproperties table.

APPLICATION_TAG

This is the application_tag in wi_applicationcode table, which is based on application _code in wi_jobmart_hpc_daily table.

Previous Topic Next Topic Index