Platform Analytics 7 Dataflow

WI_LSFFLEXEVENTMARTDAILY table

This describes the wi_lsfflexeventmartdaily table and how each column of data arises from the wi_lsfflexeventmart table.

The data in wi_lsfflexeventmartdaily table is derived from the daily aggregation of wi_lsfflexeventmart. The daily aggregation range is between the last day of the wi_jobthroughputdaily table and the last second of yesterday (ie. 23:59:59).

The aggregation is grouped by "Cluster", "ExecHost", "SubHost", "ExecHostModel", "ExecHostType", "User", "Project", "Queue", "Num_Exec_Procs", "Job_Exit_Status", "Job_Exit_Code", "localized truncated QueueTime", "localized truncated StartTime", "localized truncated SubmitTime", "TimediffHour", "CoreHour", "PendingRankCode", "JobType", "LicFeature", "LicVendor", "LicVersion", "LicServer", "LicEvent" and "LicEventReason".

This is the column description of each data column of WI_LSFFLEXEVENTMARTDAILY and how each column is filled with data.

Column Name
Description
Key

CLUSTER_CODE

This comes from the "ClusterCode" field in the wi_lsfflexeventmart.

Primary key

QUEUE_TIME

This comes from the "QueueTime" field in the wi_lsfflexeventmart. The "Queue_time" is transformed to local time and then truncated to the date (ie, set the hour part to midnight) and then transformed back to GMT time.

Primary key

LIC_FEATURE_CODE

This comes from the "Lic_Feature_Code" field in the wi_lsfflexeventmart.

Primary key

LIC_VENDOR_CODE

This comes from the "Lic_Vendor_Code" field in the wi_lsfflexeventmart.

Primary key

LIC_VERSION_CODE

This comes from the "Lic_Version_Code" field in the wi_lsfflexeventmart.

Primary key

CHECK_OUT_MAX

This is the maximum of all the aggregated records of the CHECK_OUT_MAX field of the wi_lsfflexeventmart table for the day.

DURATION

This is the sum of the aggregated records of the DURATION field of the wi_lsfflexeventmart table for the day.

CHECK_OUT_NUM

This is the sum of the aggregated records of the check_out_num field of the wi_lsfflexeventmart table for the day.

SUBMISSION_DATE

This comes from the "SUBMISSION_DATE" field in the wi_lsfflexeventmart. The "SUBMISSION_DATE" is transformed to local time and then truncated to the date (ie, set the hour part to midnight) and then transformed back to GMT time.

Primary key

CLASSIFICATION

Not used

LIC_SERVER_CODE

This comes from the "Lic_Server_Code" field in the wi_lsfflexeventmart.

Primary key

LIC_EVENT_CODE

This comes from the "Lic_Event_Code" field in the wi_lsfflexeventmart.

Primary key

LIC_EVENTREASON_CODE

This comes from the "Lic_Eventreason_Code" field in the wi_lsfflexeventmart.

Primary key

LIC_SITE_NAME_CODE This comes from the "LIC_SITE_NAME_CODE" field in the wi_lsfflexeventmart. Primary key

EVENT_COUNT

This is the sum of the aggregated records of the EVENT_COUNT field of the wi_lsfflexeventmart table for the day.

POOL_CODE

Not used.

START_TIME

This comes from the "START_TIME" field in the wi_lsfflexeventmart. The "START_TIME" is transformed to local time and then truncated to the date (ie, set the hour part to midnight) and then transformed back to GMT time.

Primary key

FINISH_TIME

This comes from the "Finish_time" field in the wi_lsfflexeventmart. The "Finish_time" is transformed to local time and then truncated to the date (ie, set the hour part to midnight) and then transformed back to GMT time.

Primary key

USER_CODE

This comes from the "User_Code" field in the wi_lsfflexeventmart.

Primary key

EXECHOST_CODE

This comes from the "Exechost_Code" field in the wi_lsfflexeventmart.

Primary key

EXECHOSTMODEL_CODE

This is the result of the lookup of the LSFHOST_MODEL in the wi_HOSTPROPERTIES table. The lookup is based on the hostname lookup in the wi_hostcode table based on the EXECHOST_CODE column in the wi_jobmart table. Then the looked up LSFHOST_MODEL will be checked whether it is null or not. If it is null, then set it as "UNKNOWN". Once we get the LSFHOST_MODEL, we then look it up in the wi_hostmodelcode table to see if we have already has a record of it in there. If we do, then we'll get the code back, otherwise, we will insert it into the wi_hostmodelcode table and generate the code. The code itself is a positive integer and each new code is equal to the maximum of the existing code+1.

Primary key

EXECHOSTTYPE_CODE

This is the result of the lookup of the LSFHOST_TYPE in the wi_HOSTPROPERTIES table. The lookup is based on the hostname lookup in the wi_hostcode table based on the EXECHOST_CODE column in the wi_jobmart table. If it is null, then set it as "UNKNOWN". Once we get the LSFHOST_TYPE, we then look it up in the wi_hosttypecode table to see if we have already has a record of it in there. If we do, then we'll get the code back, otherwise, we will insert it into the wi_hosttypecode table and generate the code. The code itself is a positive integer and each new code is equal to the maximum of the existing code+1.

Primary key

PENDINGRANK_CODE

This comes from the sum of the pending_time of the wi_jobmart table after the aggregation.

Then it rank the pending time according to the following logic:

if (pendingTime == null) {

return "Not available";

} else if (pendingTime < 5/60) {

return "0 sec to < 5 sec";

} else if (pendingTime < 30/60) {

return "5 sec to < 30 sec";

} else if (pendingTime < 5) {

return "30 sec to < 5 min";

} else if (pendingTime < 60) {

return "5 min to < 60 min";

} else if (pendingTime < 1440) {

return "60 min to < 1 day";

} else {

return "One day or more";

}

Once we get the pending_rank, we then look it up in the wi_pendingrankcode table to see if we have already has a record of it in there. If we do, then we'll get the code back, otherwise, we will insert it into the wi_pendingrankcode table and generate the code. The code itself is a positive integer and each new code is equal to the maximum of the existing code+1.

Primary key

QUEUE_CODE

This comes from the "Queue_Code" field in the wi_lsfflexeventmart.

Primary key

PROJECT_CODE

This comes from the "Project_Code" field in the wi_lsfflexeventmart.

Primary key

JOB_TYPE

This comes from the "Job_Type" field in the wi_lsfflexeventmart.

Primary key

NUM_EXEC_PROCS

This comes from the "Num_Exec_Procs" field in the wi_lsfflexeventmart.

Primary key

JOB_EXIT_STATUS

This comes from the "Job_Exit_Status" field in the wi_lsfflexeventmart.

Primary key

JOB_EXIT_CODE

This comes from the "Job_Exit_Code" field in the wi_lsfflexeventmart.

Primary key

TIMEDIFF_HOUR

This comes from the "TimeDiff_Hour" field in the wi_lsfflexeventmart.

Primary key

RUN_TIME

This is the sum of the aggregated records of the run_time field of the wi_lsfflexeventmart table for the day.

PENDING_TIME

This is the sum of the aggregated records of the pending_time field of the wi_lsfflexeventmart table for the day.

CPU_TIME

This is the sum of the aggregated records of the cpu_time field of the wi_lsfflexeventmart table for the day.

MEM_USAGE

This is the sum of the aggregated records of the mem_usage field of the wi_lsfflexeventmart table for the day.

SWAP_USAGE

This is the sum of the aggregated records of the swap_usage field of the wi_lsfflexeventmart table for the day.

SUBHOST_CODE

This comes from the "SubHost_Code" field in the wi_lsfflexeventmart.

Primary key

TIMEDIFF_MIN This comes from the "TimeDiff_Min" field in the wi_lsfflexeventmart.  

CORE_HOUR

The core hour is resulted from a mapping of the original time_stamp in local time. If the time_stamp is within the range from Monday to Friday, 9am to 5pm, then the core_hour will be 1, otherwise it will be 0.

Primary key

NUM_OF_JOB

The number of records being aggregated into this record.

JOB_GROUP_CODE

A code representing the job group. The source is the job_group field in the wi_jobgroupcode table. If the job_group code already exists, the existing code is used; otherwise, the new group code is auto-incremented and inserted into the wi_jobgroupcode table.

Primary key

INSERT_SEQ

This is a system generated sequence number. For each new record inserted, an unique sequence number is being assigned to this column.