This is the column description of each data column of WI_JOBRESUSAGEBYGROUPDAILY and how each column is filled with data.
Column Name |
Description |
Key |
CLUSTER_CODE |
This comes from the "ClusterName" field in the data file. We split the "Project" field into subfields and one of them is the cluster_name. Once we get the cluster_name, we then look it up in the wi_clustercode table to see if we have already has a record of it in there. If we do, then we'll get the code back, otherwise, we will insert it into the wi_clustercode table and generate the code. The code itself is a positive integer and each new code is equal to the maximum of the existing code+1. |
Primary key |
TIME_STAMP |
This is in GMT and is always end in the day. The record is aggregated in a localized day. After the aggregation, the time_stamp is then transformed back at GMT and stored in this field. Thus, this time_stamp always in the GMT form of the localized day in midnight. |
Primary key |
GROUP_TYPE |
This indicates what kind of grouping this record belongs to. The different group type are: "By Host", "By Jobname", "By JobGroup", "By Project", "By Queue", "By User", "By UserGroup" and "By Cluster". |
Primary key |
GROUP_CODE |
This could comes from different source and it is depends on the group_type. If the group_type is "By Host", then it is one of the host name that comes from a subfield in the "Exechostlist" from the data file. The subfield contains the hostname and is being used here. If it is "By JobName", then it comes from the "JobName" field. If it is "By Project", then it comes from the "ProjectName". If it is "By Queue", then it comes from the "QueueName". If it is "By User", then it comes from the "UserName". If it is "By UserGroup", then it comes from the "UserGroupName". If it is "By Cluster", then it comes from the "ClusterName". Once we get the name, we then look it up in the wi_dimensioncode table to see if we have already has a record of it in there by matching the dimension_name. If we do, then we'll get the code back, otherwise, we will insert it into the wi_dimensioncode table and generate the code. The code itself is a positive integer and each new code is equal to the maximum of the existing code+1. |
Primary key |
JOB_TYPE |
This comes from the "JobType" field in the data file. |
Primary key |
RESOURCE_CODE |
This depends on the sub grouping. It could be one of the following subgroups: "Memory", "Cputime", the actual license name that the job checked out, "Slot" and "Swap". Once we get the name, we then look it up in the wi_resourcecode table to see if we have already has a record of it in there by matching the resource_name. If we do, then we'll get the code back, otherwise, we will insert it into the wi_resourcecode table and generate the code. The code itself is a positive integer and each new code is equal to the maximum of the existing code+1. |
Primary key |
TOTAL_RES_USAGE |
This field is calculated in two phases. First, the data of the same sampling period are aggregated together based on "ClusterCode", "GroupBy", "TimeDiff", "ResourceCode", "JobType" and "GroupCode" as group by columns and we then sum up the value field based on the resource. For example, if the resource is "Memory", then the value field is the sum of the memory usage of the job with the same sampling interval. The second aggregation will then based on the grouping of "ClusterCode", "GroupBy", "CoreHour", "ResourceCode", "JobType", "GroupCode" across the day based on local servertime. A second record, without the jobtype in the grouping for the slot usage hourly aggregation, is produced for all job type. The total is calculated as the sum of all the data from the phase one calculation. |
|
MIN_RES_USAGE |
This field is calculated in two phases. First, the data of the same sampling period are aggregated together based on "ClusterCode", "GroupBy", "TimeDiff", "ResourceCode", "JobType" and "GroupCode" as group by columns and we then sum up the value field based on the resource. For example, if the resource is "Memory", then the value field is the sum of the memory usage of the job with the same sampling interval. The second aggregation will then based on the grouping of "ClusterCode", "GroupBy", "CoreHour", "ResourceCode", "JobType", "GroupCode" across the day based on local servertime. A second record, without the jobtype in the grouping for the slot usage hourly aggregation, is produced for all job type. The minimum of this value is calculated as the minimum of this value from the results of phase one. |
|
MAX_RES_USAGE |
This field is calculated in two phases. First, the data of the same sampling period are aggregated together based on "ClusterCode", "GroupBy", "TimeDiff", "ResourceCode", "JobType" and "GroupCode" as group by columns and we then sum up the value field based on the resource. For example, if the resource is "Memory", then the value field is the sum of the memory usage of the job with the same sampling interval. The second aggregation will then based on the grouping of "ClusterCode", "GroupBy", "CoreHour", "ResourceCode", "JobType", "GroupCode" across the day based on local servertime. A second record, without the jobtype in the grouping for the slot usage hourly aggregation, is produced for all job type. The maximum of this value is calculated as the maximum of this value from the results of phase one. |
|
COUNTER |
This field is calculated in two phases. First, the data of the same sampling period are aggregated together based on "ClusterCode", "GroupBy", "TimeDiff", "ResourceCode", "JobType" and "GroupCode" as group by columns and we then treat this as 1 record. For example, if the resource is "Memory", then the result of the first aggregation, we'll treat that as 1 record. The second aggregation will then based on the grouping of "ClusterCode", "GroupBy", "CoreHour", "ResourceCode", "JobType", "GroupCode" across the day based on local servertime. A second record, without the jobtype in the grouping for the slot usage hourly aggregation, is produced for all job type. The count will be the number of records produced in phase one. |
|
CORE_HOUR |
The core hour is resulted from a mapping of the original time_stamp in local time. If the time_stamp is within the range from Monday to Friday, 9am to 5pm, then the core_hour will be 1, otherwise it will be 0. |
|
INSERT_SEQ |
This is a system generated sequence number. For each new record inserted, an unique sequence number is being assigned to this column. |
|