This is the column description of each data column of WI_JOBRESUSAGEBYGROUP and how each column is filled with data.
Column Name |
Description |
Key |
CLUSTER_CODE |
This comes from the "ClusterName" field in the data file. We split the "Project" field into subfields and one of them is the cluster_name. Once we get the cluster_name, we then look it up in the wi_clustercode table to see if we have already has a record of it in there. If we do, then we'll get the code back, otherwise, we will insert it into the wi_clustercode table and generate the code. The code itself is a positive integer and each new code is equal to the maximum of the existing code+1. |
Primary key |
TIME_STAMP |
This is in GMT and is always end in the hour such as 02:00:00. This marks the time that this record is aggregated. For example, 02:00:00 means that all the records between 02:00:00 and 02:59:00 are aggregated into this record. |
Primary key |
GROUP_TYPE |
This indicates what kind of grouping this record belongs to. The different group type are: "By Host", "By Jobname", "By JobGroup", "By Project", "By Queue", "By User", "By UserGroup" and "By Cluster". |
Primary key |
GROUP_CODE |
This could comes from different source and it is depends on the group_type. If the group_type is "By Host", then it is one of the host name that comes from a subfield in the "Exechostlist" from the data file. The subfield contains the hostname and is being used here. If it is "By JobName", then it comes from the "JobName" field. If it is "By Project", then it comes from the "ProjectName". If it is "By Queue", then it comes from the "QueueName". If it is "By User", then it comes from the "UserName". If it is "By UserGroup", then it comes from the "UserGroupName". If it is "By Cluster", then it comes from the "ClusterName". Once we get the name, we then look it up in the wi_dimensioncode table to see if we have already has a record of it in there by matching the dimension_name. If we do, then we'll get the code back, otherwise, we will insert it into the wi_dimensioncode table and generate the code. The code itself is a positive integer and each new code is equal to the maximum of the existing code+1. |
Primary key |
JOB_TYPE |
This comes from the "JobType" field in the data file. For the slot aggregation for all job type, this field will be set with "ALL JOB TYPE". |
Primary key |
RESOURCE_CODE |
This depends on the sub grouping. It could be one of the following subgroups: "Memory", "Cputime", the actual license name that the job checked out, "Slot" and "Swap". Once we get the name, we then look it up in the wi_resourcecode table to see if we have already has a record of it in there by matching the resource_name. If we do, then we'll get the code back, otherwise, we will insert it into the wi_resourcecode table and generate the code. The code itself is a positive integer and each new code is equal to the maximum of the existing code+1. |
Primary key |
TOTAL_RES_USAGE |
This field is calculated in two phases. First, the data of the same sampling period are aggregated together based on "ClusterCode", "GroupBy", "TimeDiff", "ResourceCode", "JobType" and "GroupCode" as group by columns and we then sum up the value field based on the resource. For example, if the resource is "Memory", then the value field is the sum of the memory usage of the job with the same sampling interval. The second aggregation will then based on the grouping of "ClusterCode", "GroupBy", "TimeDiff", "ResourceCode", "JobType", "GroupCode" across the hour. A second record, without the jobtype in the grouping for the slot usage hourly aggregation, is produced for all job type. The total is calculated as the sum of all the data from the phase one calculation. |
|
MIN_RES_USAGE |
This field is calculated in two phases. First, the data of the same sampling period are aggregated together based on "ClusterCode", "GroupBy", "TimeDiff", "ResourceCode", "JobType" and "GroupCode" as group by columns and we then sum up the value field based on the resource. For example, if the resource is "Memory", then the value field is the sum of the memory usage of the job with the same sampling interval. The second aggregation will then based on the grouping of "ClusterCode", "GroupBy", "TimeDiff", "ResourceCode", "JobType", "GroupCode" across the hour. A second record, without the jobtype in the grouping for the slot usage hourly aggregation, is produced for all job type. The minimum of this value is calculated as the minimum of this value from the results of phase one. |
|
MAX_RES_USAGE |
This field is calculated in two phases. First, the data of the same sampling period are aggregated together based on "ClusterCode", "GroupBy", "TimeDiff", "ResourceCode", "JobType" and "GroupCode" as group by columns and we then sum up the value field based on the resource. For example, if the resource is "Memory", then the value field is the sum of the memory usage of the job with the same sampling interval. The second aggregation will then based on the grouping of "ClusterCode", "GroupBy", "TimeDiff", "ResourceCode", "JobType", "GroupCode" across the hour. A second record, without the jobtype in the grouping for the slot usage hourly aggregation, is produced for all job type. The maximum of this value is calculated as the maximum of this value from the results of phase one. |
|
COUNTER |
This field is calculated in two phases. First, the data of the same sampling period are aggregated together based on "ClusterCode", "GroupBy", "TimeDiff", "ResourceCode", "JobType" and "GroupCode" as group by columns and we then treat this as 1 record. For example, if the resource is "Memory", then the result of the first aggregation, we'll treat that as 1 record. The second aggregation will then based on the grouping of "ClusterCode", "GroupBy", "TimeDiff", "ResourceCode", "JobType", "GroupCode" across the hour. A second record, without the jobtype in the grouping for the slot usage hourly aggregation, is produced for all job type. The count will be the number of records produced in phase one. |
|
LOCAL_SERVERTIME |
This comes from the "Time_stamp" in this table. This is transformed into local server time. |
|
INSERT_SEQ |
This is a system generated sequence number. For each new record inserted, an unique sequence number is being assigned to this column. |
|