Platform Analytics 7 Dataflow

Overviews

Overview of Platform Analytics 7 architecture

Overview of cube and datamart flows

Overview of data collections and aggregation flows

 

See also:

Cubes

Datamarts

Datamart Data Tables

Raw Data Tables

Data Collection Loaders

 

Overview of Platform Analytics architecture

The following diagram depicts the high level architecture of Platform Analytics architecture.

 

 

Overview of cube and datamart flows

The following table illustrates how a cube gets its data. Cube data is derived from those datamarts specified below. Click a datamart for more information.
Cubes
Datamarts
Datamart Data Tables
WI_HOSTPROPERTIES
WI_JOBMART
WI_JOBMARTDAILY
WI_HOSTPROPERTIES
WI_HOSTMETRICS
WI_HOSTPROPERTIES
WI_HOSTMETRICSDAILY
WI_HOSTPROPERTIES
WI_HOSTMETRICS
WI_HOST_BOOLEANRES
WI_HOSTPROPERTIES
WI_HOSTMETRICSDAILY
WI_HOST_BOOLEANRES
WI_HOSTPROPERTIES
WI_HOSTMETRICSDAILY
WI_JOBRESUSAGECOST
WI_JOBRESUSAGECOSTDAILY
WI_HOSTPROPERTIES
WI_JOBTHROUGHPUT
WI_JOBTHROUGHPUTDAILY
WI_JOBTHROUGHPUTDAILY
WI_NUMBEROFJOBS
WI_NUMBEROFJOBSDAILY
WI_JOBSBYPENDREASON
WI_JOBSBYPENDREASONDAILY
WI_SHAREDRESOURCEUSAGE
WI_SHAREDRESOURCEUSAGEDAILY
WI_LICENSEUTILMART
WI_LICENSEUTILMARTDAILY
WI_LICENSEUTILMART
WI_LICENSECONSUMPTMART
WI_LICENSECONSUMPTMARTDAILY
WI_LICENSEUSAGEBYUSERMART
WI_LICENSEUSAGEBYUSERMARTDAILY
WI_LICUSAGEBYUSERGRPMART
WI_LICUBYUSERGRPMARTDAILY
WI_LICENSEDENIALMART
WI_LICENSEDENIALMARTDAILY
WI_LICUTILFEATUREMART
WI_LICUTILFEATUREMARTDAILY
WI_LICENSEDENIALBYFEATUREMART
WI_LICDENIALBYFEATUREMARTDAILY
WI_HOSTPROPERTIES
WI_LSFFLEXEVENTMART
WI_LSFFLEXEVENTMARTDAILY
WI_FLEXEVENTMART
WI_FLEXEVENTMARTDAILY
WI_HOSTPROPERTIES
WI_HOSTSTATUSHOURLY
WI_HOSTPROPERTIES
WI_HOSTSTATUSDAILY
WI_HOSTPROPERTIES
WI_HOSTSTATUSHOURLY
WI_HOSTPROPERTIES
WI_HOSTSTATUSDAILY
WI_JOBSLOTUSAGEBYGROUP
WI_JOBSLOTUSAGEBYGROUPDAILY

WI_HOSTMETRICS

WI_HOSTGROUPCONF
WI_HOSTMETRICSDAILY
WI_HOSTGROUPCONFDAILY
WI_JOBMART
WI_JOBMART_HPC
WI_JOBMARTDAILY
WI_JOBMART_HPC_DAILY
Cluster Capacity cube Cluster Capacity datamart WI_JOBS_VS_SLOTUTIL
WI_CLUSTER_SLOTS
Jobs vs Slot Utilization cube Jobs Vs Slot Utilization datamart WI_JOBS_VS_SLOTUTIL
Health Check cube Health Check datamart WI_HOSTMETRICS
WI_HOSTMETRICSDAILY
WI_JOBTHROUGHPUT
WI_JOBTHROUGHPUTDAILY
WI_JOBMART
WI_JOBMARTDAILY
Hardware Histogram cube Hardware Histogram datamart WI_HOSTMETRICS_HIST
License Usage by Feature Histogram cube License Usage by Feature Histogram datamart WI_LICUTILFEATUREMART_HIST
License Usage by Server Histogram cube License Usage by Server Histogram datamart WI_LICENSEUTILMART_HIST
Daily Hardware (with Boolean Resources and Core Utilization) cube Daily Hardware (with Boolean Resources and Core Utilization) datamart WI_HOSTMETRICSDAILY
WI_HOST_BOOLEANRES
Daily Core Utilization datamart WI_COREUTILDAILY
Daily Hardware (with Core Utilization) cube Daily Hardware (with Core Utilization) datamart WI_HOSTMETRICSDAILY
Daily Core Utilization datamart WI_COREUTILDAILY
Hourly Hardware (with Boolean Resources and Core Utilization) cube

Hourly Hardware (with Boolean Resources and Core Utilization) datamart

WI_HOSTMETRICS
WI_HOST_BOOLEANRES
Hourly Core Utilization datamart WI_COREUTILHOURLY
Hourly Hardware (with Core Utilization) cube Hourly Hardware (with Core Utilization) datamart WI_HOSTMETRICS
Hourly Core Utilization datamart WI_COREUTILHOURLY
Daily License Usage Statistics by Feature cube Daily License Usage Statistics by Feature - Denial datamart WI_LICDENIALBYFEATUREMARTDAILY
Daily License Usage Statistics by Feature - Utilization datamart WI_LICUTILFEATUREMARTDAILY
Daily License Usage Statistics by Feature - Consumption datamart WI_LICENSECONSUMPTMARTDAILY
Hourly License Usage Statistics by Feature cube Hourly License Usage Statistics by Feature - Denial datamart WI_LICENSEDENIALBYFEATUREMART
Hourly License Usage Statistics by Feature - Utilization datamart WI_LICUTILFEATUREMART
Hourly License Usage Statistics by Feature - Consumption datamart WI_LICENSECONSUMPTMART
Daily License Usage Statistics cube Daily License Usage Statistics - Denial datamart WI_LICENSEDENIALMARTDAILY
Daily License Usage Statistics - Utilization datamart WI_LICENSEUTILMARTDAILY
Daily License Usage Statistics - Consumption datamart WI_LICENSECONSUMPTMARTDAILY
Hourly License Usage Statistics cube Hourly License Usage Statistics - Denial datamart WI_LICENSEDENIALMART
Hourly License Usage Statistics - Utilization datamart WI_LICENSEUTILMART
Hourly License Usage Statistics - Consumption datamart WI_LICENSECONSUMPTMART

 

Overview of data collection and aggregation flows

The following table explains how database table data is generated:
Daily Datamart Data Tables
Hourly Datamart Data Tables
Raw Data Tables
Data Loaders

WI_HOSTMETRICSDAILY
(main_host_metrics_daily.xml)

WI_HOSTMETRICS
(main_host_metrics_hourly.xml)
RESOURCE_METRICS
WI_HOSTMETRICSDAILY
(main_host_notbuiltin_metrics_daily.xml)
WI_HOSTMETRICS
(main_job_resusage_bygroup.xml)
LSB_BJOBS lsfbjobsloader
WI_JOBMARTDAILY
(main_jobmart_etl_daily.xml)
WI_JOBMART
(main_jobmart_etl.xml)
LSB_EVENTS lsfeventsloader
WI_JOBMART_HPC_DAILY
(main_hpc_jobmart_etl_daily.xml)
WI_JOBMART_HPC
(main_jobmart_etl.xml)
LSB_EVENTS lsfeventsloader
WI_JOBRESUSAGECOSTDAILY
(main_jobresusagecost_etl_daily.xml)
WI_JOBRESUSAGECOST
(main_jobmart_etl.xml)
LSB_EVENTS lsfeventsloader
WI_JOBTHROUGHPUTDAILY
(main_jobthroughput_etl_daily.xml)
WI_JOBTHROUGHPUT
(main_job_throughput_etl.xml)
LSB_EVENTS lsfeventsloader
  WI_JOBEXECPROCS
(main_job_execproc_etl.xml)
LSB_EVENTS_EXECHOSTLIST lsfeventsloader
WI_JOBRESUSAGEBYGROUPDAILY
(main_jobresusagebygroup_etl_daily.xml)
WI_JOBRESUSAGEBYGROUP
(main_job_resusage_bygroup.xml)
LSB_BJOBS lsfbjobsloader
WI_JOBSLOTUSAGEBYGROUPDAILY
(main_jobslotusagebygroup_etl_daily.xml)
WI_JOBSLOTUSAGEBYGROUP
(main_job_slotsusage_bygroup.xml)
LSB_BJOBS lsfbjobsloader
WI_JOBSLOTUSAGEBYGROUP
(main_jobslotusagebyhost_etl.xml)
LSF_BHOSTS lsfbhostsloader
WI_JOBSLOTUSAGEBYGROUP
(main_jobslotusagebyqueue_etl.xml)
LSF_BQUEUES lsfbqueuesloader
WI_NUMBEROFJOBSDAILY
(main_jobnumberofjobs_etl_daily.xml)
WI_NUMBEROFJOBS
(main_jobstatistics_etl.xml)
LSB_BJOBS lsfbjobsloader

WI_SHAREDRESORUCEUSAGEDAILY

( main_sharedResourceUsageDaily.xml)

WI_SHAREDRESOURCEUSAGE

(main_sharedResourceUsageHourly.xml)

SHARED_RESOURCE_USAGE sharedresusageloader
 

WI_RESOURCEPROPERTIES
(main_sharedResoruceProperties.xml)

 

LSF_RESOURCE_PROPERTIES lsfresourcepropertiesloader
 

WI_HOSTPROPERTIES
(main_host_properties.xml)

 

WI_USERGROUPCONF
(main_userGroupConfHourly.xml)

WI_HOSTGROUPCONFDAILY
(main_host_group_conf_daily.xml)
WI_HOSTGROUPCONF
(main_host_group_conf_hourly.xml)
HOST_GROUP
WI_HOSTSTATUSDAILY
(main_lsf_host_status_daily.xml)
WI_HOSTSTATUSHOURLY
(main_lsf_host_status_hourly.xml)
LSF_BHOSTS lsfbhostsloader
WI_PENDTIME_BYGROUPSDAILY table
(main_detail_pend_reason_pendtime_daily.xml)
WI_PENDTIME_BYGROUPS table
(main_detail_pending_reason_hourly.xml)
DPR_BYINTERVAL
WI_PENDJOBNUM_BYGROUPSDAILY table
(main_detail_pend_reason_jobnum_daily.xml)
WI_PENDJOBNUM_BYGROUPS table
(main_detail_pending_reason_hourly.xml)
DPR_BYINTERVAL
WI_JOBSBYPENDREASONDAILY table
(main_job_pending_reason_daily.xml)
WI_JOBSBYPENDREASON table
(main_job_pending_reason_hourly.xml)
JOBS_PENDING_REASON lsfpendingreasonloader

WI_DPR_BYINTERVAL table
(main_detail_pending_reason_hourly.xml)

DPR_BYINTERVAL lsfpendingreasonloader
WI_LICENSEDENIALMARTDAILY
(main_license_denial_daily.xml)
WI_LICENSEDENIALMART
(main_license_denial_hourly.xml)
FLEXLM_LICENSE_EVENTS
WI_LICDENIALBYFEATUREMARTDAILY
(main_license_denial_by_feature_daily.xml)
WI_LICENSEDENIALBYFEATUREMART
(main_license_denial_hourly_byfeature.xml)
FLEXLM_LICENSE_EVENTS
WI_FLEXEVENTMARTDAILY
(main_flexEventDaily.xml)
WI_FLEXEVENTMART
(main_flexEventHourly.xml)
FLEXNET_LICENSE_EVENTS fnmloader
WI_LICENSEDENIALMARTDAILY
(main_license_denial_daily.xml)
WI_LICENSEDENIALMART
(main_flexEventDenialHourly.xml)
FLEXNET_LICENSE_EVENTS fnmloader
WI_LICDENIALBYFEATUREMARTDAILY
(main_license_denial_by_feature_daily.xml)
WI_LICENSEDENIALBYFEATUREMART
(main_flexEventDenialbyFeatureHourly.xml)
FLEXNET_LICENSE_EVENTS fnmloader
WI_LSFFLEXEVENTMARTDAILY
(main_lsfFlexEventDaily.xml)
WI_LSFFLEXEVENTMART
(main_flexEventHourly.xml)
FLEXNET_LICENSE_EVENTS fnmloader
WI_LICENSECONSUMPTMARTDAILY
(license_consumption_daily_etl.xml)
WI_LICENSECONSUMPTMART
(main_license_usage_hourly.xml)
FLEXLM_LICENSE_USAGE flexlicusageloader
WI_LICUBYUSERGRPMARTDAILY
(license_usage_daily_byusrgrp_etl.xml)
WI_LICUSAGEBYUSERGRPMART
(main_license_usage_hourly.xml)
FLEXLM_LICENSE_USAGE flexlicusageloader
WI_LICENSEUSAGEBYUSERMARTDAILY
(license_usage_daily_etl.xml)
WI_LICENSEUSAGEBYUSERMART
(main_license_usage_hourly.xml)
FLEXLM_LICENSE_USAGE flexlicusageloader
WI_LICENSEUTILMARTDAILY
(license_util_daily_etl.xml)
WI_LICENSEUTILMART
(main_license_usage_hourly.xml)
FLEXLM_LICENSE_USAGE flexlicusageloader
  WI_LICENSETOTAL
(main_license_usage_hourly.xml)
FLEXLM_LICENSE_USAGE flexlicusageloader
WI_LICUTILFEATUREMARTDAILY
(lic_util_daily_feature_etl.xml)
WI_LICUTILFEATUREMART
(main_license_usage_hourly_byfeature.xml)
FLEXLM_LICENSE_USAGE flexlicusageloader
WI_COREUTILDAILY
(main_hardware_core_util_daily.xml)
WI_COREUTILHOURLY
(main_hardware_core_util_hourly.xml)
HOST_CORE_UTILIZATION hostcoreutilloader