gtps3m0cSystem Performance and Measurement Reference

Data Collection Techniques

The basic techniques used in collecting data in the TPF system are:

  1. Reading out counters imbedded in TPF code and continuously updated by TPF, such as high-speed message counts
  2. Intercepting specific events such as file macros and program enters when the collector programs are active
  3. Dynamic sampling of parameters that fluctuate with time, such as I/O device queues and working storage blocks in use.
  4. Collecting data from certain storage controllers and certain DASD themselves. This technique is becoming increasingly more important.

In any data collection program involving real-time operations, one factor to consider is the load and interference the collection programs themselves impose on the system being measured. Minimal impact on normal processing operations is essential. Because this impact cannot be zero, every effort must be made to know the extent of the influence data collection has on the parameters to be analyzed.

Collectors

The activity in the message processing cycle is associated with 4 different collectors. These collectors retain the values of system variables involving message processing, program operation, file activity, and the overall system structure profile. These 4 collectors can be turned on singly or in combination. The data gathered by these collectors is listed in Table 1.

Table 1. Collectors and Their Information

Collector Information Collected
Message Arrival time (ECB created)
Origin, Destination address, or both
Text of message (input and output)
Length of message (input and output)
Type of input message (action code)
Output line queue lengths
Characters per line (input and output)
Count of messages (input and output)
Send time (ECB returned to the system)
I-stream number (to which output message is routed).
System Cross list queue length
Input list queue length
Ready list queue length
Deferred list queue length
Count of high-speed input messages
Count of low-speed input messages
CPU loop counters (of elapsed time in various CPU states)
Working storage block usage
System heap utilization
Count of active ECBs
Idle time accumulators
Shutdown indicators
Frame Usage indicators
Heap Frame Usage indicators
Pause time and counts
SWISC creates and enters (including tpf_fork calls)
System pools counts
Recovery log track buffers allocated
Recovery log record location
Recovery log records allocated
Recovery log tracks in use
Recovery log track writes
Commit scope buffers per commit scope
WLOGC blocked conditions
Commits per second
Rollbacks per second
Commit buffers in use in VFA.
Program Count of program ENTRCs, ENTNCs, or ENTDCs (by program name or number)
Count of program BACKCs (by program name)
Program residency (main storage, file)
I-stream number.
File Count of data record accesses (by device, channel)
Data record identity
Count of program accesses (by device, channel)
Count of tape writes (by tape ID)
Count of characters written to tape
Device queue counts (files and tapes)
VFA synchronization data
VFA RIAT option byte
Caching RIAT option byte
Last channel path ID used.

For most collectors, there are two kinds of information collected: summary data and detailed data. The summary data is collected according to specified intervals while the detailed data is collected by intercepts. File and message collectors have both interval and intercept data associated with them. The system collector has no intercept data and the program collector has no interval data.

$DCOLC macro calls imbedded in system segments centralize and assist collector implementation. (See TPF System Macros for more information about the $DCOLC macro.)

Sampling Mode

In sampling mode both summary and detailed data are collected over intervals by collectors, specified in a ZMEAS command. A collector's summary data is gathered at the beginning and end of an interval, while during the body of the interval its intercept data is collected. At the conclusion of one interval another interval begins with a (possibly) new collector. In this mode each collector is run for an interval during every period in a sequential manner. For instance, a collector might run for 5 seconds every minute so that it alone is active in a 5-second interval.

The TPF system does not allow simultaneous activation of multiple collectors because the intercept techniques would significantly interfere with message processing. When details from multiple collectors are required, the collectors must be run in sampling mode.

Continuous Mode

In continuous mode all the collectors are activated sequentially for the gathering of summary data. In addition, a single collector must be specified to gather detailed data at specified intervals. For example, if the file collector was selected, file intercepts are activated at the start time and remain active until data collection is stopped. During this period the system collector and message collector are activated at specified intervals to collect the system counters and the message counts. (Because there is no interval data for the program collector, it does not get activated during these intervals.) If the program collector is selected, the system, the file, and the message collectors are invoked in sequence for summary data while program intercept data is gathered continuously.