gtps2m20ACF/SNA Data Communications Reference

Flow Control

Flow control is an important aspect of the HPR architecture. In addition to the normal session level (LU-LU) pacing, end-to-end flow on an RTP connection is also regulated using the adaptive rate-based (ARB) algorithm to prevent one RTP endpoint from sending data faster than the network or remote RTP endpoint can process the data.

ARB Pacing

The adaptive rate-based (ARB) algorithm is the primary congestion and flow control mechanism used by RTP connections. An RTP endpoint is allowed to send a certain amount of data to the remote RTP endpoint during a given interval. That amount of data is called the send rate. Periodically, an RTP endpoint sends an NLP containing an ARB request. The remote RTP endpoint sends back an ARB reply with one of five possible values based on current conditions at the remote RTP endpoint. The ARB reply value indicates whether the RTP endpoint (that sent the ARB request) can raise its send rate, keep the send rate the same, or reduce its send rate and, if so, by how much.

The send rate value for an RTP connection is adaptive and changes based on conditions in the network and remote RTP endpoint. The send rate is unidirectional, which means that on a given RTP connection the amount of data RTP endpoint A is allowed to send to RTP endpoint B is not necessarily the same as the amount of data RTP endpoint B is allowed to send to RTP endpoint A.

The possible values in an ARB reply are:

Normal
Increase the send rate, if necessary.

Restraint
Maintain the current send rate.

Slowdown 1
Decrease the send rate by 12.5%.

Slowdown 2
Decrease the send rate by 25%.

Critical
Decrease the send rate by 50%.

How an RTP endpoint decides which value to place in an ARB reply is implementation dependent. Typically, the value is based on availability of resources in that RTP endpoint node. The TPF system sets the value in an ARB reply based on how many core blocks are available. The percentage of available blocks for each core block type (frame, ECB, common block, IOB, SWB) is calculated and compared against the SNA shutdown value for that core block type. The SNA shutdown values are defined by the ILWPx parameters on the SNAKEY macro in CTK2. Initially, the ARB rate reply value is set as normal. As each core block type is checked, the ARB rate reply value can be left as is or changed to a more severe value. For example, the initial value is normal, but based on the availability of frames the ARB rate reply is changed to slowdown 1. The availability of SWBs is then checked and there are enough SWBs available to warrant setting the ARB rate reply value to normal; however, because the value is already set to slowdown 1, it remains slowdown 1. Next, the availability of ECBs is such that slowdown 2 condition is reached. Because slowdown 2 is more severe than slowdown 1, the ARB rate reply is changed to slowdown 2.

If the value of the ILWPx parameter for a given core block type is X and the percentage of available blocks for that core block type is Y, the ARB rate reply value is set based on if Y is:

X+20 or higher
Normal

X+10 to X+19
Restraint

X to X+9
Slowdown 1

X-1 to X-9
Slowdown 2

X-10 or lower
Critical

For example, if the value of the ILWPF parameter is 40 (meaning SNA is shut down when the percentage of available frames is 40% or less) and the percentage of frames that are available is Y, the ARB rate reply is set based on if Y is:

60 or higher
Normal

50-59
Restraint

40-49
Slowdown 1

31-39
Slowdown 2

30 or lower
Critical
Note:
If the value of an ILWPx parameter is less than 20, a value of 20 is used in the calculation. If the value of an ILWPx parameter is greater than 65, a value of 65 is used in the calculation.

The flexibility of the ARB rate reply values allows RTP endpoints to react immediately to low resource conditions and even prevents seriously low resource conditions from being reached in most conditions by reacting as soon as the first sign of trouble is detected.

Intermediate nodes along the route of an RTP connection also play a role in the ARB algorithm. If an intermediate node becomes congested, it can set the slowdown 1 or slowdown 2 flag in the NHDR of any NLP. The RTP endpoint receiving the NLP lowers its send rate accordingly if either of the slowdown flags are set in the NHDR.

The ARB algorithm is not the same as traditional window-based algorithms, such as PU 5 virtual route (VR) pacing. With a traditional window-based algorithm like VR pacing, a node is allowed to send a certain amount of requests and then cannot send more requests until a pacing response has been received from the remote node. VR pacing has caused deadlock conditions in the TPF system where a large queue of output messages builds up in the TPF system, the messages cannot be sent because the TPF system is waiting for a VR pacing response, but because so many blocks are in use, the TPF system is in an input list shutdown condition and not polling the NCP; therefore, the VR pacing response cannot be read in.

Because the ARB algorithm is time-based rather than window-based, deadlock conditions are avoided. An RTP endpoint is allowed to send a certain amount of data on an RTP endpoint in a given interval; for example, 500 bytes per second. If an RTP endpoint has messages totaling 2000 bytes to send, 500 bytes are sent each second for 4 seconds. Each time 1 second passes, the RTP endpoint is allowed to send another 500 bytes. At no time is the RTP endpoint blocked because it is waiting for any response from the network or remote RTP endpoint. Unlike session-level (LU-LU) and VR pacing, there are no pacing requests and responses in the ARB algorithm. Instead, the ARB requests and replies adjust the send rates while data is flowing.

When more data is being generated by TPF applications than the allowed send rate for an RTP connection, the RTP output queue is used to control the rate at which output messages are sent. Even though the ARB algorithm controls the flow on RTP connections, session-level pacing for LU-LU sessions is still used and important. See RTP Output Queue for more information.

How often an RTP endpoint sends an ARB request depends on the implementation. The TPF system sends an ARB request at least every N messages sent on an RTP connection, where N is based on the sizes of the RTPCB table and HPRMT. Whenever the TPF system is sending an NLP that includes a status request, an ARB request is also included. Several actions can cause a status request to be sent, including an NLP after a path switch that contains the new route or when a heartbeat message is sent on an idle RTP connection. When traffic is flowing at a high rate on an RTP connection, it is important to ask for acknowledgements (by sending status requests) frequently to prevent the HPRMT from becoming full.

RTP Output Queue

The RTP output queue holds output messages that cannot be sent immediately on the RTP connection for one of the following reasons:

The RTP output queue does not replace the output message transmission (OMT) queue or the SOUTC queue. Each queue type provides a different level of service:

An output message can be on only one of these queues at a time. There is also a hierarchy here: Messages on the OMT queue move to either the RTP queue or directly to the SOUTC queue. Messages on the RTP queue always move to the SOUTC queue. Note that most output messages do not need to go on the OMT or RTP queues and are placed directly on the SOUTC queue. Table 6 explains when and why output messages are added and removed from the different queues.

Table 6. Processing Output Message Queues

Queue Type Why a Message Is Added to the Queue When Messages Are Removed from the Queue
OMT The LU-LU session pacing window is closed. The pacing response for the LU-LU session is received; messages are dequeued until the queue becomes empty or until the pacing window becomes closed again.
OMT Message is too large to go out in a single PIU. The amount of data is larger than the maximum request unit (MAXRU) size of the LU-LU session. The data needs to be split into smaller pieces. Immediately after the data has been broken into smaller pieces as long as the LU-LU session pacing window is not closed.
RTP When no path exists for the RTP connection because a path switch is in progress. The path switch is completed successfully.
RTP The send window on the RTP connection is closed. The TPF system is not allowed to send more data on this RTP connection during the current time interval. The current time interval expires; the TPF system is allowed to send another window of data. Messages are dequeued from the RTP output queue until the queue becomes empty or until the send window becomes closed again.
RTP The RTP connection resynchronization process is in progress for this RTP connection. The RTP connection resynchronization process is completed successfully.
SOUTC The PIU is ready to be sent to the network. The PIU has been received by the ALS.

The main purposes of the RTP output queue are as follows:

For performance reasons, the RTP output queue is a core memory table only, similar to the SOUTC queue. As with the SOUTC queue, items on the RTP output queue are regular core blocks (frames). Session-level pacing is still necessary for LU-LU sessions to prevent the TPF system from running out of core blocks. This is true for all types of LU-LU sessions: PU 5, PU 2.1, and HPR. Not all LU-LU sessions need to use session-level pacing. For example, if the application profile is such that one message in generates one message out, and a second message cannot be sent in until the response to the first message is received, the session is self-paced and session-level pacing is not needed. However, other application profiles, like those using pipeline sessions, require session-level pacing. If a TPF application is generating many output messages that cannot be sent out (for example, the ALS is in slowdown mode), those messages need to be queued on file on the OMT queue. If session-level pacing is not used, the pacing window will never close and the OMT queue will not be used. The result is that these output messages (which are core blocks) will go on the RTP output queue or the SOUTC queue causing the queues to become very large.

If the size of an RTP output queue becomes too large, the TPF system will break the RTP connection. In doing so, all the core blocks on that RTP output queue are returned to the system. This mechanism is designed to prevent a single RTP connection from running the TPF system out of core blocks. However, session-level pacing should still be used to prevent the RTP output queue from becoming too large in the first place.