Using workload preemption to reclaim resources

You can configure a session to be preemptive so that when the session is under-allocated, it can preempt workload of other sessions instead of waiting for other sessions to voluntarily release slots.

The following criteria is used to identify preemption candidates in the given order:

  1. Select all over-allocated sessions

  2. Order preemption candidates by preemption rank and choose the sessions with lowest rank

    If there are multiple sessions in the lowest rank, choose the session according to the preemptionCriteria configuration, i.e., either a session with the most recently started task or the most over-allocated session (default). In cases where a few tasks must be preempted at the same time, the tasks with the smallest sum of elapsed computation times are chosen.

Session preemption characteristics:

  • Preemption can only be triggered by under-allocated sessions.

  • An under-allocated session that is configured to be preemptive will preempt over-allocated sessions if the under-allocated session has any unsatisfied demand.

  • Workload preemption only happens if the under-allocated session's preemption rank is higher than or equal to the rank of the over-allocated session. Whether a session preempts another session of lower or equal rank or only preempts a session of lower rank is determined by the setting of the preemptionScope attribute in the application profile Sessions cannot preempt other sessions with higher rank.

  • Only over-allocated slots will be preempted from a session. Preemption will not cause any session to become under-allocated.

  • If multiple slots are shared by more than one session, the session with the highest preemption rank is taken into consideration. In this case, the lower rank session is "protected" by the higher rank session and will not be preempted. Similarly, if multiple sessions on one host share the same slot, the session with the highest preemption rank is taken into consideration.

  • If multiple slots are shared by more than one session, the session with the highest preemption rank is taken into consideration. In this case, the lower rank session is "protected" by the higher rank session and will not be preempted. Similarly, if multiple sessions on one host share the same slot, the session with the highest preemption rank is taken into consideration.

  • If the SSM cannot find any slots in the lowest rank sessions to preempt, either because these slots cannot be used by the session or these slots are "protected" by higher ranked sessions or under-allocated sessions, the SSM will consider the next higher ranked sessions.

  • Preemption takes effect immediately. The service instance manager and service instance are restarted and assigned to under-allocated sessions.

Configuring workload preemption

Preemption scope is configured in the Consumer element of the application profile. The default value for preemptionScope is LowerOrEqualRankedSessions.

If preemptive is set to true for a session type, the under-allocated sessions of this session type can preempt other over-allocated sessions. The default value for preemptive is false.

The preemptionRank attribute defines the session’s rank in relation to other sessions. (Sessions with a lower preemption rank will get preempted before sessions with a higher rank.) The default value for preemptionRank is 1.

Example:

<Consumer preemptionScope="LowerRankedSessions"/>
<SessionTypes><Type name="type1" …  preemptive="true" preemptionRank="2"/>

The preemptionScope, preemptive, and preemptionRank attributes can be configured through the PMC or by manually editing the application profile.

Overriding configured parameters via API

At session creation time, a client can override the session type’s preemptive and preemptionRank parameters via the API. Once the session is created, these parameters cannot be changed. Refer to the API Reference in the Knowledge Center for more information.

Best practices

Here are some preemption guidelines for various types of workload. Note that the preemption rank values are provided as an example.


Type of Workload

Preemption Rank

For workload that you do not want preempted, set the preemption rank to the highest level.

20

For normal workload that can be preempted without consequence, set the preemption rank to the lowest level.

10