The following sections detail setting the necessary environment variables for writing parallel code. Please see:
The XLSMPOPTS environment variable allows you to specify options that affect SMP execution. You can declare XLSMPOPTS by using the following bash command format:
.-:------------------------------------------. V | >>-XLSMPOPTS=--+---+----runtime_option_name--=----option_setting---+--+---+->< '-"-' '-"-'
You can specify option names and settings in uppercase or lowercase. You can add blanks before and after the colons and equal signs to improve readability. However, if the XLSMPOPTS option string contains imbedded blanks, you must enclose the entire option string in double quotation marks (").
You can specify the following run-time options with the XLSMPOPTS environment variable:
Work is assigned to threads in a different manner, depending on the scheduling type and chunk size used. A brief description of the scheduling types and their influence on how work is assigned follows:
CEILING(number_of_iterations / number_of_threads)These partitions are then assigned to each of the threads. It is these partitions that are then subdivided into chunks of iterations. If a thread is asleep, the threads that are active will complete their assigned partition of work.
Choosing chunking granularity is a tradeoff between overhead and load balancing. The syntax for this option is schedule=suboption, where the suboptions are defined as follows:
When a thread becomes available, it takes the next chunk from its preassigned partition. If there are no more chunks in that partition, the thread takes the next available chunk from a partition preassigned to another thread.
The first chunk contains CEILING(number_of_iterations / number_of_threads) iterations. Subsequent chunks consist of CEILING(number_of_iterations_left / number_of_threads) iterations.
If you have not specified n, the chunks will contain CEILING(number_of_iterations / number_of_threads) iterations. Each thread is assigned one of these chunks. This is known as block scheduling.
If you have not specified schedule, the default is set to schedule=static, resulting in block scheduling.
This option allows you full control over the number of execution threads. The default value for num is 1 if you did not specify -qsmp. Otherwise, it is the number of online processors on the machine. For more information, see the NUM_PARTHDS intrinsic function.
Set stack=num so it is within the acceptable upper limit. num can be up to 256 MB for 32-bit mode, or up to the limit imposed by system resources for 64-bit mode. An application that exceeds the upper limit may cause a segmentation fault.
The xlsmp run-time library routines use both "busy-wait" and "sleep" states in their approach to waiting for work. You can control these states with the spins, yields, and delays options.
During the busy-wait search for work, the thread repeatedly scans the work queue up to num times, where num is the value that you specified for the option spins. If a thread cannot find work during a given scan, it intentionally wastes cycles in a delay loop that executes num times, where num is the value that you specified for the option delays. This delay loop consists of a single meaningless iteration. The length of actual time this takes will vary among processors. If the value spins is exceeded and the thread still cannot find work, the thread will yield the current time slice (time allocated by the processor to that thread) to the other threads. The thread will yield its time slice up to num times, where num is the number that you specified for the option yields. If this value num is exceeded, the thread will go to sleep.
In summary, the ordered approach to looking for work consists of the following steps:
The syntax for specifying these options is as follows:
Zero is a special value for spins and yields, as it can be used to force complete busy-waiting. Normally, in a benchmark test on a dedicated system, you would set both options to zero. However, you can set them individually to achieve other effects.
For instance, on a dedicated 8-way SMP, setting these options to the following:
parthds=8 : schedule=dynamic=10 : spins=0 : yields=0
results in one thread per CPU, with each thread assigned chunks consisting of 10 iterations each, with busy-waiting when there is no immediate work to do.
Typically, parthreshold is set to be equal to the parallelization overhead. If the computation in a parallelized loop is very small and the time taken to execute these loops is spent primarily in the setting up of parallelization, these loops should be executed sequentially for better performance.
seqthreshold acts as the reverse of parthreshold.
The allowed values for this option are the numbers from 0 to 32. If you set profilefreq to one of these values, the following results will occur.
It is important to note that dynamic profiling is not applicable to user-specified parallel loops (for example, loops for which you specified the PARALLEL DO directive).
The following environment variables, which are included in the OpenMP standard, allow you to control the execution of parallel code.
The OMP_DYNAMIC environment variable enables or disables dynamic adjustment of the number of threads available for the execution of parallel regions. The syntax is as follows:
>>-OMP_DYNAMIC=--+-TRUE--+------------------------------------->< '-FALSE-'
If you set this environment variable to TRUE, the run-time environment can adjust the number of threads it uses for executing parallel regions so that it makes the most efficient use of system resources. If you set this environment variable to FALSE, dynamic adjustment is disabled.
The default value for OMP_DYNAMIC is FALSE. If your code needs to use a specific number of threads to run correctly, you should disable dynamic thread adjustment.
The omp_set_dynamic subroutine takes precedence over the OMP_DYNAMIC environment variable.
The OMP_NESTED environment variable enables or disables nested parallelism. The syntax is as follows:
>>-OMP_NESTED=--+-TRUE--+-------------------------------------->< '-FALSE-'
If you set this environment variable to TRUE, nested parallelism is enabled. This means that the run-time environment might deploy extra threads to form the team of threads for the nested parallel region. If you set this environment variable to FALSE, nested parallelism is disabled.
The default value for OMP_NESTED is FALSE.
The omp_set_nested subroutine takes precedence over the OMP_NESTED environment variable.
Currently, XL Fortran does not support OpenMP nested parallelism.
The OMP_NUM_THREADS environment variable sets the number of threads that a program will use when it runs. The syntax is as follows:
>>-OMP_NUM_THREADS=--num---------------------------------------><
The default number of threads that a program uses when it runs is the number of online processors on the machine.
If you specify the number of threads with both the PARTHDS suboption of the XLSMPOPTS environment variable and the OMP_NUM_THREADS environment variable, the OMP_NUM_THREADS environment variable takes precedence. The omp_set_num_threads subroutine takes precedence over the OMP_NUM_THREADS environment variable.
If the number of threads you request exceeds the number your execution environment can support, your application will terminate.
The following example shows how you can set the OMP_NUM_THREADS environment variable:
export OMP_NUM_THREADS=16
The OMP_SCHEDULE environment variable applies to PARALLEL DO and work-sharing DO directives that have a schedule type of RUNTIME. The syntax is as follows:
>>-OMP_SCHEDULE=--sched_type--+---------------+---------------->< '-,--chunk_size-'
This environment variable is ignored for PARALLEL DO and work-sharing DO directives that have a schedule type other than RUNTIME.
If you have not specified a schedule type either at compile time (through a directive) or at run time (through the OMP_SCHEDULE environment variable or the SCHEDULE option of the XLSMPOPTS environment variable), the default schedule type is STATIC, and the default chunk size is set to the following for the first N - 1 threads:
chunk_size = ceiling(Iters/N)
It is set to the following for the Nth thread, where N is the total number of threads and Iters is the total number of iterations in the DO loop:
chunk_size = Iters - ((N - 1) * ceiling(Iters/N))
If you specify both the SCHEDULE option of the XLSMPOPTS environment variable and the OMP_SCHEDULE environment variable, the OMP_SCHEDULE environment variable takes precedence.
The following examples show how you can set the OMP_SCHEDULE environment variable:
export OMP_SCHEDULE="GUIDED,4" export OMP_SCHEDULE="DYNAMIC"