  
  [1X8 Parallel computing with [5XSCSCP[1X[0X
  
  
  [1X8.1 Managing multiple requests[0X
  
  Using procedure calls explained in the previous section, the user can create
  several  requests  to  multiple  services to execute them in parallel, or to
  wait until the fastest result will be available.
  
  [1X8.1-1 SynchronizeProcesses[0m
  
  [2X> SynchronizeProcesses( [0X[3Xprocess1, process2, ..., processN[0X[2X ) ________[0Xfunction
  [2X> SynchronizeProcesses( [0X[3Xproclist[0X[2X ) _________________________________[0Xfunction
  [6XReturns:[0X  list of records with components [10Xobject[0m and [10Xattributes[0m
  
  The  function  collects  results of from each process given in the argument,
  and  returns  the  list, i-th entry of which is the result obtained from the
  i-th  process.  The  function  accepts  both  one argument that is a list of
  processes, and arbitrary number of arguments, each of them being a process.
  
  [4X---------------------------  Example  ----------------------------[0X
    [4X[0X
    [4Xgap> a:=NewProcess( "WS_Factorial", [10], "localhost", 26133 );[0X
    [4X#I  Creating a socket ...[0X
    [4X#I  Connecting to a remote socket via TCP/IP ...[0X
    [4X#I  Got connection initiation message[0X
    [4X#I  Requesting version 1.3 from the server ...[0X
    [4X#I  Server confirmed version 1.3 to the client ...[0X
    [4X#I  Request sent ...[0X
    [4X< process at localhost:26133 pid=2064 >[0X
    [4Xgap> b:=NewProcess( "WS_Factorial", [20], "localhost", 26134 );[0X
    [4X#I  Creating a socket ...[0X
    [4X#I  Connecting to a remote socket via TCP/IP ...[0X
    [4X#I  Got connection initiation message[0X
    [4X#I  Requesting version 1.3 from the server ...[0X
    [4X#I  Server confirmed version 1.3 to the client ...[0X
    [4X#I  Request sent ...[0X
    [4X< process at localhost:26134 pid=1975 >[0X
    [4Xgap> SynchronizeProcesses(a,b);[0X
    [4X#I  Process number 1 is ready[0X
    [4X#I  Closed 1st process, waiting for 2nd ...[0X
    [4X[ rec( attributes := [ [ "call_id", "localhost:26133:2064:yCWBGYFO" ] ], [0X
    [4X      object := 3628800 ), [0X
    [4X  rec( attributes := [ [ "call_id", "localhost:26134:1975:yAAWvGTL" ] ], [0X
    [4X      object := 2432902008176640000 ) ][0X
    [4X[0X
    [4X[0X
  [4X------------------------------------------------------------------[0X
  
  [1X8.1-2 FirstProcess[0m
  
  [2X> FirstProcess( [0X[3Xprocess1, process2, ..., processN[0X[2X ) ________________[0Xfunction
  [2X> FirstProcess( [0X[3Xproclist[0X[2X ) _________________________________________[0Xfunction
  [6XReturns:[0X  records with components [10Xobject[0m and [10Xattributes[0m
  
  The  function  waits for the result from each process given in the argument,
  and  returns the result coming first, terminating all remaining processes at
  the  same  time.  The  function  accepts both one argument that is a list of
  processes, and arbitrary number of arguments, each of them being a process.
  
  [4X---------------------------  Example  ----------------------------[0X
    [4X[0X
    [4Xgap> a:=NewProcess( "WS_Factorial", [10], "localhost", 26133 );[0X
    [4X#I  Creating a socket ...[0X
    [4X#I  Connecting to a remote socket via TCP/IP ...[0X
    [4X#I  Got connection initiation message[0X
    [4X#I  Requesting version 1.3 from the server ...[0X
    [4X#I  Server confirmed version 1.3 to the client ...[0X
    [4X#I  Request sent ...[0X
    [4X< process at localhost:26133 pid=2064 >[0X
    [4Xgap> b:=NewProcess( "WS_Factorial", [20], "localhost", 26134 );[0X
    [4X#I  Creating a socket ...[0X
    [4X#I  Connecting to a remote socket via TCP/IP ...[0X
    [4X#I  Got connection initiation message[0X
    [4X#I  Requesting version 1.3 from the server ...[0X
    [4X#I  Server confirmed version 1.3 to the client ...[0X
    [4X#I  Request sent ...[0X
    [4X< process at localhost:26134 pid=1975 >[0X
    [4Xgap>  FirstProcess(a,b); [0X
    [4X#I  Process number 1 is ready[0X
    [4Xrec( attributes := [ [ "call_id", "localhost:26133:2064:mdb8RaO2" ] ], [0X
    [4X  object := 3628800 )[0X
    [4X[0X
  [4X------------------------------------------------------------------[0X
  
  [1X8.1-3 SCSCPservers[0m
  
  [2X> SCSCPservers_______________________________________________[0Xglobal variable
  
  [2XSCSCPservers[0m  is  a  list  of  hosts  and ports to search for [5XSCSCP[0m services
  (which  may  be  not  only  represented by [5XGAP[0m services, but also by another
  [5XSCSCP[0m-compliant systems).
  
  It   is   used   by   parallel   skeletons   [2XParQuickWithSCSCP[0m  ([14X8.1-4[0m)  and
  [2XParListWithSCSCP[0m ([14X8.2-1[0m).
  
  The   initial   value   of   this   variable   is   specified  in  the  file
  [11Xscscp/configpar.g[0m and may be reassigned later.
  
  [1X8.1-4 ParQuickWithSCSCP[0m
  
  [2X> ParQuickWithSCSCP( [0X[3Xcommands, listargs[0X[2X ) __________________________[0Xfunction
  [6XReturns:[0X  record with components [10Xobject[0m and [10Xattributes[0m
  
  This  function  is  constructed using the [2XFirstProcess[0m ([14X8.1-2[0m). It is useful
  when  it  is  not known which partcular method is more efficient, because it
  allows  to  call  in parallel several procedures (given by the list of their
  names  [3Xcommands[0m)  with  the same list of arguments [3Xlistargs[0m (having the same
  meaning  as  in  [2XEvaluateBySCSCP[0m  ([14X6.1-4[0m))  and  obtain  the  result of that
  procedure call which will be computed faster.
  
  In  the example below we call two factorisation methods from the [5XGAP[0m package
  [5XFactInt[0m to factorise 2^150+1. The example is selected in such a way that the
  runtime of these two methods is approximately the same, so you should expect
  results from both methods in some random order from repeated calls.
  
  [4X---------------------------  Example  ----------------------------[0X
    [4X[0X
    [4Xgap> ParQuickWithSCSCP( [ "WS_FactorsECM", "WS_FactorsMPQS" ], [ 2^150+1 ] );[0X
    [4X#I  Creating a socket ...[0X
    [4X#I  Connecting to a remote socket via TCP/IP ...[0X
    [4X#I  Got connection initiation message [0X
    [4X#I  Request sent ...[0X
    [4X#I  Creating a socket ...[0X
    [4X#I  Connecting to a remote socket via TCP/IP ...[0X
    [4X#I  Got connection initiation message [0X
    [4X#I  Request sent ...[0X
    [4X#I  Process number 2 is ready[0X
    [4Xrec( object := [ 5, 5, 5, 13, 41, 61, 101, 1201, 1321, 8101, 63901, 268501, [0X
    [4X      13334701, 1182468601 ], [0X
    [4X attributes := [ [ "call_id", "localhost:26134:6731" ] ] )[0X
    [4X[0X
  [4X------------------------------------------------------------------[0X
  
  
  [1X8.2 MasterWorker skeleton[0X
  
  In  this  section  we  will  present  more general framework to run parallel
  computations, which has a number of useful features:
  
  --    it is implemented purely in [5XGAP[0m;
  
  --    the client (i.e. master, which orchestrates the computation) will work
        in UNIX/Linux, Mac OS X and MS Windows;
  
  --    it may orchestrate both [5XGAP[0m and non-[5XGAP[0m [5XSCSCP[0m servers;
  
  --    if  one  of  servers  (i.e.  workers)  will be lost, it will retry the
        computation on another available server;
  
  --    it  allows  to  add  dynamically new workers during the computation on
        hostnames  and  ports from a range perviously declared in [2XSCSCPservers[0m
        ([14X8.1-3[0m).
  
  To  configure  this  functionality,  the  file [11Xscscp/configpar.g[0m assigns the
  global  variable  [10XSCSCPservers[0m  which specifies a list of hosts and ports to
  search  for  [5XSCSCP[0m  services  (which  may  be  not  only  represented by [5XGAP[0m
  services, but also by another [5XSCSCP[0m-compliant systems). See comments in this
  file for further instructions.
  
  [1X8.2-1 ParListWithSCSCP[0m
  
  [2X> ParListWithSCSCP( [0X[3Xlistargs, procname[0X[2X ) ___________________________[0Xfunction
  [6XReturns:[0X  list
  
  [2XParListWithSCSCP[0m implements the well-known master-worker skeleton: we have a
  master  ([5XSCSCP[0m  client) and a number of workers ([5XSCSCP[0m servers) which obtain
  pieces  of  work  from  the client, perform the required job and report back
  with the result, waiting for the next job.
  
  It returns the list of the same length as [3Xlistargs[0m, i-th element of which is
  the result of calling the procedure [3Xprocname[0m with the argument [3Xlistargs[i][0m.
  
  It  accepts  two  options  which  should  be given as non-negative integers:
  [10Xtimeout[0m which specifies how long the client will wait for the result (if not
  given,  the  default  value is one hour) and [10Xrecallfrequency[0m which specifies
  the  number  of  iterations  after which the search for new services will be
  performed  (if not given the default value is zero meaning no such search at
  all).
  
  [4X---------------------------  Example  ----------------------------[0X
    [4X[0X
    [4Xgap> ParListWithSCSCP( List( [2..6], n -> SymmetricGroup(n)), "WS_IdGroup" );[0X
    [4X#I  master -> [ "localhost", 26133 ] : SymmetricGroup( [ 1 .. 2 ] )[0X
    [4X#I  master -> [ "localhost", 26134 ] : SymmetricGroup( [ 1 .. 3 ] )[0X
    [4X#I  [ "localhost", 26133 ] --> master : [ 2, 1 ][0X
    [4X#I  master -> [ "localhost", 26133 ] : SymmetricGroup( [ 1 .. 4 ] )[0X
    [4X#I  [ "localhost", 26134 ] --> master : [ 6, 1 ][0X
    [4X#I  master -> [ "localhost", 26134 ] : SymmetricGroup( [ 1 .. 5 ] )[0X
    [4X#I  [ "localhost", 26133 ] --> master : [ 24, 12 ][0X
    [4X#I  master -> [ "localhost", 26133 ] : SymmetricGroup( [ 1 .. 6 ] )[0X
    [4X#I  [ "localhost", 26133 ] --> master : [ 720, 763 ][0X
    [4X#I  [ "localhost", 26134 ] --> master : [ 120, 34 ][0X
    [4X[ [ 2, 1 ], [ 6, 1 ], [ 24, 12 ], [ 120, 34 ], [ 720, 763 ] ][0X
    [4X[0X
  [4X------------------------------------------------------------------[0X
  
  [1X8.2-2 SCSCPreset[0m
  
  [2X> SCSCPreset( [0X[3X[0X[2X ) ___________________________________________________[0Xfunction
  [6XReturns:[0X  nothing
  
  If   an  error  occurs  during  a  call  of  [2XParQuickWithSCSCP[0m  ([14X8.1-4[0m)  and
  [2XParListWithSCSCP[0m  ([14X8.2-1[0m), some of parallel requests may be still running at
  the  remaining  services,  making  them  inaccessible  for further procedure
  calls. [2XSCSCPreset[0m resets them by closing all open streams to [5XSCSCP[0m servers.
  
  [1X8.2-3 SCSCPLogTracesToGlobal[0m
  
  [2X> SCSCPLogTracesToGlobal( [0X[3Xtestname[0X[2X ) _______________________________[0Xfunction
  [2X> SCSCPLogTracesToGlobal( [0X[3X[0X[2X ) _______________________________________[0Xfunction
  
  To  analyse  the performance of parallel [5XSCSCP[0m framework, we make use of the
  [5XEdenTV[0m  program  [BL07]  developed initially to visualize the performance of
  parallel  programs  written in functional programming language Eden, and now
  distributed    under   the   GNU   Public   License   and   available   from
  [7Xhttp://www.mathematik.uni-marburg.de/~eden/?content=EdenTV[0m.
  
  Called with the string containing the name of the test, this functions turns
  on writing information about key activity events into trace files in current
  directories  for  the  client  and  servers listed [2XSCSCPservers[0m ([14X8.1-3[0m). The
  trace  file  will  have  the  name  of the format [3Xtestname[0m[10X.client.tr[0m for the
  client  and  [3Xtestname[0m[10X.<hostname>.<port>.tr[0m  for  the  server. After the test
  these  files  should be collected from remote servers and concatenated (e.g.
  using   [11Xcat[0m)   together   with   the   standard   preamble   from  the  file
  [11Xscscp/tracing/stdhead.txt[0m  (we recommend to put after the preamble first all
  traces  from  servers  and then the client's traces to have nicer diagrams).
  The resulting file then may be opened with [5XEdenTV[0m.
  
  In the following example we use a dual core MacBook laptop to generate trace
  files for two tests and then show their corresponding trace diagrams:
  
  [4X---------------------------  Example  ----------------------------[0X
    [4X[0X
    [4XSCSCPLogTracesToGlobal("quillen100");[0X
    [4XParListWithSCSCP( List( [1..100], i->[512,i]), "QuillenSeriesByIdGroup" );[0X
    [4XSCSCPLogTracesToGlobal();[0X
    [4XSCSCPLogTracesToGlobal( "euler" );[0X
    [4XParListWithSCSCP( [1..1000], "WS_Phi" );[0X
    [4XSCSCPLogTracesToGlobal();[0X
    [4X[0X
  [4X------------------------------------------------------------------[0X
  
  /See  diagrams in HTML and PDF versions of the manual/ The diagrams (made on
  an  dual core MacBook laptop), shows that in the first case parallelising is
  efficient  and master successfully distributes load to workers, while in the
  second  case  a single computation is just too short, so most of the time is
  spent   on  communication.  To  parallelize  the  Euler's  function  example
  efficiently,  tasks must rather be grouped in chunks, which should be enough
  large  to reduce the communication overload, but enough small to ensure that
  tasks are evenly distributed.
  
  Of course, tracing can be used to investigate communication between a client
  and  a  single  server  in a non-parallel context as well. For this purpose,
  [2XSCSCPservers[0m ([14X8.1-3[0m) must be modified to contain only one server.
  
  [2XParListWithSCSCP[0m ([14X8.2-1[0m) can be easily modified to have parallel versions of
  other  list  operations  like [2XForAll[0m ([14XReference: ForAll[0m), [2XForAny[0m ([14XReference:
  ForAny[0m),  [2XFirst[0m  ([14XReference:  First[0m),  [2XNumber[0m  ([14XReference: Number[0m), [2XFiltered[0m
  ([14XReference:  Filtered[0m), and also to have the skeleton in which the queue may
  be modified during the computation (for example, to compute orbits). We plan
  to provide such tools in one of the next versions of the package.
  
  
  [1X8.3 Example: parallelising Karatsuba multiplication for polynomials[0X
  
  The   file  [11Xscscp/example/karatsuba.g[0m  contains  an  implementation  of  the
  Karatsuba  multiplication  algorithm  for polynomials. This algorithm can be
  easily  parallelized since each recursive step creates three recursive calls
  of  the  same  function  for other polynomials. [13XWe will not parallelize each[0m
  recursive  call,  since this will create enormous data flow. Instead of this
  we  parallelize  only  the  top-level  function.  For  our  experiments with
  parallelising   Karatsuba   multiplication   for  polynomials  with  integer
  coefficients  we  used  the  multi-core workstation, on which we started one
  [5XSCSCP[0m   client   and   two   [5XSCSCP[0m   servers.   After   reading   the   file
  [11Xscscp/example/karatsuba.g[0m, the following function was created on the server
  
  [4X---------------------------  Example  ----------------------------[0X
    [4X[0X
    [4XKaratsubaPolynomialMultiplicationExtRepByString:=function(s1,s2)[0X
    [4Xreturn String( KaratsubaPolynomialMultiplicationExtRep( [0X
    [4X                 EvalString(s1), EvalString(s2) ) );[0X
    [4Xend;[0X
    [4X[0X
  [4X------------------------------------------------------------------[0X
  
  and   then  was  made  available  as  an  [5XSCSCP[0m  procedure  under  the  name
  [10XWS_Karatsuba[0m by the command
  
  [4X---------------------------  Example  ----------------------------[0X
    [4X[0X
    [4XInstallSCSCPprocedure( "WS_Karatsuba", [0X
    [4X                       KaratsubaPolynomialMultiplicationExtRepByString);[0X
    [4X[0X
  [4X------------------------------------------------------------------[0X
  
  This   function   provides   a   "bridge"   between  the  client's  function
  [10XKaratsubaPolynomialMultiplicationWS[0m     and     the     server's    function
  [10XKaratsubaPolynomialMultiplicationExtRep[0m,  which  performs the actual work on
  the  server.  [10XWS_Karatsuba[0m  converts  its  string  arguments  into  internal
  representation  of univariate polynomials (basically, lists of integers) and
  then  converts  the result back into string (since such data exchange format
  was  chosen). We are going to parallelize the following part of the client's
  code:
  
  [4X---------------------------  Example  ----------------------------[0X
    [4X[0X
    [4X...[0X
    [4Xu := KaratsubaPolynomialMultiplicationExtRep(f1,g1);[0X
    [4Xv := KaratsubaPolynomialMultiplicationExtRep(f0,g0);[0X
    [4Xw := KaratsubaPolynomialMultiplicationExtRep([0X
    [4X       PlusLaurentPolynomialsExtRep(f1,f0),[0X
    [4X       PlusLaurentPolynomialsExtRep(g1,g0) );[0X
    [4X...[0X
    [4X[0X
  [4X------------------------------------------------------------------[0X
  
  and this can be done straightforwardly - we replace two first calls by calls
  of  the  appropriate  [5XSCSCP[0m  services, then perform the 3rd call locally and
  then collect the results from the two remote calls:
  
  [4X---------------------------  Example  ----------------------------[0X
    [4X[0X
    [4X...[0X
    [4Xu := NewProcess( "WS_Karatsuba",[ String(f1), String(g1) ],"localhost", 26133);   [0X
    [4Xv := NewProcess( "WS_Karatsuba",[ String(f0), String(g0) ],"localhost", 26134);   [0X
    [4Xw := KaratsubaPolynomialMultiplicationExtRep([0X
    [4X       PlusLaurentPolynomialsExtRep(f1,f0),[0X
    [4X       PlusLaurentPolynomialsExtRep(g1,g0) );[0X
    [4Xwsresult:=SynchronizeProcesses2( u,v );[0X
    [4Xu := EvalString( wsresult[1].object );[0X
    [4Xv := EvalString( wsresult[2].object );[0X
    [4X...[0X
    [4X[0X
  [4X------------------------------------------------------------------[0X
  
  We  obtain  almost  double  speedup  on  three  cores  on randomly generated
  polynomials of degree 32000:
  
  [4X---------------------------  Example  ----------------------------[0X
    [4X[0X
    [4Xgap> fam:=FamilyObj(1);;[0X
    [4Xgap> f:=LaurentPolynomialByCoefficients( fam, [0X
    [4X>         List([1..32000],i->Random(Integers)), 0, 1 );;[0X
    [4Xgap> g:=LaurentPolynomialByCoefficients( fam, [0X
    [4X>         List([1..32000],i->Random(Integers)), 0, 1 );;[0X
    [4Xgap> t2:=KaratsubaPolynomialMultiplication(f,g);;time;[0X
    [4X5892[0X
    [4Xgap> t3:=KaratsubaPolynomialMultiplicationWS(f,g);;time;[0X
    [4X2974[0X
    [4X[0X
  [4X------------------------------------------------------------------[0X
  
