MPI\_COMM\_SPAWN\_MULTIPLE(3) Open MPI MPI\_COMM\_SPAWN\_MULTIPLE(3) MPI\_Comm\_spawn\_multiple — Spawns multiple binaries, or the same binary with multiple sets of arguments. # [SYNTAX](#SYNTAX) ## [C Syntax](#C_Syntax) ``` #include int MPI_Comm_spawn_multiple(int count, char *array_of_commands[], char **array_of_argv[], const int array_of_maxprocs[], const MPI_Info array_of_info[], int root, MPI_Comm comm, MPI_Comm *intercomm, int array_of_errcodes[]) ``` ## [Fortran Syntax](#Fortran_Syntax) ``` USE MPI ! or the older form: INCLUDE 'mpif.h' MPI_COMM_SPAWN_MULTIPLE(COUNT, ARRAY_OF_COMMANDS, ARRAY_OF_ARGV, ARRAY_OF_MAXPROCS, ARRAY_OF_INFO, ROOT, COMM, INTERCOMM, ARRAY_OF_ERRCODES, IERROR) INTEGER COUNT, ARRAY_OF_INFO(*), ARRAY_OF_MAXPROCS(*), ROOT, COMM, INTERCOMM, ARRAY_OF_ERRCODES(*), IERROR CHARACTER*(*) ARRAY_OF_COMMANDS(*), ARRAY_OF_ARGV(COUNT, *) ``` ## [Fortran 2008 Syntax](#Fortran_2008_Syntax) ``` USE mpi_f08 MPI_Comm_spawn_multiple(count, array_of_commands, array_of_argv, array_of_maxprocs, array_of_info, root, comm, intercomm, array_of_errcodes, ierror) INTEGER, INTENT(IN) :: count, array_of_maxprocs(*), root CHARACTER(LEN=*), INTENT(IN) :: array_of_commands(*) CHARACTER(LEN=*), INTENT(IN) :: array_of_argv(count, *) TYPE(MPI_Info), INTENT(IN) :: array_of_info(*) TYPE(MPI_Comm), INTENT(IN) :: comm TYPE(MPI_Comm), INTENT(OUT) :: intercomm INTEGER :: array_of_errcodes(*) INTEGER, OPTIONAL, INTENT(OUT) :: ierror ``` # [INPUT PARAMETERS](#INPUT_PARAMETERS) - **count**: Number of commands (positive integer, significant to MPI only at *root* — see NOTES). - **array\_of\_commands**: Programs to be executed (array of strings, significant only at *root*). - **array\_of\_argv**: Arguments for *commands* (array of array of strings, significant only at *root*). - **array\_of\_maxprocs**: Maximum number of processes to start for each command (array of integers, significant only at *root*). - **array\_of\_info**: Info objects telling the runtime system where and how to start processes (array of handles, significant only at *root*). - **root**: Rank of process in which previous arguments are examined (integer). - **comm**: Intracommunicator containing group of spawning processes (handle). # [OUTPUT PARAMETERS](#OUTPUT_PARAMETERS) - **intercomm**: Intercommunicator between original group and the newly spawned group (handle). - **array\_of\_errcodes**: One code per process (array of integers). - **ierror**: Fortran only: Error status (integer). # [DESCRIPTION](#DESCRIPTION) MPI\_Comm\_spawn\_multiple is identical to MPI\_Comm\_spawn <#mpi-comm-spawn> except that it can specify multiple executables. The first argument, *count*, indicates the number of executables. The next three arguments are arrays of the corresponding arguments in MPI\_Comm\_spawn <#mpi-comm-spawn>. The next argument, *array\_of\_info*, is an array of *info* arguments, one for each executable. See the INFO ARGUMENTS section for more information. For the Fortran version of *array\_of\_argv*, the element *array\_of\_argv*(i,j) is the jth argument to command number i. In any language, an application may use the constant MPI\_ARGVS\_NULL (which is likely to be **(char \*\*)0** in C) to specify that no arguments should be passed to any commands. The effect of setting individual elements of *array\_of\_argv* to MPI\_ARGV\_NULL is not defined. To specify arguments for some commands but not others, the commands without arguments should have a corresponding *argv* whose first element is null **((char \*)0** in C and empty string in Fortran). All of the spawned processes have the same MPI\_COMM\_WORLD. Their ranks in MPI\_COMM\_WORLD correspond directly to the order in which the commands are specified in MPI\_Comm\_spawn\_multiple. Assume that m1 processes are generated by the first command, m2 by the second, etc. The processes corresponding to the first command have ranks 0, 1,…, m1-1. The processes in the second command have ranks m1, m1+1, …, m1+m2-1. The processes in the third have ranks m1+m2, m1+m2+1, …, m1+m2+m3-1, etc. The *array\_of\_errcodes* argument is 1-dimensional array of size ``` _ count \ n , /_ i=1 i ``` where i is the ith element of *array\_of\_maxprocs*. Command number *i* corresponds to the i contiguous slots in this array from element ``` _ _ _ i-1 | _ i | \ n , to | \ n | -1 /_ j=1 i | /_ j=1 j | |_ _| ``` Error codes are treated as for MPI\_Comm\_spawn <#mpi-comm-spawn>. # [INFO ARGUMENTS](#INFO_ARGUMENTS) The following keys for *info* are recognized in “#PACKAGE\_NAME#”. (The reserved values mentioned in Section 5.3.4 of the MPI-2 standard are not implemented.) ``` Key Type Description --- ---- ----------- host char * Comma-separated list of hosts on which the processes should be spawned. See the orte_host man page for an explanation of how this will be used. hostfile char * Hostfile containing the hosts on which the processes are to be spawned. See the orte_hostfile man page for an explanation of how this will be used. add-host char * Add the specified hosts to the list of hosts known to this job and use it for the associated processes. This will be used similarly to the -host option. add-hostfile char * Hostfile containing hosts to be added to the list of hosts known to this job and use it for the associated process. This will be used similarly to the -hostfile option. wdir char * Directory where the executable is located. If files are to be pre-positioned, then this location is the desired working directory at time of execution - if not specified, then it will automatically be set to ompi_preload_files_dest_dir. ompi_prefix char * Same as the --prefix command line argument to mpirun. ompi_preload_binary bool If set to true, pre-position the specified executable onto the remote host. A destination directory must also be provided. ompi_preload_files char * A comma-separated list of files that are to be pre-positioned in addition to the executable. Note that this option does not depend upon ompi_preload_binary - files can be moved to the target even if an executable is not moved. ompi_stdin_target char * Comma-delimited list of ranks to receive stdin when forwarded. ompi_non_mpi bool If set to true, launching a non-MPI application; the returned communicator will be MPI_COMM_NULL. Failure to set this flag when launching a non-MPI application will cause both the child and parent jobs to "hang". ompi_param char * Pass an OMPI MCA parameter to the child job. If that parameter already exists in the environment, the value will be overwritten by the provided value. mapper char * Mapper to be used for this job map_by char * Mapping directive indicating how processes are to be mapped (slot, node, socket, etc.). rank_by char * Ranking directive indicating how processes are to be ranked (slot, node, socket, etc.). bind_to char * Binding directive indicating how processes are to be bound (core, slot, node, socket, etc.). path char * List of directories to search for the executable npernode char * Number of processes to spawn on each node of the allocation pernode bool Equivalent to npernode of 1 ppr char * Spawn specified number of processes on each of the identified object type env char * Newline-delimited list of envars to be passed to the spawned procs ``` *bool* info keys are actually strings but are evaluated as follows: if the string value is a number, it is converted to an integer and cast to a boolean (meaning that zero integers are false and non-zero values are true). If the string value is (case-insensitive) “yes” or “true”, the boolean is true. If the string value is (case-insensitive) “no” or “false”, the boolean is false. All other string values are unrecognized, and therefore false. Note that if any of the info handles have *ompi\_non\_mpi* set to true, then all info handles must have it set to true. If some are set to true, but others are set to false (or are unset), MPI\_ERR\_INFO will be returned. Note that in “#PACKAGE\_NAME#”, the first array location in *array\_of\_info* is applied to all the commands in *array\_of\_commands*. # [NOTES](#NOTES) The argument *count* is interpreted by MPI only at the root, as is *array\_of\_argv*. Since the leading dimension of *array\_of\_argv* is *count*, a nonpositive value of *count* at a nonroot node could theoretically cause a runtime bounds check error, even though *array\_of\_argv* should be ignored by the subroutine. If this happens, you should explicitly supply a reasonable value of *count* on the nonroot nodes. Similar to MPI\_Comm\_spawn <#mpi-comm-spawn>, it is the application’s responsibility to terminate each individual set of argv in the *array\_of\_argv* argument. In C, each argv array is terminated by a NULL pointer. In Fortran, each argv array is terminated by an empty string (note that compilers will not automatically insert this blank string; the application must ensure to have enough space for an empty string entry as the last element of the array). Other restrictions apply to the *array\_of\_argv* parameter; see MPI\_Comm\_spawn <#mpi-comm-spawn>’s description of the *argv* parameter for more details. MPI-3.1 implies (but does not directly state) that the argument *array\_of\_commands* must be an array of strings of length *count*. Unlike the *array\_of\_argv* parameter, *array\_of\_commands* does not need to be terminated with a NULL pointer in C or a blank string in Fortran. Older versions of Open MPI required that *array\_of\_commands* be terminated with a blank string in Fortran; that is no longer required in this version of Open MPI. Calling MPI\_Comm\_spawn <#mpi-comm-spawn> many times would create many sets of children with different MPI\_COMM\_WORLDs, whereas MPI\_Comm\_spawn\_multiple creates children with a single MPI\_COMM\_WORLD, so the two methods are not completely equivalent. Also if you need to spawn multiple executables, you may get better performance by using MPI\_Comm\_spawn\_multiple instead of calling MPI\_Comm\_spawn <#mpi-comm-spawn> several times. # [ERRORS](#ERRORS) [Important](#Important) In Open MPI, this function only works properly when the MPI job is launched via mpirun / mpiexec <#man1-mpirun>. Using this function in other run-time environments is currently unsupported, and may result in undefined behavior (e.g., the MPI job may hang). Almost all MPI routines return an error value; C routines as the return result of the function and Fortran routines in the last argument. Before the error value is returned, the current MPI error handler associated with the communication object (e.g., communicator, window, file) is called. If no communication object is associated with the MPI call, then the call is considered attached to MPI\_COMM\_SELF and will call the associated MPI error handler. When MPI\_COMM\_SELF is not initialized (i.e., before MPI\_Init <#mpi-init>/MPI\_Init\_thread <#mpi-init-thread>, after MPI\_Finalize <#mpi-finalize>, or when using the Sessions Model exclusively) the error raises the initial error handler. The initial error handler can be changed by calling MPI\_Comm\_set\_errhandler <#mpi-comm-set-errhandler> on MPI\_COMM\_SELF when using the World model, or the mpi\_initial\_errhandler CLI argument to mpiexec or info key to MPI\_Comm\_spawn <#mpi-comm-spawn>/MPI\_Comm\_spawn\_multiple. If no other appropriate error handler has been set, then the MPI\_ERRORS\_RETURN error handler is called for MPI I/O functions and the MPI\_ERRORS\_ABORT error handler is called for all other MPI functions. Open MPI includes three predefined error handlers that can be used: - **MPI\_ERRORS\_ARE\_FATAL** Causes the program to abort all connected MPI processes. - **MPI\_ERRORS\_ABORT** An error handler that can be invoked on a communicator, window, file, or session. When called on a communicator, it acts as if MPI\_Abort <#mpi-abort> was called on that communicator. If called on a window or file, acts as if MPI\_Abort <#mpi-abort> was called on a communicator containing the group of processes in the corresponding window or file. If called on a session, aborts only the local process. - **MPI\_ERRORS\_RETURN** Returns an error code to the application. MPI applications can also implement their own error handlers by calling: - MPI\_Comm\_create\_errhandler <#mpi-comm-create-errhandler> then MPI\_Comm\_set\_errhandler <#mpi-comm-set-errhandler> - MPI\_File\_create\_errhandler <#mpi-file-create-errhandler> then MPI\_File\_set\_errhandler <#mpi-file-set-errhandler> - MPI\_Session\_create\_errhandler <#mpi-session-create-errhandler> then MPI\_Session\_set\_errhandler <#mpi-session-set-errhandler> or at MPI\_Session\_init <#mpi-session-init> - MPI\_Win\_create\_errhandler <#mpi-win-create-errhandler> then MPI\_Win\_set\_errhandler <#mpi-win-set-errhandler> Note that MPI does not guarantee that an MPI program can continue past an error. See the MPI man page <#open-mpi> for a full list of MPI error codes <#open-mpi-errors>. See the Error Handling section of the MPI-3.1 standard for more information. **See also:** - MPI\_Comm\_spawn <#mpi-comm-spawn> - MPI\_Comm\_get\_parent <#mpi-comm-get-parent> - [mpirun(1)](/man/enterprise-packages/openmpi-cuda-12.4/mpirun.1) <#man1-mpirun> # [Copyright](#Copyright) 2003-2026, The Open MPI Community July 6, 2026 --- Document: MPI_Comm_spawn_multiple(3) from openmpi-cuda-12.4 5.0.10-r2 (Chainguard OS) Source: https://github.com/open-mpi/ompi