Slurm提交OpenMPI作业

OpenMPI（open Message Passing Interface），OpenMPI是MPI的一种实现，是信息传递接口库项目。

1、安装OpenMPI

# wget https://download.open-mpi.org/release/open-mpi/v4.0/openmpi-4.0.4.tar.bz2
# tar jxvf openmpi-4.0.4.tar.bz2 
# cd openmpi-4.0.4/
# ./configure --prefix=/usr/local/openmpi
# make
# make install

 

2、添加环境变量

# export PATH="/usr/local/openmpi/bin:$PATH"
# export LD_LIBRARY_PATH="/usr/local/openmpi/lib/:$LD_LIBRARY_PATH"

写环境变量文件

# vim /etc/profile
export PATH="/usr/local/openmpi/bin:$PATH"
export LD_LIBRARY_PATH="/usr/local/openmpi/lib/:$LD_LIBRARY_PATH"

 

或者使用Moudle管理环境变量

# cat /etc/modulefiles/mpi/openmpi-4.04 
#%Module 1.0
##
## OpenMPI module for use with 'environment-modules' package:
module-whatis "sets up access to openmpi-4.04"
#prereq openmpi-4.04
conflict mpi
setenv SOMEVERION 4.04
append-path PATH /usr/local/openmpi/bin
append-path LD_LIBRARY_PATH /usr/local/openmpi/lib

 

3、测试mpirun

# cd openmpi-4.0.4/examples
# mpicc hello_c.c -o hello
# mpirun --allow-run-as-root -np 2 hello
Hello, world, I am 0 of 2, (Open MPI v4.0.4, package: Open MPI root@m1 Distribution, ident: 4.0.4, repo rev: v4.0.4, Jun 10, 2020, 102)
Hello, world, I am 1 of 2, (Open MPI v4.0.4, package: Open MPI root@m1 Distribution, ident: 4.0.4, repo rev: v4.0.4, Jun 10, 2020, 102)

多节点运行
多节点尽量使用NFS共享存储，不然所有节点都需要安装OpenMPI；

# for i in `seq 1 3` ; do echo "node$i" >> hostlist ; done
# /usr/local/openmpi/bin/mpirun --allow-run-as-root -np 6 --hostfile hostlist /openmpi-4.0.4/examples/hello
Hello, world, I am 0 of 6, (Open MPI v4.0.4, package: Open MPI root@m1 Distribution, ident: 4.0.4, repo rev: v4.0.4, Jun 10, 2020, 102)
Hello, world, I am 1 of 6, (Open MPI v4.0.4, package: Open MPI root@m1 Distribution, ident: 4.0.4, repo rev: v4.0.4, Jun 10, 2020, 102)
Hello, world, I am 4 of 6, (Open MPI v4.0.4, package: Open MPI root@m1 Distribution, ident: 4.0.4, repo rev: v4.0.4, Jun 10, 2020, 102)
Hello, world, I am 5 of 6, (Open MPI v4.0.4, package: Open MPI root@m1 Distribution, ident: 4.0.4, repo rev: v4.0.4, Jun 10, 2020, 102)
Hello, world, I am 2 of 6, (Open MPI v4.0.4, package: Open MPI root@m1 Distribution, ident: 4.0.4, repo rev: v4.0.4, Jun 10, 2020, 102)
Hello, world, I am 3 of 6, (Open MPI v4.0.4, package: Open MPI root@m1 Distribution, ident: 4.0.4, repo rev: v4.0.4, Jun 10, 2020, 102)

 

4、Slurm提交mpi任务

Slurm部署：https://www.cnblogs.com/liu-shaobo/p/13285839.html

# cat hello.sh
内容如下：
#!/bin/bash
#SBATCH --output=/tmp/job.%j.out
#SBATCH --error=/tmp/job.%j.err
#SBATCH --nodes=3                    ##使用节点数量
#SBATCH --ntasks-per-node=2          ##每个节点的进程数
mpirun --allow-run-as-root -np $SLURM_NPROCS ./openmpi-4.0.4/examples/hello

提交mpi任务

# sbatch hello.sh