在CentOS上安装Singularity高性能容器

什么是singularity容器

Singularity是劳伦斯伯克利国家实验室专门为大规模、跨节点HPC和DL工作负载而开发的容器化技术。具备轻量级、快速部署、方便迁移等诸多优势,且支持从Docker镜像格式转换为Singularity镜像格式。除了兼容docker的镜像之外,singularity还有一个不太明显的优势:可以通过非root帐号来拉起容器,这样对于某些安全性要求比较高的场景来说还是有用途的。

环境准备

目前singularity支持了redhat系列Linux发行版的二进制安装,这里我们使用的基础系统是CentOS:

[root@centos /]# cat /etc/redhat-release 
CentOS Linux release 8.3.2011

安装配置epel扩展源

首先我们需要更新所有的系统软件:

[root@centos /]# yum update -y           
CentOS Linux 8 - AppStream                                                                                                                                                                                  1.1 MB/s | 6.3 MB     00:05    
CentOS Linux 8 - BaseOS                                                                                                                                                                                     1.7 MB/s | 2.3 MB     00:01    
Dependencies resolved.
Nothing to do.
Complete!

安装epel扩展源:

[root@centos /]# yum install -y epel-release
Last metadata expiration check: 0:00:22 ago on Wed 13 Jan 2021 09:24:36 AM UTC.
Dependencies resolved.
============================================================================================================================================================================================================================================
 Package                                                      Architecture                                           Version                                                   Repository                                              Size
============================================================================================================================================================================================================================================
Installing:
 epel-release                                                 noarch                                                 8-8.el8                                                   extras                                                  23 k

Transaction Summary
============================================================================================================================================================================================================================================
Install  1 Package

Total download size: 23 k
Installed size: 32 k
Downloading Packages:
epel-release-8-8.el8.noarch.rpm                                                                                                                                                                             140 kB/s |  23 kB     00:00    
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                        34 kB/s |  23 kB     00:00     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                                                                                                    1/1 
  Installing       : epel-release-8-8.el8.noarch                                                                                                                                                                                        1/1 
  Running scriptlet: epel-release-8-8.el8.noarch                                                                                                                                                                                        1/1 
  Verifying        : epel-release-8-8.el8.noarch                                                                                                                                                                                        1/1 

Installed:
  epel-release-8-8.el8.noarch                                                                                                                                                                                                               

Complete!

安装好epel源之后,需要再次更新系统软件:

[root@centos /]# yum update -y
Extra Packages for Enterprise Linux Modular 8 - x86_64                                                                                                                                                      9.2 kB/s | 527 kB     00:57    
Extra Packages for Enterprise Linux 8 - x86_64                                                                                                                                                               10 kB/s | 8.8 MB     14:46    
Last metadata expiration check: 0:00:19 ago on Wed 13 Jan 2021 09:26:09 AM UTC.
Dependencies resolved.
============================================================================================================================================================================================================================================
 Package                                                      Architecture                                           Version                                                     Repository                                            Size
============================================================================================================================================================================================================================================
Upgrading:
 epel-release                                                 noarch                                                 8-10.el8                                                    epel                                                  22 k

Transaction Summary
============================================================================================================================================================================================================================================
Upgrade  1 Package

Total download size: 22 k
Downloading Packages:
epel-release-8-10.el8.noarch.rpm                                                                                                                                                                            7.8 kB/s |  22 kB     00:02    
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                       6.1 kB/s |  22 kB     00:03     
warning: /var/cache/dnf/epel-05da96c052a128d8/packages/epel-release-8-10.el8.noarch.rpm: Header V4 RSA/SHA256 Signature, key ID 2f86d6a1: NOKEY
Extra Packages for Enterprise Linux 8 - x86_64                                                                                                                                                              1.6 MB/s | 1.6 kB     00:00    
Importing GPG key 0x2F86D6A1:
 Userid     : "Fedora EPEL (8) <epel@fedoraproject.org>"
 Fingerprint: 94E2 79EB 8D8F 25B2 1810 ADF1 21EA 45AB 2F86 D6A1
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-8
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                                                                                                    1/1 
  Running scriptlet: epel-release-8-10.el8.noarch                                                                                                                                                                                       1/1 
  Upgrading        : epel-release-8-10.el8.noarch                                                                                                                                                                                       1/2 
  Cleanup          : epel-release-8-8.el8.noarch                                                                                                                                                                                        2/2 
  Running scriptlet: epel-release-8-8.el8.noarch                                                                                                                                                                                        2/2 
  Verifying        : epel-release-8-10.el8.noarch                                                                                                                                                                                       1/2 
  Verifying        : epel-release-8-8.el8.noarch                                                                                                                                                                                        2/2 

Upgraded:
  epel-release-8-10.el8.noarch                                                                                                                                                                                                              

Complete!

配置epel国内镜像源

使用默认的源地址去下载软件的话,有可能会导致下载速度缓慢甚至下载失败。这里推荐使用华为的国内镜像源地址,配置方法如下:

[root@centos /]# sed -i "s/#baseurl/baseurl/g" /etc/yum.repos.d/epel.repo
[root@centos /]# sed -i "s/metalink/#metalink/g" /etc/yum.repos.d/epel.repo
[root@centos /]# sed -i "s@https\?://download.fedoraproject.org/pub@https://mirrors.huaweicloud.com@g" /etc/yum.repos.d/epel.repo

再次更新所有源:

[root@centos /]# yum update -y
Extra Packages for Enterprise Linux 8 - x86_64                                                                                                                                                              5.9 MB/s | 8.8 MB     00:01    
Last metadata expiration check: 0:00:02 ago on Wed 13 Jan 2021 09:55:51 AM UTC.
Dependencies resolved.
Nothing to do.
Complete!

使用yum安装singularity

到这里为止,基本的环境配置就完成了,可以正式开始singularity的安装:

[root@centos /]# yum install -y singularity
Last metadata expiration check: 0:00:59 ago on Wed 13 Jan 2021 09:55:51 AM UTC.
Dependencies resolved.
============================================================================================================================================================================================================================================
 Package                                                     Architecture                                           Version                                                      Repository                                            Size
============================================================================================================================================================================================================================================
Installing:
 singularity                                                 x86_64                                                 3.7.0-1.el8                                                  epel                                                  42 M

Transaction Summary
============================================================================================================================================================================================================================================
Install  1 Package

Total download size: 42 M
Installed size: 138 M
Downloading Packages:
singularity-3.7.0-1.el8.x86_64.rpm                                                                                                                                                                          3.4 MB/s |  42 MB     00:12    
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                       3.4 MB/s |  42 MB     00:12     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                                                                                                    1/1 
  Installing       : singularity-3.7.0-1.el8.x86_64                                                                                                                                                                                     1/1 
  Running scriptlet: singularity-3.7.0-1.el8.x86_64                                                                                                                                                                                     1/1 
  Verifying        : singularity-3.7.0-1.el8.x86_64                                                                                                                                                                                     1/1 

Installed:
  singularity-3.7.0-1.el8.x86_64                                                                                                                                                                                                            

Complete!

可以通过如下方式来验证singularity是否安装成功,并且查看版本号:

[root@centos /]# singularity --version
singularity version 3.7.0-1.el8

singualrity的基本使用方法

首先我们可以通过软件自身的help来查看软件支持的功能:

[root@centos /]# singularity --help

Linux container platform optimized for High Performance Computing (HPC) and
Enterprise Performance Computing (EPC)

Usage:
  singularity [global options...]

Description:
  Singularity containers provide an application virtualization layer enabling
  mobility of compute via both application and environment portability. With
  Singularity one is capable of building a root file system that runs on any 
  other Linux system where Singularity is installed.

Options:
  -c, --config string   specify a configuration file (for root or
                        unprivileged installation only) (default
                        "/etc/singularity/singularity.conf")
  -d, --debug           print debugging information (highest verbosity)
  -h, --help            help for singularity
      --nocolor         print without color output (default False)
  -q, --quiet           suppress normal output
  -s, --silent          only print errors
  -v, --verbose         print additional information
      --version         version for singularity

Available Commands:
  build       Build a Singularity image
  cache       Manage the local cache
  capability  Manage Linux capabilities for users and groups
  config      Manage various singularity configuration (root user only)
  delete      Deletes requested image from the library
  exec        Run a command within a container
  help        Help about any command
  inspect     Show metadata for an image
  instance    Manage containers running as services
  key         Manage OpenPGP keys
  oci         Manage OCI containers
  plugin      Manage Singularity plugins
  pull        Pull an image from a URI
  push        Upload image to the provided URI
  remote      Manage singularity remote endpoints, keyservers and OCI/Docker registry credentials
  run         Run the user-defined default command within a container
  run-help    Show the user-defined help for an image
  search      Search a Container Library for images
  shell       Run a shell within a container
  sif         siftool is a program for Singularity Image Format (SIF) file manipulation
  sign        Attach digital signature(s) to an image
  test        Run the user-defined tests within a container
  verify      Verify cryptographic signatures attached to an image
  version     Show the version for Singularity

Examples:
  $ singularity help <command> [<subcommand>]
  $ singularity help build
  $ singularity help instance start


For additional help or support, please visit https://www.sylabs.io/docs/

这里可以看到跟docker的接口还是有较大区别的,但是容器的本质还是基于namespace和cgroup的隔离方案,这点上都是大同小异。由于本文的主要目的在于介绍singularity的安装,这里不详细展开singularity的使用介绍,仅简单介绍一个使用的案例:适用singularity搭建一个ubuntu的容器环境。

  • 首先我们创建一个容器沙箱,这里用的基础镜像还是从dockerhub获取的ubuntu基础镜像
[root@centos /]# singularity build --sandbox ubuntu docker://ubuntu
INFO:    Starting build...
Getting image source signatures
Copying blob da7391352a9b done  
Copying blob 14428a6d4bcd skipped: already exists  
Copying blob 2c2d948710f2 [--------------------------------------] 0.0b / 0.0b
Copying config aa23411143 done  
Writing manifest to image destination
Storing signatures
2021/01/14 01:20:39  info unpack layer: sha256:da7391352a9bb76b292a568c066aa4c3cbae8d494e6a3c68e3c596d34f7c75f8
2021/01/14 01:20:39  info unpack layer: sha256:14428a6d4bcdba49a64127900a0691fb00a3f329aced25eb77e3b65646638f8d
2021/01/14 01:20:39  info unpack layer: sha256:2c2d948710f21ad82dce71743b1654b45acb5c059cf5c19da491582cef6f2601
INFO:    Creating sandbox directory...
INFO:    Build complete: ubuntu
  • 在执行结束后可以在当期目录下生成一个名为ubuntu的目录,通过拉起该目录为容器,可以修改镜像配置,制作自己的容器镜像
[root@centos /]# singularity shell -w ubuntu
  • 修改完配置之后,可以将该沙箱build成一个镜像文件
[root@centos /]# singularity build ubuntu-base.sif ubuntu/
INFO:    Starting build...
INFO:    Creating SIF file...
INFO:    Build complete: ubuntu-base.sif
  • 创建好sif镜像之后,会在本地生成一个sif文件,可通过该sif文件拉起一个容器,这里我们可以用一个非root帐号来拉起
[dechin@centos /]# singularity shell ubuntu.sif

这里我们不过多赘述容器的使用方法,后续会出一些源码安装的方案以及更多的使用和配置场景。

版权声明

本文首发链接为:https://www.cnblogs.com/dechinphy/p/singularity-install.html
作者ID:DechinPhy
更多原著文章请参考:https://www.cnblogs.com/dechinphy/

参考链接

  1. 北大超算集群使用教程
  2. singularity开源地址
“留一手”加剧内卷,“讲不清”浪费时间。
posted @ 2021-01-14 09:59  DECHIN  阅读(343)  评论(0编辑  收藏  举报