MindSpore 1.2.0-rc1版本一键模型迁移体验

1 MindSpore版本号

2 硬件平台和操作系统

官方安装指南推荐Ubuntu 18.04
但是手头只有Ubuntu 20.04，尝试了下，也成功安装了。

3个人邮箱号

unseenme@163.com

4 学习笔记

4.1 关于新特性：动态图分布式训练效率的大幅提升

MindSpore提供了Staging功能，该功能可以在PyNative模式下将Python函数或者Python类的方法编译成计算图，通过图优化等技术提高运行速度。
虽然没有分布式环境可以实践，但是可以学习一下编译成计算图的方法。

import numpy as np
import mindspore.nn as nn
from mindspore import context, Tensor
import mindspore.ops as ops
from mindspore import ms_function

context.set_context(mode=context.PYNATIVE_MODE, device_target="GPU")

class TensorAddNet(nn.Cell):
    def __init__(self):
        super(TensorAddNet, self).__init__()
        self.add = ops.Add()

    @ms_function
    def construct(self, x, y):
        res = self.add(x, y)
        return res

x = Tensor(np.ones([4, 4]).astype(np.float32))
y = Tensor(np.ones([4, 4]).astype(np.float32))
net = TensorAddNet()

z = net(x, y) # Staging mode
add = ops.Add()
res = add(x, z) # PyNative mode
print(res.asnumpy())

上述示例代码中，在TensorAddNet类的construct之前加装了ms_function装饰器，该装饰器会将construct方法编译成计算图，在给定输入之后，以图的形式下发执行，而上一示例代码中的add会直接以普通的PyNative的方式执行。

4.2 一键模型迁移

MindConverter是一款用于将PyTorch（ONNX）、TensorFlow（PB）模型转换到MindSpore模型定义脚本以及权重文件的工具。结合转换报告的信息，用户只需对转换后的脚本进行微小的改动，即可实现快速迁移。
在1.2.0这个版本中，MindConverter已经是MindInsight的子模块，所以需要安装MindInsight。

4.2.1 pip安装

先尝试从通过pip安装
参考：https://gitee.com/mindspore/mindinsight/blob/r1.2/README_CN.md#pip%E5%AE%89%E8%A3%85
安装ReadMe中并没有针对1.2.0版本给出指定的pip命令参数，需要自己替换。于是做了以下尝试，都失败了。

(py375mindspore) unseenme@PC1660:~$ pip install https://ms-release.obs.cn-north-4.myhuaweicloud.com/1.2.0/MindInsight/ascend/ubuntu_x86/mindinsight--cp37-cp37m-linux_x86_64.whl --trusted-host ms-release.obs.cn-north-4.myhuaweicloud.com -i https://pypi.tuna.tsinghua.edu.cn/simple
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting mindinsight==
  ERROR: HTTP error 404 while getting https://ms-release.obs.cn-north-4.myhuaweicloud.com/1.2.0/MindInsight/ascend/ubuntu_x86/mindinsight--cp37-cp37m-linux_x86_64.whl
ERROR: Could not install requirement mindinsight== from https://ms-release.obs.cn-north-4.myhuaweicloud.com/1.2.0/MindInsight/ascend/ubuntu_x86/mindinsight--cp37-cp37m-linux_x86_64.whl because of HTTP error 404 Client Error: Not Found for url: https://ms-release.obs.cn-north-4.myhuaweicloud.com/1.2.0/MindInsight/ascend/ubuntu_x86/mindinsight--cp37-cp37m-linux_x86_64.whl for URL https://ms-release.obs.cn-north-4.myhuaweicloud.com/1.2.0/MindInsight/ascend/ubuntu_x86/mindinsight--cp37-cp37m-linux_x86_64.whl
(py375mindspore) unseenme@PC1660:~$ pip install https://ms-release.obs.cn-north-4.myhuaweicloud.com/1.2.0-rc1/MindInsight/ascend/ubuntu_x86/mindinsight--cp37-cp37m-linux_x86_64.whl --trusted-host ms-release.obs.cn-north-4.myhuaweicloud.com -i https://pypi.tuna.tsinghua.edu.cn/simple
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting mindinsight==
  ERROR: HTTP error 404 while getting https://ms-release.obs.cn-north-4.myhuaweicloud.com/1.2.0-rc1/MindInsight/ascend/ubuntu_x86/mindinsight--cp37-cp37m-linux_x86_64.whl
ERROR: Could not install requirement mindinsight== from https://ms-release.obs.cn-north-4.myhuaweicloud.com/1.2.0-rc1/MindInsight/ascend/ubuntu_x86/mindinsight--cp37-cp37m-linux_x86_64.whl because of HTTP error 404 Client Error: Not Found for url: https://ms-release.obs.cn-north-4.myhuaweicloud.com/1.2.0-rc1/MindInsight/ascend/ubuntu_x86/mindinsight--cp37-cp37m-linux_x86_64.whl for URL https://ms-release.obs.cn-north-4.myhuaweicloud.com/1.2.0-rc1/MindInsight/ascend/ubuntu_x86/mindinsight--cp37-cp37m-linux_x86_64.whl
(py375mindspore) unseenme@PC1660:~$

既然找不到URL，那就暂时放弃这种安装方法吧。

4.2.2 源码编译安装

尝试从源码编译安装
参考：https://gitee.com/mindspore/mindinsight/blob/r1.2/README_CN.md#%E6%BA%90%E7%A0%81%E7%BC%96%E8%AF%91%E5%AE%89%E8%A3%85
先安装requirements.txt
然后执行setup.py脚本。出错如下

(py375mindspore) unseenme@PC1660:~/study/mind/mindinsight$ python setup.py install
running install
running build
running build_py
creating build/lib
creating build/lib/mindinsight
copying mindinsight/_version.py -> build/lib/mindinsight
copying mindinsight/__main__.py -> build/lib/mindinsight
copying mindinsight/__init__.py -> build/lib/mindinsight
running egg_info
building crc32 ...
/home/unseenme/anaconda3/envs/py375mindspore/bin/python3: No module named pybind11
building crc32 failure

这里感觉上是因为requirements.txt的内容不足，于是手动安装pybind11，之后再次执行setup.py脚本，错误变成了找不到npm。

(py375mindspore) unseenme@PC1660:~/study/mind/mindinsight$ python setup.py install
running install
running build
running build_py
creating build/lib/mindinsight
copying mindinsight/_version.py -> build/lib/mindinsight
copying mindinsight/__main__.py -> build/lib/mindinsight
copying mindinsight/__init__.py -> build/lib/mindinsight
running egg_info
building crc32 ...
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Updating submodules
Submodule 'third_party/securec' (https://gitee.com/openeuler/libboundscheck.git) registered for path 'third_party/securec'
Cloning into '/home/unseenme/study/mind/mindinsight/third_party/securec'...
Submodule path 'third_party/securec': checked out '737ad71e2c5cec766ad672e3fa7a66bbd37489cd'
-- Configuring done
-- Generating done
-- Build files have been written to: /home/unseenme/study/mind/mindinsight/build/build_securec
Scanning dependencies of target securec
[  2%] Building C object CMakeFiles/securec.dir/third_party/securec/src/fscanf_s.c.o
[  5%] Building C object CMakeFiles/securec.dir/third_party/securec/src/fwscanf_s.c.o
[  7%] Building C object CMakeFiles/securec.dir/third_party/securec/src/gets_s.c.o
[ 10%] Building C object CMakeFiles/securec.dir/third_party/securec/src/memcpy_s.c.o
[ 12%] Building C object CMakeFiles/securec.dir/third_party/securec/src/memmove_s.c.o
[ 15%] Building C object CMakeFiles/securec.dir/third_party/securec/src/memset_s.c.o
[ 17%] Building C object CMakeFiles/securec.dir/third_party/securec/src/scanf_s.c.o
[ 20%] Building C object CMakeFiles/securec.dir/third_party/securec/src/securecutil.c.o
[ 22%] Building C object CMakeFiles/securec.dir/third_party/securec/src/secureinput_a.c.o
[ 25%] Building C object CMakeFiles/securec.dir/third_party/securec/src/secureinput_w.c.o
[ 27%] Building C object CMakeFiles/securec.dir/third_party/securec/src/secureprintoutput_a.c.o
[ 30%] Building C object CMakeFiles/securec.dir/third_party/securec/src/secureprintoutput_w.c.o
[ 32%] Building C object CMakeFiles/securec.dir/third_party/securec/src/snprintf_s.c.o
[ 35%] Building C object CMakeFiles/securec.dir/third_party/securec/src/sprintf_s.c.o
[ 37%] Building C object CMakeFiles/securec.dir/third_party/securec/src/sscanf_s.c.o
[ 40%] Building C object CMakeFiles/securec.dir/third_party/securec/src/strcat_s.c.o
[ 42%] Building C object CMakeFiles/securec.dir/third_party/securec/src/strcpy_s.c.o
[ 45%] Building C object CMakeFiles/securec.dir/third_party/securec/src/strncat_s.c.o
[ 47%] Building C object CMakeFiles/securec.dir/third_party/securec/src/strncpy_s.c.o
[ 50%] Building C object CMakeFiles/securec.dir/third_party/securec/src/strtok_s.c.o
[ 52%] Building C object CMakeFiles/securec.dir/third_party/securec/src/swprintf_s.c.o
[ 55%] Building C object CMakeFiles/securec.dir/third_party/securec/src/swscanf_s.c.o
[ 57%] Building C object CMakeFiles/securec.dir/third_party/securec/src/vfscanf_s.c.o
[ 60%] Building C object CMakeFiles/securec.dir/third_party/securec/src/vfwscanf_s.c.o
[ 62%] Building C object CMakeFiles/securec.dir/third_party/securec/src/vscanf_s.c.o
[ 65%] Building C object CMakeFiles/securec.dir/third_party/securec/src/vsnprintf_s.c.o
[ 67%] Building C object CMakeFiles/securec.dir/third_party/securec/src/vsprintf_s.c.o
[ 70%] Building C object CMakeFiles/securec.dir/third_party/securec/src/vsscanf_s.c.o
[ 72%] Building C object CMakeFiles/securec.dir/third_party/securec/src/vswprintf_s.c.o
[ 75%] Building C object CMakeFiles/securec.dir/third_party/securec/src/vswscanf_s.c.o
[ 77%] Building C object CMakeFiles/securec.dir/third_party/securec/src/vwscanf_s.c.o
[ 80%] Building C object CMakeFiles/securec.dir/third_party/securec/src/wcscat_s.c.o
[ 82%] Building C object CMakeFiles/securec.dir/third_party/securec/src/wcscpy_s.c.o
[ 85%] Building C object CMakeFiles/securec.dir/third_party/securec/src/wcsncat_s.c.o
[ 87%] Building C object CMakeFiles/securec.dir/third_party/securec/src/wcsncpy_s.c.o
[ 90%] Building C object CMakeFiles/securec.dir/third_party/securec/src/wcstok_s.c.o
[ 92%] Building C object CMakeFiles/securec.dir/third_party/securec/src/wmemcpy_s.c.o
[ 95%] Building C object CMakeFiles/securec.dir/third_party/securec/src/wmemmove_s.c.o
[ 97%] Building C object CMakeFiles/securec.dir/third_party/securec/src/wscanf_s.c.o
[100%] Linking C static library libsecurec.a
[100%] Built target securec
building ui ...
build/scripts/ui.sh: line 24: npm: command not found
building ui failure

于是手动安装npm

(py375mindspore) unseenme@PC1660:~/study/mind/mindinsight$ sudo apt install npm

之后再次进行编译安装，终于成功了。

(py375mindspore) unseenme@PC1660:~/study/mind/mindinsight$ mindinsight start
Workspace: /home/unseenme/mindinsight
Summary base dir: /home/unseenme/study/mind/mindinsight
Web address: http://127.0.0.1:8080
service start state: success
(py375mindspore) unseenme@PC1660:~/study/mind/mindinsight$

4.2.3 转换PyTorch模型脚本

然后试一下转换一个PyTorch脚本。

(py375mindspore) unseenme@PC1660:~/study/mind/mindinsight$ mindconverter --in_file ../20210404_cnvt/model.py --output ../20210404_cnvt/output --report ../20210404_cnvt/output

[INFO] MINDCONVERTER: MindConverter: conversion is completed.

(py375mindspore) unseenme@PC1660:~/study/mind/mindinsight$

成功，在output文件夹下，生成了转换好的脚本文件和报告

到此，转换完成。真的实现了一键模型迁移。赞一个。

posted @ 2021-12-20 15:21 MS小白阅读(35) 评论(0) 收藏举报

刷新页面返回顶部