# 并行模式配置报错：Parallel mode dose not support semi\_parallel

**1. 系统环境**

```
Hardware Environment(Ascend/GPU/CPU): Ascend
Software Environment:
MindSpore version (source or binary): 1.6.0
Python version (e.g., Python 3.7.5): 3.7.5
OS platform and distribution (e.g., Linux Ubuntu 16.04):
GCC/Compiler version (if compiled from source): 7.3.0
```

**2. 运行脚本**

在分布式运行环境中，需要bash启动脚本、python训练脚本以及rank\_table的配置文件，将三个文件保存在同一目录下，三文件分别如下：

bash启动脚本

将以下shell脚本保存为run.sh

```
#!/bin/bash

RANK_SIZE=$1
EXEC_PATH=$(pwd)
export RANK_SIZE=${RANK_SIZE}
export HCCL_CONNECT_TIMEOUT=120 # 避免复现需要很长时间，设置超时为120s
export RANK_TABLE_FILE=${EXEC_PATH}/rank_table_8pcs.json # rank_table file的存放位置

for((i=0;i&lt;${RANK_SIZE};i++))
do
    rm -rf device$i
    mkdir device$i
    cp ./train.py ./device$i
    cd ./device$i
    export DEVICE_ID=$i
    export RANK_ID=$i
    echo &quot;start training for device $i&quot;
    env &gt; env$i.log
    python ./train.py &gt; train.log$i 2&gt;&amp;1 &amp;
    cd ../
done
echo &quot;The program launch succeed, the log is under device*/train.log*.;
```

**3. 训练脚本**

将以下python脚本保存为train.py

```
Parallel Example
import numpy as np

from mindspore import context, Parameter
from mindspore.nn import Cell, Momentum
from mindspore.ops import operations as P
from mindspore.train import Model
from mindspore.nn.loss import SoftmaxCrossEntropyWithLogits
import mindspore.dataset as ds
import mindspore.communication.management as D
from mindspore.train.callback import LossMonitor
from mindspore.train.callback import ModelCheckpoint
from mindspore.common.initializer import initializer

step_per_epoch = 4

def get_dataset(*inputs):
    def generate():
        for _ in range(step_per_epoch):
            yield inputs
    return generate


class Net(Cell):
    &quot;&quot;&quot;define net&quot;&quot;&quot;
    def __init__(self):
        super().__init__()
        self.matmul = P.MatMul().shard(((2, 4), (4, 1)))
        self.weight = Parameter(initializer(&quot;normal&quot;, [32, 16]), &quot;w1&quot;)
        self.relu = P.ReLU().shard(((8, 1),))

    def construct(self, x):
        out = self.matmul(x, self.weight)
        out = self.relu(out)
        return out


if __name__ == &quot;__main__&quot;:
    context.set_context(mode=context.GRAPH_MODE, device_target=&quot;Ascend&quot;, save_graphs=True)
    D.init()
    rank = D.get_rank()
    context.set_auto_parallel_context(parallel_mode=&quot;semi_parallel&quot;, device_num=8, full_batch=True)

    np.random.seed(1)
    input_data = np.random.rand(16, 32).astype(np.float32)
    label_data = np.random.rand(16, 16).astype(np.float32)
    fake_dataset = get_dataset(input_data, label_data)

    net = Net()

    callback = [LossMonitor(), ModelCheckpoint(directory=&quot;{}&quot;.format(rank))]
    dataset = ds.GeneratorDataset(fake_dataset, [&quot;input&quot;, &quot;label&quot;])
    loss = SoftmaxCrossEntropyWithLogits()

    learning_rate = 0.001
    momentum = 0.1
    epoch_size = 1
    opt = Momentum(net.trainable_params(), learning_rate, momentum)

    model = Model(net, loss_fn=loss, optimizer=opt)
    model.train(epoch_size, dataset, callbacks=callback, dataset_sink_mode=False)
```

rank\_table文件

将下面json文件保存为rank\_table\_8pcs.json

```
{
    &quot;version&quot;: &quot;1.0&quot;,
    &quot;server_count&quot;: &quot;1&quot;,
    &quot;server_list&quot;: [
        {
            &quot;server_id&quot;: &quot;10.90.41.205&quot;,
            &quot;device&quot;: [
                {
                    &quot;device_id&quot;: &quot;0&quot;,
                    &quot;device_ip&quot;: &quot;192.98.92.107&quot;,
                    &quot;rank_id&quot;: &quot;0&quot;
                },
                {
                    &quot;device_id&quot;: &quot;1&quot;,
                    &quot;device_ip&quot;: &quot;192.98.93.107&quot;,
                    &quot;rank_id&quot;: &quot;1&quot;
                },
                {
                    &quot;device_id&quot;: &quot;2&quot;,
                    &quot;device_ip&quot;: &quot;192.98.94.107&quot;,
                    &quot;rank_id&quot;: &quot;2&quot;
                },
                {
                    &quot;device_id&quot;: &quot;3&quot;,
                    &quot;device_ip&quot;: &quot;192.98.95.107&quot;,
                    &quot;rank_id&quot;: &quot;3&quot;
                },
                {
                    &quot;device_id&quot;: &quot;4&quot;,
                    &quot;device_ip&quot;: &quot;192.98.92.108&quot;,
                    &quot;rank_id&quot;: &quot;4&quot;
                },
                {
                    &quot;device_id&quot;: &quot;5&quot;,
                    &quot;device_ip&quot;: &quot;192.98.93.108&quot;,
                    &quot;rank_id&quot;: &quot;5&quot;
                },
                {
                    &quot;device_id&quot;: &quot;6&quot;,
                    &quot;device_ip&quot;: &quot;192.98.94.108&quot;,
                    &quot;rank_id&quot;: &quot;6&quot;
                },
                {
                    &quot;device_id&quot;: &quot;7&quot;,
                    &quot;device_ip&quot;: &quot;192.98.95.108&quot;,
                    &quot;rank_id&quot;: &quot;7&quot;
                }
            ],
            &quot;host_nic_ip&quot;: &quot;reserve&quot;
        }
    ],
    &quot;status&quot;: &quot;completed&quot;
}
```

4. 报错分析

启动脚本

如果本地机器有8个Ascend 910设备，那么可以设置启动命令如下：

```
bash run.sh 8
```

错误信息分析

如下图所示，明显的报错信息说明当前的并行模式中不支持semi\_parallel这种模式。

![](figures/15.png)

5. 报错解决

修改错误

在MindSpore主页中找到set\_auto\_parallel\_context接口，观察到参数parallel\_mode 的可选项有如下几种，正确配置即可。

![](figures/16.png)

6. 成功运行

在合理配置并行模式之后（如配置成semi\_auto\_parallel），成功运行输出loss值，如下图所示：

![](figures/17.png)

