# 使用MindSpore的LayerNorm报错ValueError: For 'LayerNorm', gamma or beta shape must match input shape.

**1. 系统环境**

硬件环境\(Ascend/GPU/CPU\): Ascend/GPU/CPU

软件环境: MindSpore 版本: 2.0

执行模式（静态图/动态图）： 不限

Python 版本: 3.7.6

操作系统平台: linux

**2. 报错信息**

2.1 问题描述

将begin\_norm\_axis=1, begin\_params\_axis=1固定，并未实现与PyTorch完全一致的的功能。

2.2 报错信息

```
ValueError                                Traceback (most recent call last)
Cell In[12], line 3
      1 add_norm = AddNorm([3, 4], 0.5)
      2 add_norm.set_train(False)
----> 3 add_norm(ops.ones((2, 3, 4)), ops.ones((2, 3, 4))).shape

File ~/anaconda3/envs/MindSpore/lib/python3.9/site-packages/mindspore/nn/cell.py:664, in Cell.__call__(self, *args, **kwargs)
    662 except Exception as err:
    663     _pynative_executor.clear_res()
--> 664     raise err
    666 if isinstance(output, Parameter):
    667     output = output.data

File ~/anaconda3/envs/MindSpore/lib/python3.9/site-packages/mindspore/nn/cell.py:661, in Cell.__call__(self, *args, **kwargs)
    659     _pynative_executor.new_graph(self, *args, **kwargs)
    660     output = self._run_construct(args, kwargs)
--> 661     _pynative_executor.end_graph(self, output, *args, **kwargs)
    662 except Exception as err:
    663     _pynative_executor.clear_res()

File ~/anaconda3/envs/MindSpore/lib/python3.9/site-packages/mindspore/common/api.py:1304, in _PyNativeExecutor.end_graph(self, obj, output, *args, **kwargs)
   1291 def end_graph(self, obj, output, *args, **kwargs):
   1292     """
   1293     Clean resources after building forward and backward graph.
   1294 
   (...)
   1302         None.
   1303     """
-> 1304     self._executor.end_graph(obj, output, *args, *(kwargs.values()))

ValueError: For 'LayerNorm', gamma or beta shape must match input shape, but got input shape: [const vector][2, 3, 4], gamma shape: [const vector][3, 4], beta shape: [const vector][3, 4].

----------------------------------------------------
- C++ Call Stack: (For framework developers)
----------------------------------------------------
mindspore/core/ops/layer_norm.cc:111 InferShape
```

2.3 脚本信息

```
class AddNorm(nn.Cell):
    """残差连接后进行层规范化"""
    def __init__(self, normalized_shape, dropout, **kwargs):
        super(AddNorm, self).__init__(**kwargs)
        self.dropout = nn.Dropout(p=dropout)
        self.ln = nn.LayerNorm(normalized_shape) #begin_norm_axis=1, begin_params_axis=1需要增加

    def construct(self, X, Y):
        return self.ln(self.dropout(Y) + X)

add_norm = AddNorm([3, 4], 0.5)
add_norm.set_train(False)
add_norm(ops.ones((2, 3, 4)), ops.ones((2, 3, 4))).shape
```

**3. 根因分析**

问题出在nn.LayerNorm这个算子上，单独把算子抽出来看。

如下代码就能复现问题，normalized\_shape取\[3, 4\]， 输入shape\[2, 3, 4\]，报错是一样的。

```
import mindspore as ms
import numpy as np
x = ms.Tensor(np.ones([2, 3, 4]), ms.float32)
m = ms.nn.LayerNorm([3, 4])
output = m(x).shape
print(output)
```

gamma和beta的shape都等于normalized\_shape，最终调用的是算子ops.LayerNorm。所以报错信息才会有For 'LayerNorm', gamma or beta shape must match input shape。

**4. 解决方案**

nn.LayerNorm的参数 begin\_norm\_axis设置为1， 同时begin\_params\_axis也设置为1。

```
import mindspore as ms
import numpy as np
x = ms.Tensor(np.ones([2, 3, 4]), ms.float32)
m = ms.nn.LayerNorm([3, 4],begin_norm_axis =1,begin_params_axis=1)
output = m(x).shape
print(output)
```

