Hi, friends, I found I cannot set opt_level=O1 using BERT of PyTorch code and I got the prompt "RuntimeError: expected scalar type Half but found Float".
Here are some details.
"/anaconda3/envs/cuda100p37/lib/python3.7/site-packages/apex/normalization/fused_layer_norm.py", line 25, in forward
input_, ctx.normalized_shape, weight_, bias_, ctx.eps)
RuntimeError: expected scalar type Half but found Float (data_ptrc10::Half at
/anaconda3/envs/cuda100p37/lib/python3.7/site-packages/torch/include/ATen/core/TensorMethods.h:5747)
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) + 0x6d (0x7f2072ac8e7d in /anaconda3/envs/cuda100p37/lib/python3.7/site-packag
es/torch/lib/libc10.so)
frame #1: c10::Half* at::Tensor::data_ptrc10::Half() const + 0x456 (0x7f203c9bc696 in /anaconda3/envs/cuda100p37/lib/python3.7/site-packages/fused_layer_norm_cuda.cpython-37m-x86_64-linux-gnu.so)
frame #2: cuda_layer_norm(at::Tensor*, at::Tensor*, at::Tensor*, at::Tensor*, int, int, c10::ArrayRef, at::Tensor*, at::Tensor*, double) + 0x3e8 (0x7f203c9b8058 in /anaconda3/envs/cuda100p37/lib/python3.7/site
-packages/fused_layer_norm_cuda.cpython-37m-x86_64-linux-gnu.so)
frame #3: layer_norm_affine(at::Tensor, c10::ArrayRef, at::Tensor, at::Tensor, double) + 0x2ba (0x7f203c9a16ea in /anaconda3/envs/cuda100p37/lib/python3.7/site-packages/fused_layer_norm_cuda.cpython-37m-x86_64
-linux-gnu.so)