Created by: chengduoZH
fix https://github.com/PaddlePaddle/Paddle/issues/8728 related PR: https://github.com/PaddlePaddle/Paddle/pull/9216
CUDA pinned memory is different with CPU memory and GPU memory, physically, it's at CPU side, but it can be accessed by CPU and GPU. In the last PR, I add an argument, is_pinned_, to Tensor, but if a user is careless, it is dangerous that pinned memory's tensor is involved in the model computation, which will lead to slow training speed and very difficult to find. So After talking with @typhoonzero, I add the CUDAPinnedPlace.
Note: Currently, pinned memory is only used for memory copying.
This PR's work:
- add CUDAPinnedPlace
- add CUDAPinnedContext. In order to be compatible with CPUPlace and CUDAPlace.
- add Copy case (CUDAPinnedPlace-> CUDAPlace, CUDAPlace-> CUDAPinnedPlace, CUDAPinnedPlace-> CPUPlace, CPUPlace-> CUDAPinnedPlace, CUDAPinnedPlace-> CUDAPinnedPlace)