* logsigmoid and LogSigmoid, test=develop

* add elu gelu relu, test=develop

* update to_variable to to_tensor, test=develop

* address review comments, test=develop

* address review comments, test=develop

* change to_variable to to_tensor in test, test=develop

Update the sigmoid, Sigmoid layer for the api2.0

* test=develop,test=document_fix, remove the out args

* Revert "test=develop,test=document_fix, remove the out args"

This reverts commit f430799b.

* test=develop,test=document_fix, remove the out args

* remove all  the out args

* remove all  the out args

* test=develop, remove all  the out args

* test=develop, remove all  the out args

* test=develop, remove out args in matmul testcase

* test=develop, fix the bugs of out args testcase in logsumexp

* test=develop, fix the bugs of out args testcase in elementwise_add

* test=develop, fix the bugs of out args testcase in elementwise_div

* test=develop, fix the bugs of out args testcase in activation

* test=develop, fix the bugs of out args testcase in addcmul

* test=develop, fix the bugs of out args

* test=develop, fix the bugs of out args in API example

* test=develop, fix the bugs of out args in kron testcase

* test=develop, fix the bugs of out args in kron testcase

* test=develop, remove out args

* test=develop, remove out args

* test=develop, fix the coverage

* test=develop, polish codestyle

* test=develop, Update math.py

* test=develop, polish codestyle

* test=develop, polish codestyle

* test=develop, polish codestyle

* test=develop, polish codestyle

* test=develop, polish the test_activation_op.py

* test=develop, Update math.py

* test=develop, fix the failed CI

* test=develop, add core.ops

* test=develop, add core.ops change clamp to clip

* test=develop, add core.ops change clamp to clip

* test=develop, write testcase for clamp OP

* New features, add sinh and cosh op, test=develop

* remove duplicate test function and remove out paramters, test=develop

* Add out paramters temporary, remove later. test=develop

* remove out args, PR 25570, test=develop

* remove TestParameter, test=developx

* add test api for static dygraph, test=develop

* add backword unittests for sinh and cosh, test=develop

* fix test_activation_op, test=develop

* remove paddle.mul related unittest, test=develop

add new tensor: max,min,t,eye,log1p; test=develop

1. update error info of the ops (abs, acos, asin, atan, ceil, cos, exp, floor, log, pow, reciprocal, round, rsqrt, sin, sqrt, square, tanh)
2. add the unittests of the above refered ops (test error info)

Ops(relu6/selu/soft_relu/softshrink/stanh/swish/thresholded_relu/hard_shrink/hard_sigmoid/hard_swish/hsigmoid/maxout) error message enhancement (#23718)

*  add new op paddle.tensor.mul(x, y, x_num_col_dims=1, y_num_col_dims=1, out=None, name=None)
* add new op paddle.tensor.pow(input, exponent, out=None, name=None)
* add the aboved two new ops unittest (test api param out and name)

* sin sqrt tanh atan add out, test=develop

* optimize doc, test=develop

* add dygraph test, test=develop

add approximation for gelu, default value is False (only kernel with eigen is added, remove code for computing gelu with MKLDNN temporarily)

enhanced ops: acos, asin, brelu, ceil, cos, elu, floor, hard_shrink,
hard_sigmoid, hard_swish, relu6, round, rsqrt, sin, softshrink,
cos_sim, cross_entropy, im2squence, kldiv_loss

* Remove self-set accuracy parameters of op tests: max_relative_error
test=develop

* fix errors
test=develop

sequence_unpad, expand, pad, pad_constant_like,
norm, bilinear_tensor_product, flatten2, im2sequence,
unpool, cos_sim, strided_slice, flatten, elementwise_min,
abs, acos

test=develop

* fix PythonAPI test in Op unittest, test=develop
* fix unit tests that do not need to inherit OpTest, test=develop

test=develop

* elu input check
test=develop

* test=develop

improve pow op according to reviews:
1. Delete unnecessary judgement statements in PowGradOpDescMaker;
2. Improve test of test_api;

overload GetKernelTypeForVar

add stop_gradient=True when attr(factor) is tensor Variable, change examples in API pow.
test=develop,test=document_preview

* add hard_swish activation op (new op)
test=develop

* remove redundancy files

* modify document content of HardSwish OP

* add API test in test_layers.py

* add dynamic_graph for test_hard_swish

test=develop

test=develop

add rsqrt op

test=develop

* staged.

* polish code

* polish code. test=develop

* polish code. test=develop

* api change. test=develop

* fix default value. test=develop

* fix default value. test=develop

* staged.

* polish code

* polish code. test=develop

* polish code. test=develop

* api change. test=develop

* fix default value. test=develop

* fix default value. test=develop

* add fp16 backward support
test=develop

* add sum_op fp16 test

* disable test_dist_save_load
test=develop

* add check_grad for sum

* add unit test for softmax_grad fp16
test=develop

* add scale_op unit test

* add mul_grad_op unit test for fp16

* add cross_entropy_grad and eman_grad unit test for fp16
test=develop

* fix cross_entropy unit test

* add pool2d fp16 unit test

* refine conv2d fp16 unit test
test=develop

* refine activation unit test
test=develop

* fix ci
test=develop

* follow zhihong's comment, copy from https://github.com/PaddlePaddle/Paddle/pull/12796
test=develop

* Add Preface

* Add demo code

* Save file

* Refine code

* seems can work

* use elementwise strategy

* Use ElementwiseComputeEx

* Add comments

* extract functions from operator

* Refine code

* Follow comment

* code refine

* follow comments

* follow comments