* vectorize lamb kernel

* remove flags, add ut

* remove useless codes

* refine code, add param order

* add multi tensor apply l2 norm

* add multi_tensor_apply code

* make sizeof(TensorMeta) smalller

* move code to distributed_fused_lamb_op.cu

* remove useless FLAGS

* add DistributedFusedLamb op

* polish code

* fix compile error

* compatible with pten changement

* fix rocm compile error

* improve converage

* update upstream/develop

* fix cast_with_ptr.h

* add FLAGS_distributed_lamb_divide_nranks_when_allreduce=1

* fix clip before allreduce

* add use_master_param_norm

* code polish

* fix bug

* fix ROCM ci

* cache core.ops, test=develop

* refine, test=develop

* update 2.0 public api in all left files

* reverse device.py all list;
fix some flake8 errors

* test=develop, fix test_lookahead

* test=develop, add model_average and lookahead