Co-authored-by: Ntianshuo78520a <707759223@qq.com>

* add log for memory stats

* fix string_split in einsum

* Set random seed for test_tensordot (#53004)

---------
Co-authored-by: NRuibiao Chen <chenruibiao@baidu.com>

* add host memory stats

* add ut

* Add large dim test of log_softmax

* fix

* delete bf16 of cross entropy in new frl

* delete bf16 of cross entropy grad

* add utest

* rm hack code

* add log overwrite flag

* use strtobool

Co-authored-by: Nsneaxiy <32832641+sneaxiy@users.noreply.github.com>

add utest

fix utest

* print modifed flags

* fix ref, opt print

* fix default getter

* fix ut

Co-authored-by: Ngongweibao <gongweibao@baidu.com>

* Fix the calculation of layer_norm_bwd

* fix

* save env log for each worker

* fix ut

Co-authored-by: Ntianshuo78520a <707759223@qq.com>