- 26 10月, 2021 1 次提交
-
-
由 Steffy-zxf 提交于
* Add FasterTokenizer Operator (#34491) Add Tokenizer related functionalities for Transformer model in order that the process of training and predicting is consistent. * support the text string as an input Tensor * support the "VOCAB"unordered_map<wstring, int> as an input Tensor to lookup tokens * Tokenizer used for BERT. This tokenizer applies an end-to-end, text string to wordpiece tokenization. * It first applies basic tokenization, followed by wordpiece tokenization. * optimize fast tokenizer * remove const_cast Co-authored-by: Nzhoushunjie <zhoushunjie@baidu.com> Co-authored-by: Nwawltor <fangzeyang0904@hotmail.com>
-
- 31 8月, 2021 1 次提交
-
-
由 Shang Zhizhou 提交于
* Revert "Revert "Add copy from tensor (#34406)" (#35173)" This reverts commit 32c1ec42. * add template instantiation
-
- 27 8月, 2021 1 次提交
-
-
由 zhangchunle 提交于
This reverts commit ac33c0ca.
-
- 26 8月, 2021 1 次提交
-
-
由 Shang Zhizhou 提交于
* add api * temp save * revert * copytocpu async ok * fix style * copy sync ok * fix compile error * fix compile error * api done * update python async api * fix compile * remove async python api; add c++ async unittest * remove python async api * update unittest * update unittest * add C++ unittest for copytensor * add unittest * update namespace utils to class TensorUtils * add unittest * update unittest * update unittest * update code style * update code style * update unittest
-
- 12 8月, 2021 1 次提交
-
-
由 Wilber 提交于
-
- 14 7月, 2021 1 次提交
-
-
由 Wilber 提交于
-
- 05 3月, 2021 1 次提交
-
-
由 石晓伟 提交于
-