1. 26 10月, 2021 1 次提交
    • S
      [Cherry-pick] Add FasterTokenizer Operator (#36716) · edff5b79
      Steffy-zxf 提交于
      * Add FasterTokenizer Operator (#34491)
      
      Add Tokenizer related functionalities for Transformer model in order that the process of training and predicting is consistent.
      
      * support the text string as an input Tensor
      * support the "VOCAB"unordered_map<wstring, int> as an input Tensor to lookup tokens
      * Tokenizer used for BERT. This tokenizer applies an end-to-end, text string to wordpiece tokenization.
      * It first applies basic tokenization, followed by wordpiece tokenization.
      
      * optimize fast tokenizer
      
      * remove const_cast
      Co-authored-by: Nzhoushunjie <zhoushunjie@baidu.com>
      Co-authored-by: Nwawltor <fangzeyang0904@hotmail.com>
      edff5b79
  2. 26 8月, 2021 1 次提交
    • W
      gc for newexecutor (#35085) · f1472039
      wanghuancoder 提交于
      * gc for newexecutor, test=develop
      
      * refine, test=develop
      
      * add interpretercore_gc_helper.h,test=develop
      
      * backup
      
      * gc whit thread and device_event, test=develop
      
      * refine, test=develop
      
      * refine, test=develop
      
      * refine, test=develop
      
      * refine, test=develop
      
      * fix bug, test=develop
      
      * refine, test=develop
      
      * refine, test=develop
      
      * refine, test=develop
      
      * add CheckGC, test=develop
      f1472039
  3. 20 7月, 2021 1 次提交
  4. 13 7月, 2021 1 次提交
  5. 26 3月, 2021 1 次提交
  6. 04 2月, 2021 1 次提交
  7. 24 9月, 2020 1 次提交
    • W
      use iwyu clean include (#27267) · df43905f
      wanghuancoder 提交于
      * use iwyu clean include, test=develop, test=win
      
      * compilation error, test=develop
      
      * fix compilation error2, test=develop
      
      * fix compilation error3, test=develop
      
      * fix compilation error4, test=develop
      
      * fix compilation error5, test=develop
      
      * fix compilation error6, test=develop
      
      * fix compilation error7, test=develop
      
      * fix compilation error8, test=develop
      
      * fix compilation error8, test=develop
      
      * fix compilation error10, test=develop
      
      * fix compilation error11, test=develop
      df43905f
  8. 24 7月, 2020 1 次提交
  9. 18 9月, 2019 1 次提交
  10. 26 3月, 2019 1 次提交
  11. 25 3月, 2019 1 次提交
  12. 24 3月, 2019 1 次提交