1. 14 9月, 2020 1 次提交
    • Z
      Update amp_check_finite_and_scale_op and add an updating_loss_scaling op for... · d708b210
      Zhen Wang 提交于
      Update amp_check_finite_and_scale_op and add an updating_loss_scaling op for static graph amp training. (#26240)
      
      * update amp_check_finite_and_scale_op for static_amp.
      
      * use amp_check_finite_and_scale in static graph amp.
      
      * update grads to zero when grads own infinite values(as for amp_checkout_finite_and_scale op).
      
      * add update_loss_scaling op in cpp.
      
      * add update_loss_scaling_op unit test.
      
      * update the doc of the check_finite_and_unscale op
      
      * Update the process of gradients updating skipping if the gradients have infinite values.
      
      * update the way to zero grads.
      
      * update test_update_loss_scaling_op.py
      
      * add log info when find infinite grads.
      
      * add the unit test for UpdateLossScaling Layer.
      d708b210
  2. 13 8月, 2020 1 次提交
    • L
      Feature/Enable Auto-Mixed-Precision in dynamic graph (#24903) · 2d95280e
      Leo Chen 提交于
      * add auto_cast, test=develop
      
      * add loss scaler, test=develop
      
      * add comments, test=develop
      
      * refine code, test=develop
      
      * refine code, test=develop
      
      * do not set flags automatically, test=develop
      
      * fix custom op bug, test=develop
      
      * add more test, test=develop
      
      * refine enable logic, test=develop
      
      * enable amp test with GPU, test=develop
      
      * add unittest
      
      * add test for found_inf
      
      * follow comments
      
      * follow comments
      
      * remove global variable, use singleton
      
      * add some notes
      
      * update comments
      
      * update comments
      
      * update comments
      
      * add use_dynamic_loss_scaling argument
      
      * refine found_inf
      
      * refine found_inf
      2d95280e