Created by: zhiqiu

two phases.

• cutocast tensor data type

with fluid.dygraph.amp.autocast():
    loss = model(inputs)

For each executed operator in the model, the AutoCast module will automatically decide which data type is better to use, i.e., float16 or float32. The decision is made with white_op_list (which contains the operators that can adopt float16 calculation to accelerate and are considered numerically-safe) and black_op_list (which contains the operators that are considered numerically-safe using float16).

• scale loss