• L
    Input multiplier range check for tanh/logistic kernel. (#2023) · 24010d0e
    lukmanr-cadence 提交于
    Code snippet which calculate Input multiplier and shift values below,
    
    ```cc
          double multiplier =
              static_cast<double>(input->params.scale) * 4096.0 * 3.0;
          data->input_left_shift = 0;
    
          while (multiplier <= 32767.0 / 2.0 && data->input_left_shift <= 30) {
            data->input_left_shift++;
            multiplier = multiplier * 2.0;
          }
    ```
    
    Usually this multiplier  value will be in 16 bit even though this variable is a 32 bit integer. Above while loop make sure that this value is normalized 16 bit, except for the cases where input->params.scale is more than 2.67. Scale factor for 16 bit activation will be more than 2.67 for the cases where float dynamic range is more than 174978.45(2.67*65535).This is extremely rare.
    
    Code snippet below where input multiplier is used in the kernel.
    
    ```cc
        int32_t input_data =
            ((*ptr_input_data) * input_multiplier + round) >>input_left_shift;
    ```
    
    It’s actually a 16x32 bit multiplication ( *ptr_input_data is 16 bit variable and input_multiplier is 32 bit variable) and result is stored in 32 bit. Non-saturated lower 32 bit result of 16x32 multiplication are stored in the input_data. So anyway the result will be wrong for the cases, where input multiplier is more than 16 bits.
    
    So a condition check in the prepare function for these kernel will be good. So the programmer can make use of 16x16 operation instead of 16x32 bit multiplication.
    
    BUG=see description
    24010d0e
tanh.cc 7.4 KB