Created by: kuke

Implement the CTC beam search decoder in C++ to speedup decoding. Compared with the prototype decoder in Python, this optimized decoder gets the identical decoding results and has the advantage of about 3x speedup in single thread (measured by the time module in Python) when given the same parameters.

To further achieve real-time decoding for deployment in reality, the width of beam search can be set appropriately. An experiment is carried out to illustrate the effect of beam size on the WER and speed of decoding with 100 samples, and here are some results:

It is not hard to find that when beam size < 200, the average time of one sample's decoding is limited to 1s without a significant decay in WER. Therefore by setting a proper beam size in this range, the decoding in deployment can be completed within acceptable time.