Created by: lcy-seso

In NLP field, Neural Machine Translation (NMT for short) is one of the most important tasks to evaluate how near our models and techniques approach the ultimate goal: understanding natural language. Besides, NMT models are always among the most complicated models in the NLP field. They require a deep learning framework should be both highly flexibile and computation efficient.

Making popular NMT models being well supported help us to speed up the process that our framework is capable of solving practical problems.

Here, I first list some candidate models advised by our partners. At the very beginning, we should make sure our design has a native support for these models. (Later, I will create a project to manage all the functional requirements.)

1. The RNN search model. This is a baseline model that MUST be well supported.
2. ConvS2S . This is also a very popular and basic model that is expected to be well supported. The key point of this model is through highly parallel computing to speed up the training process.
   • https://github.com/facebookresearch/fairseq
3. Transformer
   • Our design should support the Transformer model naturally and easily.
   • https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/models/transformer.py

Other models, we should think about:

1. Minimum Risk Training for Neural Machine Translation
   • Decoding during training helps to improve the performance of NMT. We should think about this.
2. Dual learning for NMT
   • This model is time-consuming, but we'd better think about its training process.

About decoding:

• Think about Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search besides the standard beam search.

Finally, I hope to summarize up:

• The top priority gives to RNN search model and ConvS2S.
• Guarantee our design can support the Transformer.