Deep Structured Semantic Models (DSSM) is simple but powerful DNN based model for matching web search queries and the URL based documents. This example demonstrates how to use PaddlePaddle to implement a generic DSSM model for modeling the semantic similarity between two strings.
DSSM \[[1](##References)]is a classic semantic model proposed by the Institute of Physics. It is used to study the semantic distance between two texts. The general implementation of DSSM is as follows.
In the original paper \[[1](#References)] the DSSM model uses the implicit semantic relation between the user search query and the document as metric. The model structure is as follows
<palign="center">
<imgsrc="./images/dssm.png"/><br/><br/>
图 1. DSSM 原始结构
Figure 1. DSSM In the original paper
</p>
其贯彻的思想是， **用DNN将高维特征向量转化为低纬空间的连续向量（图中红色框部分）** ，
**在上层用cosin similarity来衡量用户搜索词与候选文档间的语义相关性** 。
在最顶层损失函数的设计上，原始模型使用类似Word2Vec中负例采样的方法，
一个Query会抽取正例 $D+$ 和4个负例 $D-$ 整体上算条件概率用对数似然函数作为损失，
这也就是图 1中类似 $P(D_1|Q)$ 的结构，具体细节请参考原论文。
随着后续优化DSSM模型的结构得以简化\[[3](#参考文献)\]，演变为：
With the subsequent optimization of the DSSM model to simplify the structure \[[3](#References)]，the model becomes：
The blank box in the figure can be replaced by any model, such as fully connected FC, convoluted CNN, RNN, etc. The structure is designed to measure the semantic distance between two elements (such as strings).
- 分类
- [-1, 1] 值域内的回归
- Pairwise-Rank
In practice，DSSM model serves as a basic building block, with different loss functions to achieve specific functions, such as
在生成低纬语义向量的模型结构上，本模型支持以下三种：
- In ranking system, the pairwise rank loss function.
- In the CTR estimate, instead of the binary classification on the click, use cross-entropy loss for a classification model
- In regression model, the cosine similarity is used to calculate the similarity
- FC, 多层全连接层
- CNN，卷积神经网络
- RNN，递归神经网络
## Model Implementation
At a high level, DSSM model is composed of three components: the left and right DNN, and loss function on top of them. In complex tasks, the structure of the left DNN and the light DNN can be different. In this example, we keep these two DNN structures the same. And we choose any of FC, CNN, and RNN for the DNN architecture.
In PaddlePaddle, the loss functions are supported for any of classification, regression, and ranking. Among them, the distance between the left and right DNN is calculated by the cosine similarity. In the classification task, the predicted distribution is calculated by softmax.
本例中为了简便和通用，将左右两个DNN的结构都设为相同的，因此只有三个选项FC,CNN,RNN等。
Here we demonstrate:
在损失函数的设计方面，也支持三种，分类, 回归, 排序；
其中，在回归和排序两种损失中，左右两边的匹配程度通过余弦相似度（cossim）来计算；
在分类任务中，类别预测的分布通过softmax计算。
- How CNN, FC do text information extraction can refer to [text classification](https://github.com/PaddlePaddle/models/blob/develop/text_classification/README.md#模型详解)
- The contents of the RNN / GRU can be found in [Machine Translation](https://github.com/PaddlePaddle/book/blob/develop/08.machine_translation/README.md#gated-recurrent-unit-gru)
- For Pairwise Rank learning, please refer to [learn to rank](https://github.com/PaddlePaddle/models/blob/develop/ltr/README.md)
在其它教程中，对上述很多内容都有过详细的介绍，例如：
- 如何CNN, FC 做文本信息提取可以参考 [text classification](https://github.com/PaddlePaddle/models/blob/develop/text_classification/README.md#模型详解)
Since the input (embedding table) is a list of the IDs of the words corresponding to a sentence, the word vector table outputs the sequence of word vectors.
In the construction of FC, we use `paddle.layer.pooling` for the maximum pooling operation on the word vector sequence. Then we transform the sequence into a fixed dimensional vector.
1. Huang P S, He X, Gao J, et al. Learning deep structured semantic models for web search using clickthrough data[C]//Proceedings of the 22nd ACM international conference on Conference on information & knowledge management. ACM, 2013: 2333-2338.
2.[Microsoft Learning to Rank Datasets](https://www.microsoft.com/en-us/research/project/mslr/)
