*[Have I installed Fluid successfully?](#have-i-installed-fluid-successfully)
*[Prepare Data](#prepare-data)
*[Train and evaluate](#train-and-evaluate)
*[Models](#models)
*[Results](#results)
## Introduction
### a brief review of the Quora Question Pair (QQP) Task
...
...
@@ -25,19 +41,19 @@ Based on the Quora Question Pair Dataset, we will implement some classic models
## Environment Preparation
### Install fluid release 1.0
### Install Fluid release 1.0
You can follow the fluid's [official document](http://www.paddlepaddle.org/documentation/docs/en/1.0/build_and_install/pip_install_en.html) to install the fluid.
[Attention] You are supposed to install python and pip before installing fluid
### cpu version
#### cpu version
```
pip install paddlepaddle==1.0.1
```
### gpu version
#### gpu version
Assume you have downloaded cuda(cuda9.0) and cudnn(cudnn7) lib, here is an expample:
...
...
@@ -57,12 +73,6 @@ python -c "import paddle"
Fluid is installed successfully if no error message is prompted. If you get any error, feel free to open issues under the [PaddlePaddle repository](https://github.com/PaddlePaddle/Paddle/issues).
### Install nltk module
```shell
pip install nltk
```
## Prepare Data
Please download the Quora dataset firstly from [google drive](https://drive.google.com/file/d/0B0PlTAo--BnaQWlsZl9FZ3l1c28/view?usp=sharing)
...
...
@@ -90,7 +100,7 @@ $HOME/.cache/paddle/dataset
## Train and evaluate
We provide multiple models and configs, details are shown in models and configs directory. For quick start, you can run the cdssmNet with cdssm_base config:
We provide multiple models and configs, details are shown in `models` and `configs` directory. For quick start, you can run the cdssmNet with cdssm_base config:
We have implemeted 4 models for now, CDSSM(Convolutional Deep Structured Semantic Models) is a convolution-based model, Infer Sent Model and SSE(Shortcut-Stacked Encoders) are RNN-based models， and DecAtt(Decompose Attention) model is an attention-based model.
title={Learning semantic representations using convolutional neural networks for web search},
author={Shen, Yelong and He, Xiaodong and Gao, Jianfeng and Deng, Li and Mesnil, Gr{\'e}goire},
booktitle={Proceedings of the 23rd International Conference on World Wide Web},
pages={373--374},
year={2014},
organization={ACM}
}
```
|Model|dev accuracy| test accuracy
|:----:|:----:|:----:|
|CDSSM|||
|InferSent|||
|SSE|||
|DecAtt|||
### InferSent
```
@article{conneau2017supervised,
title={Supervised learning of universal sentence representations from natural language inference data},
author={Conneau, Alexis and Kiela, Douwe and Schwenk, Holger and Barrault, Loic and Bordes, Antoine},
journal={arXiv preprint arXiv:1705.02364},
year={2017}
}
```
### SSE
```
@article{nie2017shortcut,
title={Shortcut-stacked sentence encoders for multi-domain inference},
author={Nie, Yixin and Bansal, Mohit},
journal={arXiv preprint arXiv:1708.02312},
year={2017}
}
```
### DecAtt
```
@article{tomar2017neural,
title={Neural paraphrase identification of questions with noisy pretraining},
author={Tomar, Gaurav Singh and Duque, Thyago and T{\"a}ckstr{\"o}m, Oscar and Uszkoreit, Jakob and Das, Dipanjan},
journal={arXiv preprint arXiv:1704.04565},
year={2017}
}
```
## Results
In our experiment, we found that LSTM-based models outperform convolution-based model in test set accuracy. DecAtt model has fewer parameters than LSTM-based models, but it is very sensitive to the hyper-parameters when training.
