Merge pull request #274 from reyoung/feature/refine_word_vectors

Use sparse update for word vectors.

04.word2vec/README.en.md

@@ -235,7 +235,8 @@ def wordemb(inlayer):
             name="_proj",
             initial_std=0.001,
             learning_rate=1,
-            l2_rate=0, ))
+            l2_rate=0,
+            sparse_update=True))
     return wordemb
 ```
 
@@ -301,10 +302,10 @@ cost = paddle.layer.classification_cost(input=predictword, label=nextword)
 
 ```python
 parameters = paddle.parameters.create(cost)
-adam_optimizer = paddle.optimizer.Adam(
+adagrad = paddle.optimizer.AdaGrad(
     learning_rate=3e-3,
     regularization=paddle.optimizer.L2Regularization(8e-4))
-trainer = paddle.trainer.SGD(cost, parameters, adam_optimizer)
+trainer = paddle.trainer.SGD(cost, parameters, adagrad)
 ```
 
 Next, we will begin the training process. `paddle.dataset.imikolov.train()` and `paddle.dataset.imikolov.test()` is our training set and test set. Both of the function will return a **reader**: In PaddlePaddle, reader is a python function which returns a Python iterator which output a single data instance at a time.

04.word2vec/README.md

@@ -220,7 +220,8 @@ def wordemb(inlayer):
             name="_proj",
             initial_std=0.001,
             learning_rate=1,
-            l2_rate=0, ))
+            l2_rate=0,
+            sparse_update=True))
     return wordemb
 ```
 
@@ -290,10 +291,10 @@ cost = paddle.layer.classification_cost(input=predictword, label=nextword)
 
 ```python
 parameters = paddle.parameters.create(cost)
-adam_optimizer = paddle.optimizer.Adam(
+adagrad = paddle.optimizer.AdaGrad(
     learning_rate=3e-3,
     regularization=paddle.optimizer.L2Regularization(8e-4))
-trainer = paddle.trainer.SGD(cost, parameters, adam_optimizer)
+trainer = paddle.trainer.SGD(cost, parameters, adagrad)
 ```
 
 下一步，我们开始训练过程。`paddle.dataset.imikolov.train()`和`paddle.dataset.imikolov.test()`分别做训练和测试数据集。这两个函数各自返回一个reader——PaddlePaddle中的reader是一个Python函数，每次调用的时候返回一个Python generator。

04.word2vec/index.en.html

@@ -277,7 +277,8 @@ def wordemb(inlayer):
             name="_proj",
             initial_std=0.001,
             learning_rate=1,
-            l2_rate=0, ))
+            l2_rate=0,
+            sparse_update=True))
     return wordemb
 ```
 
@@ -343,10 +344,10 @@ cost = paddle.layer.classification_cost(input=predictword, label=nextword)
 
 ```python
 parameters = paddle.parameters.create(cost)
-adam_optimizer = paddle.optimizer.Adam(
+adagrad = paddle.optimizer.AdaGrad(
     learning_rate=3e-3,
     regularization=paddle.optimizer.L2Regularization(8e-4))
-trainer = paddle.trainer.SGD(cost, parameters, adam_optimizer)
+trainer = paddle.trainer.SGD(cost, parameters, adagrad)
 ```
 
 Next, we will begin the training process. `paddle.dataset.imikolov.train()` and `paddle.dataset.imikolov.test()` is our training set and test set. Both of the function will return a **reader**: In PaddlePaddle, reader is a python function which returns a Python iterator which output a single data instance at a time.

04.word2vec/index.html

@@ -262,7 +262,8 @@ def wordemb(inlayer):
             name="_proj",
             initial_std=0.001,
             learning_rate=1,
-            l2_rate=0, ))
+            l2_rate=0,
+            sparse_update=True))
     return wordemb
 ```
 
@@ -332,10 +333,10 @@ cost = paddle.layer.classification_cost(input=predictword, label=nextword)
 
 ```python
 parameters = paddle.parameters.create(cost)
-adam_optimizer = paddle.optimizer.Adam(
+adagrad = paddle.optimizer.AdaGrad(
     learning_rate=3e-3,
     regularization=paddle.optimizer.L2Regularization(8e-4))
-trainer = paddle.trainer.SGD(cost, parameters, adam_optimizer)
+trainer = paddle.trainer.SGD(cost, parameters, adagrad)
 ```
 
 下一步，我们开始训练过程。`paddle.dataset.imikolov.train()`和`paddle.dataset.imikolov.test()`分别做训练和测试数据集。这两个函数各自返回一个reader——PaddlePaddle中的reader是一个Python函数，每次调用的时候返回一个Python generator。