@@ -35,7 +35,7 @@ Figure 2. Candidate generation model architecture
...
@@ -35,7 +35,7 @@ Figure 2. Candidate generation model architecture
- Output layer: A softmax classifier is connected to do discriminating millions of classes (videos). To speed up training process, a technique is applied that samples negative classes from background distribution with importance weighting. The previous mentioned high-dimensional "embedding" of the candidate video  is obtained by weight and bias of the softmax layer. At serving time, the most likely N classes (videos) is computed for presenting to the user. To Score millions of items under a strict serving laterncy, the scoring problem reduces to a nearest neighbor search in the dot product space, and Locality Sensitive Hashing is relied on.
- Output layer: A softmax classifier is connected to do discriminating millions of classes (videos). To speed up training process, a technique is applied that samples negative classes from background distribution with importance weighting. The previous mentioned high-dimensional "embedding" of the candidate video  is obtained by weight and bias of the softmax layer. At serving time, the most likely N classes (videos) is computed for presenting to the user. To Score millions of items under a strict serving laterncy, the scoring problem reduces to a nearest neighbor search in the dot product space, and Locality Sensitive Hashing is relied on.
## Data Pre-processing
## Data Pre-processing
In this example, we moked click log of users as sample data, and its format is as follows:
In this example, here moke the click log of users as sample data, and its format is as follows:
```
```
user-id \t province \t city \t history-clicked-video-info-sequence \t phone
user-id \t province \t city \t history-clicked-video-info-sequence \t phone
We improves the original networks in \[[1](#References)\] by modifying that the embeddings of video watches are not simply averaged but are connected to a LSTM layer with max temporal pooling instead, so that the deep sequential information related to user interests can be learned well. Considering data scale and efficiency of training, we only apply two ReLU layers, which also leads to good performance.
We improves the original networks in \[[1](#References)\] by modifying that the embeddings of video watches are not simply averaged but are connected to a LSTM layer with max temporal pooling instead, so that the deep sequential information related to user interests can be learned well. Considering data scale and efficiency of training, only two ReLU layers are applied, which also leads to good performance.
For online prediction，we adopt Approximate Nearest Neighbor(ANN) to directly recall top N most likely watch video. However, our ANN system currently only supports cosin sorting, not by inner product sorting, which leads to big effect difference.
For online prediction，Approximate Nearest Neighbor(ANN) is adopted to directly recall top N most likely watch video. However, our ANN system currently only supports cosin sorting, not by inner product sorting, which leads to big effect difference.
As a result, we sliently modify user and video vectors by a SIMPLE-LSH conversion\[[4](#References)\], so that inner sorting is equivalent to cosin sorting after conversion.
To solve it, user and video vectors are sliently modified by a SIMPLE-LSH conversion\[[4](#References)\], so that inner sorting is equivalent to cosin sorting after conversion.
Details are as follows:
Details are as follows:
- For video vector , we have . The modified video vector , and let .
- For video vector , . The modified video vector , and let .
- For user vector , and the modified user vector , and let , where  is normalized .
- For user vector , and the modified user vector , and let , where  is normalized .
When online predicting, for a coming , we need recall  by inner product sorting. After  conversion, the order of inner prodct sorting is unchanged. Since  and  are both equal to 1, %20%3D%20%5Ctilde%7B%5Cmathbf%7Bu%7D%7D%5Ccdot%20%5Ctilde%7B%5Cmathbf%7Bv%7D%7D), which makes cosin-supported-only ANN system works.
When online predicting, for a coming , it should recall  by inner product sorting. After  conversion, the order of inner prodct sorting is unchanged. Since  and  are both equal to 1, %20%3D%20%5Ctilde%7B%5Cmathbf%7Bu%7D%7D%5Ccdot%20%5Ctilde%7B%5Cmathbf%7Bv%7D%7D), which makes cosin-supported-only ANN system works.
And in order to retain precision, we find that  is also equivalent.
And in order to retain precision, use  is also equivalent.
Use `user_vector.py` and `vector.py` to calculate user and item vectors. For example, run the following commands:
Use `user_vector.py` and `vector.py` to calculate user and item vectors. For example, run the following commands:
