Recommendation System

1 Recall

1.1 ItemCF Recall:

cos similarity,

1.2 Swing:

Another ItemCF that calculates the weights of different users. When two items can not calcualte the similarity directly, system can use this algorithm. But this algorithm needs to consider users should come from differnt groups.

• u1 (user) -> J1 (items)
• u2 (user) -> J2 (items)
• overlap(u1,u2) = J1∩J2. overlap higher, weights lower
• i1 (item) -> W1 (users)
• i2 (item) -> W2(users)

Sim i1 and i2:

$$\operatorname{sim}(i_1, i_2) = \sum_{u_1 \in V} \sum_{u_2 \in V} \frac{1}{\alpha + \operatorname{overlap}(u_1, u_2)}$$

1.3 UserCF

choose top k similar users. 

 

decrease the hot items' weight. if an item is very hot, we should decrease its weight. 

$$\operatorname{sim}(u_1, u_2) = \frac{\sum_{l \in I} \frac{1}{\log(1 + n_l)}}{\sqrt{|\mathcal{J}_1| \cdot |\mathcal{J}_2|}}$$

1.4 Matrix Compensation

use exposing data to predict unexposing items. It isn't an ideal method in industry.

1.5 two towers model

can not use as pre-ranking model because of no interaction.

Train

Pairwise: cos(sample+, a) > cos(sample-, a)

listwise: yi, cos(aij, yij), softmax

Loss, Triplet hinge loss

$$L(a, b^{+}, b^{-}) = \max\left\{0, \cos(a, b^{-}) + m - \cos(a, b^{+})\right\}$$

Sampling

Positive:  exposed and clicked

Negative:  sampling posibility = (clicking times)^0.75.  Batch negatives, cos(a, bi) - log pi.

can not use exposed not clicked as negative samples. because they are similary.

Online serving

item->database,

user->nn computing, dynamic.

Updating model

tf-record

every day. incremental

Data Augmentation

to improve low exposure item recommentation,

Feature transformation (random mask, droupout, complementry) -> softmax -> loss is cross entropy

 

ANN (Approximate Nearest Neighbor, ANN)

• clustering (k-means)
• index
• similarly search (IVF)
• HNSW (GRAPH)
• Vector database Milvus

 

1.6 Bloom Filter

hash, length to represent a item. can use lru to be the filter, can not delete. 

bilibili.com/video/BV1v24y1B7JH?spm_id_from=333.788.videopod.sections&vd_source=8c42b55896f8a24fb4f65b7a3383faa1

 

2 Ranking

1.1 multi-target ranking (final rank)

features: 

• user
• item
• query
• is_weekend ....

 

• loss = sum (ai*crossEntropy(yi, pi))
• sample imbalance: down sampling
• adjust prediction: $$p_{\text{true}} = \frac{\alpha \cdot p_{\text{pred}}}{(1 - p_{\text{pred}}) + \alpha \cdot p_{\text{pred}}}$$

1.2 Multiple Gates Mixture of Experts

• each expert is like a full-connection nn. 
• gate is softmax result as weights.
• than calculate the output, expert * weight = target.

1.3 score fusion

• different formula according to background

1.4 features

• item
• user
• context
• statistics

embedding

• id
• each item/user has its over embedding table, every time, look up to the table to get the embedding vector, then update. multiple cases is useful.

bucket

• age

continuous features

discrete features

1.5 pre-rank 

3 towers. can not use as recall because expensive, no ANN, index. 

3 Feature Crossing

3.1 FM

• cross every 2 feature pair

3.2 DCN Deep & Cross Network

Cross (Hadamard product)

Deep & Cross

3.2 LHUC

similar as CN. Use sigmoid * 2 to process features.

 

4 User history behavior

4.1 lastN

• choose last n items for a user
• calculate embedding
• average the embeddings

4.2 Din

• weight sum of item & last N user items (like attention)

5 recommendation diversity (Rerank)

5.1 Content based

Clip. (Text & Picture)

5.2 MMR (Maximal Marginal Relevance)

$$\mathrm{MR}_i = \theta \cdot \mathrm{reward}_i - (1-\theta)\cdot \max_{j\in S}\mathrm{sim}(i,j)$$

• i is an item that is already choosen
• j is an item that is not choosen

5.3 DPP

Hyperparallelepiped. 

$$\mathcal{P}(v_1,\ldots,v_k)=\left\{\alpha_1v_1+\cdots+\alpha_kv_k \;\middle|\; 0\le\alpha_1,\ldots,\alpha_k\le1\right\}$$

A larger volume indicates a higher level of recommendation diversity.

$$\underset{S:\,|S|=k}{\operatorname{argmax}} \;\theta \left(\sum_{j\in S}\mathrm{reward}_j\right)+ (1-\theta)\log\det\left(V_S^{T}V_S\right)$$

 

6 Cold Start

When you do AB test in cold start experiments, set your traffic carefully.

6.1 Recall

Two Towers Model

ItemID Embedding. All new items share a same default embedding. Find the most similar k items to calculate default embedding.

Cluster Recall

• Caculate x clusters like k-means. You can use similarity to calculate distance. Features are the results of CNN and Beart or other NN. Use Triplet Hinge Loss to train.
• Put a new item to the closest cluster. 
• From x clusters get n new items as canditates.

Look Alike (User based cold start, like red note or facebook, when you create a new note or vedio)

Calculate similarity among users.

• UserCF
• Embedding. Cosim
• Update item's embedding
• Find user candidates those may like this item

 

https://www.bilibili.com/video/BV1eZ4y1a7tG?spm_id_from=333.788.videopod.sections&vd_source=8c42b55896f8a24fb4f65b7a3383faa1