筆記：Wide & Deep Learning for Recommender Systems

筆記：Wide & Deep Learning for Recommender Systems

前兩天自從看到一張圖後：

就一直想讀一下相關論文，這兩天終於有時間把論文看了一下，就是這篇Wide & Deep Learning for Recommender Systems

首先簡介，主要說了什麼是Wide和Deep：

Wide就是：wide是指高維特征+特征組合的LR， 原文Generalized linear models with nonlinear feature transformations are widely used for large-scale regression and classification problems with sparse inputs. Memorization of feature interactions through a wide set of cross-product feature transformations are effective and interpretable, while generalization requires more feature engineering effort. 

Deep就是：深度神經網絡，原文：With less feature engineering, deep neural networks can generalize better to unseen feature combinations through low-dimensional dense embeddings learned for the sparse features. However, deep neural networks with embeddings can over-generalize and recommend less relevant items when the user-item interactions are sparse and high-rank. 

然後就是本文介紹如何整合Wide和Deep

主要內容：

兩個有意思的概念Memorization和Generalization：

Memorization can be loosely defined as learning the frequent co-occurrence of items or features and exploiting the correlation available in the historical data.

Generalization, on the other hand, is based on transitivity of correlation and explores new feature combinations that have never or rarely occurred in the past.

回顧LR和深度學習的方法。

介紹他們的實踐，一些細節

目標App Acquisitions

對比join training和ensemble。ensemble是disjoint的。join training可以一起優化整個模型。

訓練時候LR部分是FTRL+L1正則，深度學習用的AdaGrad?

訓練數據有500 個billion。這是怎麼算的，這麼NB?

連續值先用累計分布函數CDF歸一化到[0,1]，再劃檔離散化。這個倒是不錯的trick。

文章不長寫的挺有意思的，大家可以下來細讀一下。