websearch
Multi-Task Learning and Web Search Ranking
Gordon Sun (孙国政)
Yahoo! Inc
March 2007
Outline:
Brief Review: Machine Learning in web search ranking and Multi-Task learning.
MLR with Adaptive Target Value Transformation – each query is a task.
MLR for Multi-Languages – each language is a task.
MLR for Multi-query classes – each type of queries is a task.
Future work and Challenges.
MLR (Machine Learning Ranking)
General Function Estimation and Risk Minimization:
Input: x = {x1, x2, …, xn}
Output: y
Training set: {yi, xi}, i = 1, …, n
Goal: Estimate mapping function y = F(x)
In MLR work:
x = x (q, d) = {x1, x2, …, xn} --- ranking features
y = judgment labeling: e.g. {P E G F B} mapped to {0, 1, 2, 3, 4}.
Loss Function: L(y, F(x)) = (y – F(x))2
Algorithm: MLR with regression.
Rank features construction
Query features:
query language, query word types (Latin, Kanji, …), …
Document features:
page_quality, page_spam, page_rank,…
Query-Document dependent features:
Text match scores in body, title, anchor text (TF/IDF, proximity), ...
Evaluation metric – DCG (Discounted Cumulative Gain)

where grades Gi = grade values for {P, E, G, F, B} (NDCG – 2n) DCG5 -- (n=5), DCG10 -- (n=10)
Distribution of judgment grades
Milti-Task Learning
Single-Task Learning (STL)
One prediction task (classification/regression):
to estimate a function based on oneTraining/testing set:
T= {yi, xi}, i = 1, …, n

Multi-Task Learning (MTL)
Multiple prediction tasks, each with their own training/testing set:

下一页