Scaling Up LLM Reviews for Google Ads Content Moderation
Large language models (LLMs) are powerful tools for content moderation, but
their inference costs and latency make them prohibitive for casual use on large
datasets, such as the Google Ads repository. This study proposes a method for
scaling up LLM reviews for content moderation in Google Ads. First, we use
heuristics to select candidates via filtering and duplicate removal, and create
clusters of ads for which we select one representative ad per cluster. We then
use LLMs to review only the representative ads. Finally, we propagate the LLM
decisions for the representative ads back to their clusters. This method
reduces the number of reviews by more than 3 orders of magnitude while
achieving a 2x recall compared to a baseline non-LLM model. The success of this
approach is a strong function of the representations used in clustering and
label propagation; we found that cross-modal similarity representations yield
better results than uni-modal representations.
65d81210d1a8b32d807037a1
Wei Qiao
65d81210d1a8b32d807037a2
Tushar Dogra
65d81210d1a8b32d807037a3
Otilia Stretcu
claimed_verified
2024-03-07T09:16:05.611Z
65e8a55bf4335428792a9c16
/avatars/f6c2cf3076fa8146d6c1892f65d86c05.svg
Otilia Stretcu
user
otiliastr
65d81210d1a8b32d807037a4
Yu-Han Lyu
65d81210d1a8b32d807037a5
Tiantian Fang
65d81210d1a8b32d807037a6
Dongjin Kwon
65d81210d1a8b32d807037a7
Chun-Ta Lu
65d81210d1a8b32d807037a8
Enming Luo
65d81210d1a8b32d807037a9
Yuan Wang
65d81210d1a8b32d807037aa
Chih-Chun Chia
65d81210d1a8b32d807037ab
Ariel Fuxman
65d81210d1a8b32d807037ac
Fangzhou Wang
65d81210d1a8b32d807037ad
Ranjay Krishna
65d81210d1a8b32d807037ae
Mehmet Tek
Scaling Up LLM Reviews for Google Ads Content Moderation
Large language models (LLMs) are powerful tools for content moderation, but
their inference costs and latency make them prohibitive for casual use on large
datasets, such as the Google Ads repository. This study proposes a method for
scaling up LLM reviews for content moderation in Google Ads. First, we use
heuristics to select candidates via filtering and duplicate removal, and create
clusters of ads for which we select one representative ad per cluster. We then
use LLMs to review only the representative ads. Finally, we propagate the LLM
decisions for the representative ads back to their clusters. This method
reduces the number of reviews by more than 3 orders of magnitude while
achieving a 2x recall compared to a baseline non-LLM model. The success of this
approach is a strong function of the representations used in clustering and
label propagation; we found that cross-modal similarity representations yield
better results than uni-modal representations.
A method is proposed for scaling content moderation with large language models by clustering ads and reviewing representatives, achieving high recall with cross-modal similarity representations.
large language models LLMs content moderation Google Ads heuristics filtering duplicate removal clusters representative ads label propagation cross-modal similarity representations uni-modal representations