@jemoka / Jemoka Knowledge Base › raw / paper / moe_review / kbhmoereview_sukhbaatar_branch_train_mix
Public View
Suggest
Download this page (.md) Download entire wiki (.zip)
Clone entire wiki

MOEReview Sukhbaatar: Branch-Train-MiX

Its MOEReview Li: Branch-Train-Merge but MoEs now. Each layer is combined by standard moe routing with a weight that is tuned.

Page updated