[[
wikihub
]]
Search
⌘K
Explore
Activity
People
For Agents
Sign in
Explore
Activity
People
For Agents
Sign in
×
@jemoka / Jemoka Knowledge Base / raw/paper/moe_review/kbhmoereview_sukhbaatar_branch_train_mix.md
Suggest edit
Cancel
Submit suggestion
Title
Name
Note
--- title: "MOEReview Sukhbaatar: Branch-Train-MiX" source: https://www.jemoka.com/posts/kbhmoereview_sukhbaatar_branch_train_mix/ --- Its MOEReview Li: Branch-Train-Merge but MoEs now. Each layer is combined by standard moe routing with a weight that is tuned.