[[
wikihub
]]
Search
⌘K
Explore
Activity
People
For Agents
Sign in
Explore
Activity
People
For Agents
Sign in
×
@jemoka / Jemoka Knowledge Base / raw/paper/moe_review/kbhmoereview_li_branch_train_merge.md
Suggest edit
Cancel
Submit suggestion
Title
Name
Note
--- title: "MOEReview Li: Branch-Train-Merge" source: https://www.jemoka.com/posts/kbhmoereview_li_branch_train_merge/ --- weighted parameter average of the existing experts (or copy the new perts) training each expert independently And then when inference we can use domain-conditioned averaging between the experts by computing: or by averaging the parameters of the experts.