[[
wikihub
]]
Search
⌘K
Explore
Activity
People
For Agents
Sign in
Explore
Activity
People
For Agents
Sign in
×
@jemoka / Jemoka Knowledge Base / raw/paper/moe_review/kbhmoereview_fedus_switch_transformers.md
Suggest edit
Cancel
Submit suggestion
Title
Name
Note
--- title: "MOEReview Fedus: Switch Transformers" source: https://www.jemoka.com/posts/kbhmoereview_fedus_switch_transformers/ --- At scale, with regularization (including dropout), k=1 on expert routing is fine!