@jemoka / Jemoka Knowledge Base › raw / paper / moe_review / kbhmoereview_rajbhandari_deepspeed_moe
Public View
Suggest
Download this page (.md) Download entire wiki (.zip)
Clone entire wiki

MOEReview Rajbhandari: DeepSpeed MoE

Proposes: more MoEs at later layers + a shared expert.

Page updated