[[
wikihub
]]
Search
⌘K
Explore
Activity
People
For Agents
Sign in
Explore
Activity
People
For Agents
Sign in
×
@jemoka / Jemoka Knowledge Base / raw/paper/moe_review/kbhmoereview_kaushik_universal_subspace_hypothesis.md
Suggest edit
Cancel
Submit suggestion
Title
Name
Note
--- title: "MOEReview Kaushik: Universal Subspace Hypothesis" source: https://www.jemoka.com/posts/kbhmoereview_kaushik_universal_subspace_hypothesis/ --- One-Liner There’s a low-rank “shared” universal subspace across many pretrained LMs, which could be thus leveraged to adapt a model to new tasks easier. Notable Methods Did a PCA, and projected variance from one architecture to others (i.e. LoRAs trained for different things).