“How do we build well developed AI systems without a bangin’ company” Two main paradigms transfer learning: (pretrain a model, and) faster convergence, better performance *monolithic models: (pretrain a model, and) just use the pretrained model Problems with monolythic models Continual development of large language models mostly don’t exist: no incremental updates To get better improvements, we throw out the old monolythic model Most of the research community can’t participate in their development New Alternative Paradigm A very simple routing layer A very large collection of specialist models all from a base model Collaborative model development means that a large amount of contributors can band together to contribute to the development of the models Why Specialist models are cheaper and better to train few shot parameter efficient fine tuning is better liu et al few shot fine-tuning is better than few-shot in-context learning Specialist models can be communicable, incremental updates to a base model think: PEFT each of the specialist models can only need to update a small percent of the weights think “adapters”: parameter efficient updates Routing task2vec: task embedding for meta learning Achille et al efficiently tuned parameters are task embeddings Zhou et al distinction between MoE instead of routing in sub-layer level routing, we are routing at the input level we look at the Novel Tasks (Model Merging) Tasks can be considered as a composition of skills. each task can be encoded as a composition of skills we can merge the skills of sub-models Usual updates we take a pretrained model we adapt it to some target task Model Merging Fisher-weight averaging “Merging models with fisher-weight averaging”, Matena et al Merging can be shown as an optimization problem:

“a merged model is the set of parameters which would maximize the log-posterior of each model \mathcal{D}_{i}, controlled by \lambda_{i}” Task arthmetic “Editing models with Task Arthmetic”, llharco et al “Resolving inference when merging models” by Yadev et al You can create multi-task models by just doing maff:

this apparently works ok. Soft MoE Soft merging of experts with adaptive routing, Muqeeth et al MoE, but instead of choosing an expert to activate, the router’s probability densities will result in a mixture of the experts’ weights. So, mulitple experts can be invoked in a linear way. Git-Theta Git-Theta: A Git Extension for Collaborative Development of Machine Learning Models, Kandpal et al Communal and iterative development of model checkpoints. Saves only LoRA’d parameters, and removes any weights that didn’t change between diffs. Petals Petals: Collaborative Inference and Fine-Tuning of Large Models, Borzunov et al. Distributed fine-tuning and model inference by using different sub-worker nodes to run different layers of the model. https://health.petals.dev/