Predicting Scaling Laws
Consider in Kaplan et al., 2020 scaling laws, at scale, we can get reasonably smooth trends for MMLU / ARC / etc.
We would like to predict these IN ADVANCE at smaller scale.
Consider in Kaplan et al., 2020 scaling laws, at scale, we can get reasonably smooth trends for MMLU / ARC / etc.
We would like to predict these IN ADVANCE at smaller scale.