As dangerous as underfitting is.

Overfitting and data sufficiency

Ironically enough, it is neural nets themselves that provide us an *echo-chamber of information*; to which we end up becoming overfit Aka - YouTube, Facebook, Twitter optimize the over-fitting of humans

Then obtain a balance with SVM kernles

there needs to be an essay like “worse is better” but for training models. there should always be a getting started memo for every domain that’s important.

I think we've learnt in the past months that most LLMs are actually suffering from a specific sort of underfitting. Datasets too small & models too large. Train smaller models for many more epochs on larger datasets

We all overfit as we grow older…

Haha, true that! If in real-life context, would you instead get more data, reduce the model size or perform regularization?

The point may be that you are always either over or under fitting. There is no perfect fit, only plausibly functional and serviceable fits. “All models are wrong, but some are useful”— Quip from George Box

Yes