I am really confused by OpenAI's PR strategy for GPT-2; they push it heavily as the future of text writing, but all of their GPT-2 related repos are archived w/o any updates expected. newyorker.com/magazine/2019/…
2
2
16
Replying to @minimaxir
We tend to act as a seed crystal for new advances. There are now many popular, actively maintained GPT-2 code repositories (such as your very own github.com/minimaxir/gpt-2-s…) or interactive websites (talktotransformer.com). We focus on making the next breakthrough.

Oct 7, 2019 · 5:02 PM UTC

2
6
Replying to @gdb
The catalyst for my development of gpt-2-simple was *because* there wasn't a good, easy-to-use interface for working with GPT-2 at the time. I don't get any personal benefit from maintaining it now; I do it for the community.
3
Replying to @gdb @minimaxir
I feel a bit behind, it's kind of hard to figure out what the latest/best fork is for fine-tuning. I think it's probably github.com/mkturkcan/GPTune ? Is is still impossible to train all layers with Adam on a 774M? (Not that I can ever get a Colab T4 anyway...)

GitHub - mkturkcan/GPTune: Fine tuning experiments for the GPT-2 model by OpenAI.

Fine tuning experiments for the GPT-2 model by OpenAI. - GitHub - mkturkcan/GPTune: Fine tuning experiments for the GPT-2 model by OpenAI.

github.com