shawwn/gpt-2
View on GitHub ↗ ★ 108 Python
Code for the paper "Language Models are Unsupervised Multitask Learners"
5 mentions on Hacker News by 1 person: 0 stories, 5 comments · first 2019-11-05 · latest 2020-06-14
20082026
-
…just replace `api.foo`'s definition with whatever I want after attaching with a debugger. Example: https://github.com/shawwn/gpt-2/blob/e4868225382e8a475ef3c3d... I assume there's a much better way to do this, but the class system seems to…
comment by sillysaurusx · 2020-06-14 · in Hot-swapping Python 3 code -
…a simpler notebook: https://colab.research.google.com/drive/1ohuxvB7nuvcjpLLIF1L... !git clone https://github.com/shawwn/gpt-2 /content/gpt-2 %cd gpt-2 !pip3 install -r requirements.txt !python3 download_model.py 1558M !python3 train.py…
comment by sillysaurusx · 2020-01-30 · in Standardizing OpenAI’s deep learning framework on PyTorch -
…window. You'll notice that this file is one continuous sample. You can try it yourself: git clone https://github.com/shawwn/gpt-2 cd gpt-2 sudo pip3 install -r requirements.txt python3 download_model.py 117M python3 src/generate_samples.py…
comment by sillysaurusx · 2019-12-25 · in Show HN: I trained a GPT-2 1.5B Subreddit Simulator in 3 days using 88 TPUs -
…speed gains. Yet we're 11x slower than the GPU case, and here's a notebook showing a 11x slowdown." https://github.com/shawwn/gpt-2/issues/5 I'm suspecting that memory bandwidth is the bottleneck here for large models. This paper even…
comment by sillysaurusx · 2019-12-01 · in Filesystem devs should aim to make “badly written” app code “just work” (2009) -
…Snapshots are 5.8GB. notebook: https://twitter.com/theshawwn/status/1191800180192010246 code: https://github.com/shawwn/gpt-2 It's a fork of nshepperd's gpt-2 codebase ( https://github.com/nshepperd/gpt-2 ) which lets you fine-tune 117M…
comment by sillysaurusx · 2019-11-05 · in OpenAI Releases Largest GPT-2 Text Generation Model