google-research/text-to-text-transfer-transformer
View on GitHub ↗ ★ 6,565 Python
Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"
8 mentions on Hacker News by 8 people: 1 story, 7 comments · first 2019-10-25 · latest 2024-03-22
20082026
-
> Has Google/Alphabet publicly released any of their AI models yet? You mean NLP field changing models from Google like BERT [1]? or Transformers paper [2]? or T5 model [3] (used by company doing ChatGPT like search currently on the front…
comment by lossolo · 2022-12-08 · in AlphaCode Attention Visualization -
T5 was open-sourced on release (up to 11B params): https://github.com/google-research/text-to-text-transfer-tra... It is also available via Hugging Face transformers. However, the paper mentions T5-XXL is 4.6B, which doesn't fit any of the…
comment by minimaxir · 2022-05-23 · in Imagen, a text-to-image diffusion model -
I'm not sure whether I'd call 10e10 nearly the size of 10e12, but the dataset is open, so it shouldn't be an issue. https://github.com/google-research/text-to-text-transfer-tra...
comment by GrantZvolsky · 2021-02-18 · in Google Open-Sources Trillion-Parameter AI Language Model Switch Transformer -
I have to say that I really doubt this as they've released model weights for their several last few models that have been of nearly this size (and have been trained on the same dataset). See https://github.com/google-research/text-to-text-t…
comment by robkop · 2021-02-18 · in Google Open-Sources Trillion-Parameter AI Language Model Switch Transformer -
Why is Google's T5 not mentioned? https://github.com/google-research/text-to-text-transfer-tra...
comment by ghenson2 · 2021-01-11 · in Natural language generation: The commercial state of the art in 2020 -
The dataset can be obtained around the web. It's mostly CommonCrawl, Reddit, Toronto Book Corpus, and Wikipedia. You can find a very comparable corpus open sourced and easy to use on the [T5 repo]( https://github.com/google-research/text-to…
comment by stephenroller · 2020-07-20 · in Giving GPT-3 a Turing Test -
Although those are some great results, I wish I could try it out locally... https://github.com/google-research/text-to-text-transfer-tra... It drives me nuts that most of these papers / publications don't have code where I can just run: >…
comment by lettergram · 2019-10-25 · in Google T5 scores 88.9 on SuperGLUE Benchmark, approaching human baseline