Hacker GitHub

← All repos

meta-pytorch/gpt-fast

View on GitHub ↗ ★ 6,269 Python

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

6 mentions on Hacker News by 5 people: 2 stories, 4 comments · first 2023-11-30 · latest 2024-06-13

Mentions per year 6 mentions · busiest year 2023 (5)
20082026
AllStoriesComments NewestTopOldest
  1. …example, their vllm benchmark on 2 GPUs sees 102 tokens/s for BS=1, gpt-fast gets around 190 tok/s. https://github.com/pytorch-labs/gpt-fast 2. As others have pointed out, they're comparing H100 running with TP=2 vs. 2 AMD GPUs running…
    comment by chillee · 2024-06-13 · in AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
  2. I'm wondering if gpt-fast has a version that can be run from Windows Command Prompt or Powershell? https://github.com/pytorch-labs/gpt-fast/issues/45
    comment by maxloo1976 · 2023-12-12 · in Accelerating Generative AI with PyTorch II: GPT, Fast
  3. …labs optimizations for Sam and llama2 ? https://github.com/pytorch-labs/segment-anything-fast https://github.com/pytorch-labs/gpt-fast
    comment by naveen99 · 2023-12-02 · in Show HN: 80% faster, 50% less memory, 0% loss of accuracy Llama finetuning
  4. …is that people will copy-paste and modify it for their own needs :) Code can also be found here: https://github.com/pytorch-labs/gpt-fast And a twitter thread summary here: https://twitter.com/cHHillee/status/1730293330213531844
    comment by chillee · 2023-11-30 · in Accelerating Generative AI with PyTorch II: GPT, Fast