pytorch/torchchat

View on GitHub ↗ ★ 3,609 Python archived

Run PyTorch LLMs locally on servers, desktop and mobile

4 mentions on Hacker News by 4 people: 3 stories, 1 comment · first 2024-07-30 · latest 2024-10-11

Mentions per year 4 mentions · busiest year 2024 (4)

20082026

All Stories Comments Newest Top Oldest

PyTorch has a native llm solution It supports all the LLama models. It supports CPU, MPS and CUDA https://github.com/pytorch/torchchat Getting 4.5 tokens a second using 3.1 8B full precision using CPU only on my M1
comment by Ship_Star_1010 · 2024-10-11 · in Run Llama locally with only PyTorch on CPU