Hacker GitHub

← All repos

Shopify/torch-grammar

View on GitHub ↗ ★ 68 Python archived

3 mentions on Hacker News by 2 people: 0 stories, 3 comments · first 2023-07-22 · latest 2023-08-14

Mentions per year 3 mentions · busiest year 2023 (3)
20082026
AllStoriesComments NewestTopOldest
  1. Yes! This is closer to the approach I took in my port of llama.cpp's grammar support to PyTorch: https://github.com/Shopify/torch-grammar/blob/main/torch_gra... ... it generates a tensor mapping each PDA stack to a map of which tokens are…
    comment by burke · 2023-08-14 · in Show HN: LLMs can generate valid JSON 100% of the time
  2. …or https://github.com/mkuchnik/relm or https://github.com/ggerganov/llama.cpp/pull/1773 or https://github.com/Shopify/torch-grammar Overall there are a ton of these logit based guidance systems, the reason they don't get tons of traction…
    comment by BoorishBears · 2023-08-14 · in Show HN: LLMs can generate valid JSON 100% of the time
  3. I implemented this for PyTorch too at https://github.com/Shopify/torch-grammar . I have a hacked version of text-generation-inference that uses it—happy to share that if it’s useful to anyone.
    comment by burke · 2023-07-22 · in Llama: Add grammar-based sampling