Hacker GitHub

← All repos

openai/tiktoken

View on GitHub ↗ ★ 19,581 Python

tiktoken is a fast BPE tokeniser for use with OpenAI's models.

28 mentions on Hacker News by 26 people: 1 story, 27 comments · first 2022-12-16 · latest 2026-09-05

Mentions per year 28 mentions · busiest year 2023 (10)
20082026
AllStoriesComments NewestTopOldest
  1. ▲ 153
    Tiktoken: OpenAI’s Tokenizer
    story by azhenley · 2022-12-16 · 74 comments