Hacker GitHub

← All repos

javirandor/anthropic-tokenizer

View on GitHub ↗ ★ 173 Python

Approximation of the Claude 3 tokenizer by inspecting generation stream

4 mentions on Hacker News by 4 people: 2 stories, 2 comments · first 2024-03-12 · latest 2024-06-18

Mentions per year 4 mentions · busiest year 2024 (4)
20082026
AllStoriesComments NewestTopOldest
  1. Sorta: https://github.com/javirandor/anthropic-tokenizer
    comment by GranPC · 2024-06-18 · in Show HN: Token price calculator for 400+ LLMs
  2. Here you go https://github.com/javirandor/anthropic-tokenizer
    comment by J_Shelby_J · 2024-06-18 · in Show HN: Token price calculator for 400+ LLMs
  3. ▲ 9
    Approximation of the Claude 3 tokenizer by inspecting generation stream
    story by tosh · 2024-03-22 · 1 comment
  4. ▲ 2
    The Worst (But Only) Claude 3 Tokenizer
    story by dpaleka · 2024-03-12 · 0 comments