Hacker GitHub

← All repos

kagisearch/pyllms

View on GitHub ↗ ★ 814 Python archived

Minimal Python library to connect to LLMs (OpenAI, Anthropic, Google, Groq, Reka, Together, AI21, Cohere, Aleph Alpha, HuggingfaceHub), with a built-in model performance benchmark.

11 mentions on Hacker News by 5 people: 2 stories, 9 comments · first 2023-04-11 · latest 2024-09-05

Mentions per year 11 mentions · busiest year 2023 (7)
20082026
AllStoriesComments NewestTopOldest
  1. …library to connect to various Language Models (LLMs) with a built-in model performance benchmark. https://github.com/kagisearch/pyllms
    comment by selcuka · 2024-09-05 · in Kagi Assistant
  2. https://github.com/kagisearch/pyllms/blob/70ec2a4fa4a233ee80...
    comment by SushiHippie · 2024-07-31 · in Kagi LLM Benchmarking Project
  3. …OpenAIProvider('gpt-4o') Median Latency: 3.25 | Aggregated speed: 53.75 | Accuracy: 74.07% | [1] https://github.com/kagisearch/pyllms
    comment by freediver · 2024-07-24 · in Large Enough
  4. …Total Cost: 0.00735 | Aggregated speed: 117.53 tok/sec | Accuracy: 48.15% [1] https://github.com/kagisearch/pyllms
    comment by freediver · 2024-07-18 · in GPT-4o mini: advancing cost-efficient intelligence
  5. …juice was too sweet. Wonder if the synthetic benchmarks are very biased and not that useful. [1] https://github.com/kagisearch/pyllms/blob/6c11cd51b099d33e61...
    comment by freediver · 2023-07-06 · in InternLM – new open source 7B LLM
  6. For anyone reading this, these are the actual prompts being used to assess the models. https://github.com/kagisearch/pyllms/blob/ca9ad4d4bfdd9d58fe...
    comment by sebzim4500 · 2023-05-31 · in Chain-of-Thought Hub: Measuring LLMs' Reasoning Performance
  7. Less scientific, but arguably more practical benchmarks here: https://github.com/kagisearch/pyllms#model-benchmarks
    comment by freediver · 2023-05-30 · in Chain-of-Thought Hub: Measuring LLMs' Reasoning Performance
  8. …Anthropic for this demo, big fans. The entire demo is 20 lines of code, and uses PyLLMs [1] [1] https://github.com/kagisearch/pyllms
    comment by freediver · 2023-04-27 · in FastGPT: A fast, full search, AI answering engine
  9. …(openai, anthropic, AI21..) and as a part of that we designed a LLM benchmark. All open source. [1] https://github.com/kagisearch/pyllms/tree/main#benchmarks
    comment by freediver · 2023-04-11 · in Anthropic’s $5B, 4-year plan to take on OpenAI