abacaj/code-eval

Run evaluation on LLMs using human-eval benchmark

4 mentions on Hacker News by 2 people: 0 stories, 4 comments · first 2023-07-07 · latest 2023-08-14

Mentions per year 4 mentions · busiest year 2023 (4)

20082026

All Stories Comments Newest Top Oldest

…3B model so it's the easiest/fastest to use. Perhaps interestingly none of these are based on LLama https://github.com/abacaj/code-eval - This is a large mostly up to date list of benchmarks https://huggingface.co/WizardLM/WizardCoder-15B-V…
comment by thewataccount · 2023-08-14 · in Ask HN: LLM Leaderboard for Code Generation?
You will want to look at HumanEval ( https://github.com/abacaj/code-eval ) and Eval+ ( https://github.com/my-other-github-account/llm-humaneval-ben... ) results for coding. While Llama2 is an improvement over LLaMA v1, it's still nowhere…
comment by lhl · 2023-07-27 · in LLaMA2 Chat 70B outperformed ChatGPT
…has reproduced that announcement as part of code-eval, a repo for getting human-eval results: https://github.com/abacaj/code-eval Those interested in this area may also want to take a look at this repo…
comment by lhl · 2023-07-11 · in Claude 2
…to that's what I'm most interested in, and the latest code assistants are scoring quite well: https://github.com/abacaj/code-eval - a recent replit-3b fine tune the human-eval results for open models (as a point of reference, GPT-3.5 gets…
comment by lhl · 2023-07-07 · in GPT-4 API General Availability