defog-ai/sql-eval
View on GitHub ↗ ★ 743 Jupyter Notebook
Evaluate the accuracy of LLM generated outputs
5 mentions on Hacker News by 5 people: 0 stories, 5 comments · first 2024-02-05 · latest 2024-08-13
20082026
-
…models, check out sqleval [2] 1 https://prollm.toqan.ai/leaderboard/stack-unseen?type=concep... 2 https://github.com/defog-ai/sql-eval
comment by underlines · 2024-08-13 · in Postgres.new: In-browser Postgres with an AI interface -
…Suites, Leaderboards: https://github.com/underlines/awesome-ml/blob/master/llm-too... : - sql-eval: https://github.com/defog-ai/sql-eval : > This repository contains the code that Defog uses for the evaluation of generated SQL. It's based…
comment by westurner · 2024-05-25 · in Show HN: We open sourced our entire text-to-SQL product -
…bit, but I'd be interested in knowing more about its shortcomings when evaluating these systems. https://github.com/defog-ai/sql-eval
comment by HanClinto · 2024-03-22 · in Show HN: DaLMatian – Text2sql that works -
…Mostly thinking out loud :) but maybe ya'll have more ideas. Congrats on the release, OP! [0]: https://github.com/defog-ai/sql-eval
comment by rgbrgb · 2024-02-05 · in Show HN: Natural-SQL-7B, a strong text-to-SQL model -
Is this the data that was used for fine tuning? https://github.com/defog-ai/sql-eval/blob/main/data/question...
comment by bottlepalm · 2024-02-05 · in Show HN: Natural-SQL-7B, a strong text-to-SQL model