lchen001/LLMDrift
View on GitHub ↗ ★ 336 Jupyter Notebook
5 mentions on Hacker News by 4 people: 2 stories, 3 comments · first 2023-07-19 · latest 2023-08-26
20082026
-
…questions from the LLM Drift paper testing to 70% (35/50) vs the March model's 52% (26/50). see: * https://github.com/lchen001/LLMDrift/blob/main/generation/ * https://twitter.com/Si_Boehm/status/1681801371656536068
comment by lhl · 2023-08-26 · in Beating GPT-4 on HumanEval with a fine-tuned CodeLlama-34B -
…you use API versions. One good thing is that the full dataset/methodology was published on Github: https://github.com/lchen001/LLMDrift (everyone should do this!) so it was easy to replicate/validate, but there are some issues: * Simon…
comment by lhl · 2023-07-21 · in GPT-4 is getting worse over time, not better -
▲ 3
LLM Drifts: How Is ChatGPT’s Behavior Changing over Time?story by croes · 2023-07-21 · 0 comments
-
▲ 3
LLM Drifts: How Is ChatGPT’s Behavior Changing over Time?story by vicnov · 2023-07-19 · 2 comments
-
They do explain their methodology in some detail in the accompanying GitHub repo: https://github.com/lchen001/LLMDrift They seem to have been taking to the API directly and requesting the two different model snapshots. I'm not convinced by…
comment by simonw · 2023-07-19 · in GPT-4 is getting worse over time, not better